Collections

TODO: Organize these somehow, add one-line blurbs

Organize by usage? (classification, recommendation etc.)

Collections of Collections

ML Data … repository supported by Pascal 2.
DBPedia
UCI Machine Learning Repo
http://mloss.org/community/blog/2008/sep/19/data-sources/
Linked Library Data via CKAN
InfoChimps Free and purchasable datasets
http://www.linkedin.com/groupItem?view=&srchtype=discussedNews&gid=3638279&item=35736572&type=member&trk=EML_anet_ac_pst_ttle LinkedIn discussion of lots of data sets

Categorization Data

20Newsgroups
RCV1 data set
10 years of CLEF Data
http://ece.ut.ac.ir/DBRG/Hamshahri/ (Approximately 160k categorized docs) There is a newer beta verson here:http://ece.ut.ac.ir/DBRG/Hamshahri/ham2/ (Approximately 320k categorized docs)
Lending Club load data https://www.lendingclub.com/info/download-data.action

Recommendation Data

Book usage and recommendation data from the University of Huddersfield
Last.fm - Non-commercial use only
Amazon Product Review Data via Jindal and Liu – Scroll down
GroupLens/MovieLens Movie Review Dataset

Multilingual Data

http://urd.let.rug.nl/tiedeman/OPUS/OpenSubtitles.php - 308,000 subtitle files covering about 18,900 movies in 59 languages (July 2006 numbers). This is a curated collection of subtitles from an aggregation site, [http://www.openSubTitles.org] The original site, OpenSubtitles.org, is up to 1.6m subtitles files.
Statistical Machine Translation - devoted to all things language translation. Includes multilingual corpuses of European and Canadian legal tomes.

Geospatial

Natural Earth Data
Open Street Maps And other crowd-sourced mapping data sites.

Airline

Open Flights - Crowd-sourced database of airlines, flights, airports, times, etc.
Airline on-time information - 1987-2008 - 120m CSV records, 12G uncompressed

General Resources

theinfo
WordNet
Common Crawl - freely available web crawl on EC2

Stuff