3

As I am a French student in IT I have to work on a Big-Data project using French data (the country, not the language). What I'm looking for is a data set of something interesting like weather or traffic which is large (more than 100 Gb) and frequently updated.

I do not really know where to look at and I will appreciate and considere any suggestion.

Patrick Hoefler
  • 5,790
  • 4
  • 31
  • 47
FlowQ
  • 39
  • 3

5 Answers5

5

I also think that Twitter data is the best way to get >100Gb french dataset.

OpenStreetMaps gives really great open data but you can't reach 100Gb easily : the Integral Metropolitan France dataset is 2.6 GB.

There is no open weather data by MeteoFrance (you have to pay the licence) but there is an Open Meteo Forecast that you can try. I never had the opportunity to use it but I guess it can provide you large amount of data.

You also have the dataset "Tous les documents" from the BNF that is 1.5GB and looks quite interesting.

Hope it can help. Cheers

Terpo
  • 1,157
  • 8
  • 17
  • Too bad MeteoFrance does not have an Open Data project. There is still a lot to do in France ... Twitter data are great but there is a lot of work to identify trustable tweets and interesting ones

    I'll take a look to the BNF one, thanks !

     – FlowQ May 15 '14 at 08:21
  • Yes, clearly, but maybe it's going to move forward after the Trojette Rapport this year. – Terpo May 15 '14 at 08:47
3

You can check the databases of the INSEE (National Institute of Statistics and Economic Studies).

If you are a teacher or student you can use the online statistical data for non-commercial use details over here

3nrique0
  • 151
  • 6
2

There was a similar (French) question recently and I posted several answers. The difference may be that you are interested in France-data and the other question was for French-language data. I think the most appropriate answer was for collecting Twitter data, which I'll describe here with some additions to make it specific for France.

  • Twitter data can easily grow to more than 100 GB, if you let the public live stream run or you search constantly. Once you have your own decent-sized private corpus, you can do all the testing/training while the data collects in the background. You can select French-language tweets by including 'lang:fr' in the search/stream paramters. To include only tweets from France, you can filter by geolocation. Note that only a sub-set of tweets have geolocation activated. Also, note that the language is sometimes incorrectly identified, especially for short, cryptic tweets.

For other France-specific data, consider downloading from the OpenStreetMaps project. Perhaps someone who is more familiar with that project can expand on what data is avaiable that meets your needs.

Community
  • 1
philshem
  • 17,647
  • 7
  • 68
  • 170
  • I read the other question (and answers) before asking and considered twitter data. But the goal of my work is to work with some data provided by a French company, like data.gouv.fr .. As you said I'm looking for 'French Data' The government published interesting datasets but they are too small for this project – FlowQ May 12 '14 at 10:01
2

The US National Intelligence Agency (NGA) maintains a geographic feature database for each country in the world. The data is kept in the Geographic Name Server (GNS). The format is somewhat arcane by today's standards. I have converted each country's dataset into a standardized CSV format to make it more friendly to developers.

The France dataset contains ~98,000 records.

http://www.opengeocode.org/cude1.1/NGA/GNS/FR.zip

Metadata on the vocabulary used for the fields in the dataset is defined here:

http://www.opengeocode.org/cude1.1/LinkedCSV-Vocab.php

Andrew - OpenGeoCode
  • 8,657
  • 17
  • 28
2

Website of the government open data project.

http://www.data.gouv.fr/

Everything you want and more.

lmorin
  • 29
  • 1
  • Yes there is lots of data, but we can't consider any of those sets as a 'Big Data' set, can we ? If you find some large one can you link it here please ? :) – FlowQ May 15 '14 at 08:18
  • @lmorin - can you expand on some specific data sets that would be interesting? – philshem Nov 26 '14 at 10:36