19

I want to get historical weather data (Winter 2014) of temperature, humidity, air-pressure, wind_speed, wind direction, rain in specific latitude/longitude.

Is there any API that I can use to get these informations.

Patrick Hoefler
  • 5,790
  • 4
  • 31
  • 47
user3001937
  • 403
  • 1
  • 3
  • 5
  • Please take at a look at answers here http://opendata.stackexchange.com/a/1620/1511 and here http://opendata.stackexchange.com/q/2009/1511. I personally recommend the Wunderground API - https://gist.github.com/philshem/8864437#file-wunderground_historical-py – philshem Dec 30 '14 at 07:50
  • 1
    Thanks! @philshem can u write it as an answer to post it as a valid one. – user3001937 Dec 30 '14 at 14:53
  • Depending on your needs, existing city level data might satisfy. In spatial statistics there is a concept of "given observations at certain locations, weather stations for example, estimate the weather at other locations (such as a grid that can be used for contour plots). That is called "change of support" and is a very useful data manipulation tool. – respectPotentialEnergy Jan 02 '15 at 13:49
  • Check also this answer here for how to use Google BigQuery to get this data: http://opendata.stackexchange.com/a/7583/11083 – Praxiteles Apr 20 '16 at 23:00

4 Answers4

13

For international and historical data, and for a modest number of requests per day, I personally recommend the Wunderground API. Once you register, you can get 500 free requests per day.

The URL for historical data will look like this:

http://api.wunderground.com/api/Your_Key/history_YYYYMMDD/q/CA/San_Francisco.json

I've posted a sample code (python 2.7) that you can use (and improve!) - LINK. I would run this code every day, just changing the year (currently it's set for 2013). The output of the code is a CSV file, but you can store the JSON and/or parse as needed.

philshem
  • 17,647
  • 7
  • 68
  • 170
  • Is there any way to specify lat/long to get that data ? – user3001937 Dec 31 '14 at 14:45
  • 1
    This endpoint gives you the closest city to a latitude/longitude - http://www.wunderground.com/weather/api/d/docs?d=data/geolookup – philshem Jan 01 '15 at 18:11
  • Anyone know how to use the API to get the weather at a specific historical time? – Guillochon Jan 02 '17 at 01:31
  • 1
    Wunderground API is now only available if you register a weather station to your account on the site apparently. That's the message I have when going into the API Key section in my profile: "No API key. You must own a Personal Weather Station in order to generate an API key." – Link14 Jul 01 '20 at 01:56
5

I have written some sample code for directly building a CSV starting with a given date and ending with a given date: https://github.com/joshmalina/pollution/blob/master/notebooks/Build_historical_weather_data.ipynb

The city is currently set to Beijing, but you can change that easily. The data will also be cleaned of null values.

compguy24
  • 246
  • 2
  • 4
3

For Canada, you can download historical data by city in bulk csv or xml files from Environment & Climate Change Canada.

The example provided here uses wget to download all available hourly data for Yellowknife A, from 1998 to 2008, in .csv format

for year in `seq 1998 2008`;
    do for month in `seq 1 12`;
    do wget --content-disposition "http://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=1706&Year=${year}&Month=${month}&Day=14&timeframe=1&submit= Download+Data" ;
    done;
done

WHERE;
• year = change values in command line (seq 1998 2008)
• month = change values in command line (seq 1 12)
• format= [csv|xml]: the format output
• timeframe = 1: for hourly data
• timeframe = 2: for daily data
• timeframe = 3 for monthly data
• Day: the value of the "day" variable is not used and can be an arbitrary value
• For another station, change the value of the variable stationID
• For the data in XML format, change the value of the variable format to xml in the URL.

You can grab a list of stations from this csv or search for a station

Community
  • 1
raphael
  • 227
  • 1
  • 9
0

If you solve ML task and want to try weather historical data as features, I recommend you to try python library upgini for smart enrichment. It contains 12 years history weather data by 68 countries.

My code of usage is following:

%pip install -Uq upgini
from upgini import SearchKey, FeaturesEnricher
from upgini.metadata import CVType, RuntimeParameters

define search keys


search_keys = {
    "Date": SearchKey.DATE, 
    "country": SearchKey.COUNTRY,
    "postal_code": SearchKey.POSTAL_CODE
}


define X_train / y_train


X_train=df_prices.drop(columns=['Target'])
y_train = df_prices.Target


define Features Enricher


features_enricher = FeaturesEnricher(
search_keys = search_keys,
cv = CVType.time_series
)


X_enriched=features_enricher.fit_transform(X_train, y_train, calculate_metrics=True)

As a result you will get dataframe with new features with non-zero feature importance on your target variable, such as temperature, wind speed etc

Web: https://upgini.com GitHub: https://github.com/upgini

aftermidnight
  • 32
  • 3