NOAA Weather Data

Yeah so I'm a little weird. I get excited when someone says "Do you have access to X data set?" "Can you find data about X?" and other sorts of questions about data. When I see a new data set I have this overwhelming urge to put the data in a database and start running queries. You can shake your head here.

So yesterday when a friend asked me "How much weather data do you have access to?" I was excited. Excited enough to eat lunch at my desk and pound out a weather data load and weather search website.

I started searching for weather data and quickly landed on the NOAA web site. They have made huge strides in the past 5 years (like everyone else) to distribute data to the masses in easily consumable forms. Starting with this site: http://www.ncdc.noaa.gov/ and clicking on Data Access landed me here: http://www.ncdc.noaa.gov/data-access.

My friend wanted temperature high, low and precip amount. Using the Land-Based station section on the NOAA data access page I quickly stumbled on this: http://cdo.ncdc.noaa.gov/qclcd_ascii/

Gamechanger! CSVs. How can you not get excited about CSVs!!!!

So now all we need is a script to grab the zip file, extract the files, clean up the CSV a bit, load the data, and build a web front end to expose the data.

Python Script Extract and Load





Database

I just needed a couple tables and a few stored procedures to expose the data to the web site. <disclaimer soap box>I understand this code will not scale well. I did not want to spend a lot of time modifying the data. Remember this was on my lunch break. Most of the stuff I write is throw-away code or is only used by a handful of people. This is good because I typically do not have to worry about scale, and bad because I am not learning the  techniques of writing code that scales.</disclaimer soap box>


Web Site

Now that we have the data we can build a quick UI using the Foundation Framework. You need to know the weather conditions at Big Bear City Airport in California from 8/15/2014 to 9/21/2014, no fret I got the deets. Holding weather conditions down over here. What What.

Sorry for that last sentence. I was trying to make weather conditions sound hip to the millennial crowd.

Anyway another dataset consumed, another victory lap taken, another celebatory Mt. Dew opened. Its a great time for consumers of data to be alive.