Friday, February 27, 2015

LCD Finally

After days of troubleshooting, resoldering, re-coding and such I was finally able to get my LCD to work with the Arduino. I had a few choice words for the LCD right before I took this picture.

I guess the important thing is that I got it to work finally. I found a data sheet on my off brand cheap LCD. Turns out I had the pins reversed so all the hookups were backwards. Once I fixed that boom it worked first time. Amazing. I had spent time resoldering my header pins to the LCD because this was my second solder job. Oh well. It finally works.

If anyone else bought the QC2004 LCD off Amazon see here for the data sheet:

Added a temp sensor and sent the output to LCD with a few more lines of code.

Thursday, February 19, 2015

Missouri School Enrollment Numbers

I somehow stumbled on this page in a search for education data: One click led to another and now I have Missouri Enrollment education data from 1991 - 2015 in a database for consumption. This is more of a CanvasJS chart practice exercise than an actual deep dive into MO Education enrollment trends. However quickly you can see a few interesting things from a surface level view of the data.

Kansas City 33

The Kansas City School district lost accreditation September 20, 2011 effective January 2012. However from the chart below you can see a definite downward trend in enrollment prior to 2011. In November 2007 KC transferred seven schools to the Independence school district. You can see a sizable dip between the 2008 and 2009 school year.

Here are the numbers from Kansas City for 2007 - 2010 showing the large drop in enrollment pretty much across the board.

2008 Recession Theory

Several district show a dip in Kindergarten enrollment between the 2014 and 2015 school year. My first theory is the 2008-2009 recession. People slowed down on family expansion during that year which would start to show up in Kindergarten enrollment numbers. Here is an example from the Liberty School District. It shows a dip starting in 2011 and bottoming out in 2015. The next few years trend will be interesting to see in Liberty as the economy tries to rebound, more houses are being built, and people continue expanding their family.

Largest Jumps

Using some cool SQL Server Windowing Functions I was able to quickly sort out the districts that had the largest enrollment drops and gains in a single year. I filtered out only those schools with at least 100 students in the grade the previous year. Several rural schools have dramatic year to year increases and decreases. The 100 student filter eliminated some of the noise.

Top 20 K Decreases
MEADOW HEIGHTS R-II108361994-72-66.67%
AFFTON 1013691731995-196-53.12%
CHILLICOTHE R-II2941481992-146-49.66%
HICKMAN MILLS C-111125831993-529-47.57%
CENTER 583101751997-135-43.55%
ROCKWOOD R-VI217912511993-928-42.59%
STRAFFORD R-VI113652006-48-42.48%
FORSYTH R-III100591992-41-41.00%
SENECA R-VII122742015-48-39.34%
ARCADIA VALLEY R-II113692000-44-38.94%
SPECL. SCH. DST. ST. LOUIS CO.114722009-42-36.84%
ST. CHARLES CO. R-V131831994-48-36.64%
PATTONVILLE R-III7404701993-270-36.49%
WEBSTER GROVES5463481993-198-36.26%
LICKING R-VIII100642014-36-36.00%

Top 20 K Increases
PATTONVILLE R-III2467401992494200.81%
HICKMAN MILLS C-15631112199254997.51%
CENTER 58162310199614891.36%
KIRKWOOD R-VII289550199426190.31%
ROCKWOOD R-VI115221791992102789.15%
WARRENSBURG R-VI11521119929683.48%
WINDSOR C-112020419928470.00%
UNIVERSITY CITY211341199213061.61%
AFFTON 101246369199412350.00%
WEBSTER GROVES373546199217346.38%
MONETT R-I13619620036044.12%
FREDERICKTOWN R-I10614919954340.57%
MEXICO 5914820820076040.54%
ST. JAMES R-I10113820003736.63%
CARUTHERSVILLE 1810213920063736.27%
AURORA R-VIII12717320104636.22%
KENNETT 3913618519944936.03%

K Findings

  • Looks like the ETHEL HEDGEMAN  Acamedy has shutdown.
  • I was hoping to see more 2015/2014 year decreases to back up my 2008 recession theory.
  • I sent Pattonville R-III a tweet to verify the jump in Enrollment in 1992 is legit. I also asked them what happened to cause a 200% enrollment increase followed by a 36% decrease the following year.

Top 20 1st Decreases

ST. CHARLES CO. R-V142801994-62-43.66%
BROOKFIELD R-III101572015-44-43.56%
MALDEN R-I108712001-37-34.26%
SPECL. SCH. DST. ST. LOUIS CO.107712015-36-33.64%
WARSAW R-IX129862004-43-33.33%
ARCADIA VALLEY R-II107722001-35-32.71%
MOUNTAIN GROVE R-III1631102000-53-32.52%
CARUTHERSVILLE 181611102006-51-31.68%
MALDEN R-I127871994-40-31.50%
DESOTO 732451702001-75-30.61%
WINFIELD R-IV138971994-41-29.71%
STRAFFORD R-VI101712007-30-29.70%
CAMERON R-I135951996-40-29.63%
CHARLESTON R-I115812003-34-29.57%
SPECL. SCH. DST. ST. LOUIS CO.127902009-37-29.13%
SULLIVAN C-21791272000-52-29.05%
ARCADIA VALLEY R-II101721995-29-28.71%

Top 20 1st Grade Increases

LOGAN-ROGERSVILLE R-VIII11617420095850.00%
INDEPENDENCE 308231143200932038.88%
ODESSA R-VII13418420045037.31%
MEXICO 5915020120085134.00%
DUNKLIN R-V11214919923733.04%
RICHMOND R-XVI11214920043733.04%
ST. CLAIR R-XIII13217520054332.58%
MOUNTAIN GROVE R-III12316319994032.52%
ST. CLAIR R-XIII13017220144232.31%
WILLARD R-II23030420047432.17%
WEBB CITY R-VII26935220058330.86%
SMITHVILLE R-II13317420054130.83%
FREDERICKTOWN R-I12416220143830.65%
DESOTO 7317022220025230.59%
HANCOCK PLACE10313420153130.10%
SMITHVILLE R-II12316020033730.08%
KNOB NOSTER R-VIII12616320093729.37%
PERRY CO. 3212215719963528.69%

First Grade Findings
  • DESOTO 73 shows up in both largest increase and largest decrease lists. 2001 saw a 30.61% decrease in 2001 followed by a 30.59% increase in 2002. Desoto is a kinda small (population 6400 during 2010 census). I searched for a few minutes about what happened in Desoto MO in 2001 with 0 results. 
  • ST. LOUIS LANGUAGE IMMERSION school had a big boost. This is a new charter school in the St. Louis area. Given the state of large metro schools in KC and St. Louis, no surprises here people wanting to try out alternatives.
  • INDEPENDENCE 30 shows up on the biggest increase list in 2009. This was after the previously  mentioned building acquisition from the KC school district.

So I only showed the full results for K and 1st Grade cause I started getting bored with that. I did however want to show the SQL Server Windowing functions I used to get the data. Those are pretty cool.

One last cool thing

While looking at the 5th grade trends (as mentioned I started getting bored with the year to year) I did notice something in the Clinton district. In 1994 Clinton had a 32% decrease in 5th grade enrollment. I wanted to see if you could see that trend before and after the drop in 5th grade. Using the graph I compared all the grades to 5th grade for Clinton. You can see the previous years drop in enrollment leads to the 5th grade year drop as well as the subsequent years drop in enrollment. It was like a ripple effect, cool to see visually.

K vs 5th

1st vs 5th

2nd vs 5th

3rd vs 5th

4th vs 5th

5th vs 6th

5th vs 7th

5th vs 8th

5th vs 9th

5th vs 10th

5th vs 11th

5th vs 12th

Anyway good times. This was just one of the plethora of data sets on the MO DOE web site. Too much data, too little time.

Friday, February 13, 2015

Google Spreadsheet Python

Just found an awesome python module this morning called gspread. The module allows you to work with Google Spreadsheets. You can read data, write data, work with worksheets, and all kinds of other functions. You can find out more about the module on GitHub

I went from install, to sample code, to hacking spreadsheets in about 35 seconds. It was so easy.

Easy_Install the Module

  1. Open up a console and browse to my Python\scripts folder
  2. Run easy_install (setup instructions for easy_Install) like so: easy_install gspread
  3. Easy Install does all the heavy lifting.

Create a Spreadsheet

Create a spreadsheet in Google Drive and call it whatever you want. Remember the name because you will need it for your sample script.

Sample Script

You will see some sample scripts on the GitHub project page. Here is my script.

It really is that simple. You just setup your username and password. The module will then will convert that to an Oauth session for you to authenticate to Google securely.  We open the spreadsheet with the"Pythontest").sheet1 command.

You can see the update command is commented out: wks.update_acell('B2',"some data here"). You can fetch a range of cells and print out the values. You can retrieve whole columns or rows.

This module is awesome and so easy to use.

Monday, February 9, 2015

Starburst Superfruit Flavor Review

Co-worker just brought in some Starburst Superfruit chews to sample. Here is how they stack up.

  • Strawberry Starfruit: Fortunately Wrigley did not jack with the Strawberry flavor we all know and love too much. This pretty much tastes like a strawberry Starburst with a hint of something weird. I'd assume the weird part is the star fruit part. Anyway, they didn't hose it up too much and this is by far the best flavor in the pack. 8/10
  • Blueberry Acai: I don't even know what an Acai is. At first bite this flavor was third in the standings. However it finished strong at the end and powered ahead of the others. Pretty tasty representation of blueberry, and I'd have to let some hipster weigh in on the Acai part cause I have no clue. 5/10
  • Raspberry Pomegranate: This flavor tasted like watermelon bubble gum at first until the pomegranate taste overpowered the flavor near the middle of the tasting. This was a very close race between the Blueberry flavor but again the Blueberry flavor finished strong. 4.5/10
  • Passion fruit punch: I knew I would hate this flavor before trying it. Any candy with the word fruit punch in the title is terrible. Hawaiian C in the liquid form is terrible enough. How can you make it worse? Put it in a chewable format, of course. This one is terrible and should be thrown out like all Orange Flavored regular Starbursts. 1/10

Here is a picture with the flavors ranked in order from best (top) to throw it out (bottom).

Forecast Tracker

Complex Forecast. New data is in. Not really sure what is going to happen. We'll have to wait and see what the weekend holds. Could be anything from a dusting to 6" of snow.

These are all quotes I have heard while watching our local Kansas City news. Let's face it. Weather teams have the daunting task of predicting the future and are crucified when they get it wrong. Don't feel too bad for them though, they are typically paid pretty well for being wrong a lot. It is a very tough job and they do get it right more often than not. Weather teams in Kansas City seem to have an even tougher job given the setup of Kansas City weather.

The Plan

Measure the accuracy of the local weather stations forecasts to see who is the most accurate. I wanted the process to be hands off, completely automated. I didn't want to enter the data manually etc. I was hoping the local weather stations had some kind of parsable forecast feed on their websites I could use for the data source.

I then decided to compare the forecasts with the observed temperature at the Kansas City International airport. For some reason I decided the time to compare the temps would be 12:00PM.

The Method

I started reviewing the Hourly Forecast for each local weather website. Three of the four weather stations in town had the data in a text format I could scrape from their site. The fourth weather website had the hourly forecast in an image format. I am not skilled enough to figure out how to work with an image so I just through Fox4 out of the running.

I used python to do the heavy lifting. Here are some sample scripts for each site.




I then used Weather Underground to get the temp at 12:00PM at KCI. The Weather Underground temperature is collected at 11:53AM. For my purposes here that is close enough. I could not find on the various weather team websites what location their hourly forecast is predicting. Again this is not an exact calculation of accuracy here...just a general ballpark. If your 12:00PM hourly forecast is off by 8 degrees at KCI, odds are the forecast was incorrect at other spots in the metro.


You can tell by the scripts above that this system is pretty fragile. Any small change in website design, CSS changes, object name changes, etc. will throw this off. I am hoping the thing at least works for a few weeks so I can collect some data.

The python script executes at 11:58AM in order to ensure that the 11:53 Wunderground temperature collection is complete, and before the weather websites remove the 12:00PM hourly forecast from their web sites.

Then at 12:15PM I have a job that executes to measure the accuracy between the forecasted temperature and observed temperature. The results are stored in a table.


I then front-ended the data with a little website used to display the results. Keep in mind that I only have 3 days of data so far, so this is a very limited sample size. Again, I hope the system can work for a few weeks before it completely blows up.

Three Day Desk Temp Study

The office at my current gig runs a little warm. This is not me complaining. I like wearing short sleeves in the dead of winter and it can always be worse. I would much rather the office run hot than cold. Cold is terrible and miserable. I brought my trusty Acurite Digital Thermometer to work a while back during my Fridge Cooling Zones project, so I thought why not track the temperature at my desk for a few days.

Temperature Collection Technique

A while back I had created a PowerShell Script Reminder System. You set the number of minutes you want to delay the timer and run the script. After the amount of time you entered elapsed, a pop up box would display in the center of your screen. Low tech I know but it seemed to work well.

Here is an example of a 15 minute timer.
param([string]$time = "00:15:00")
$timeReformat = $time -replace ",","."
$seconds = ([TimeSpan]::Parse($timeReformat)).TotalSeconds

Start-Sleep -s $seconds
Add-Type -AssemblyName System.Windows.Forms

$Form = New-Object system.Windows.Forms.Form
$Font = New-Object System.Drawing.Font("Calibri",24)
$Form.Font = $Font
$Label = New-Object System.Windows.Forms.Label
$Label.AutoSize = $True
$Form.Width = 400
$Form.Height = 200
$Form.MinimizeBox = $False
$Form.AutoSize = $True
$Form.MaximizeBox = $False
$Form.StartPosition = "CenterScreen"
$Form.BackColor = "White"
$Form.ForeColor = "Blue"
$Form.TopMost = "True"

Here is the pop up message.

As I  mentioned pretty low tech. A few problems with this.

  1. I have to manually run the timer over and over again
  2. I hardcoded the time instead of passing that into the script.
  3. Sometimes I would forget to restart the timer after 15 minutes
I may tweak that script later.

So anyway, I would run the powershell script and then start working on something else. After 15 minutes I would get that fancy pop up box. I would then hop into a Notepad++ document where I would record the date/time (using a shortcut mapped to Crtl+F5), the temperature, and the humidity. 

I would then fire up the PowerShell Script for another 15 minute interval. Again several problems with this low-tech solution. The main problem is that I would forget to restart the PowerShell Script after recording the time.

Day 1 

Date: 2/3/2015
KC High Temp: 46
Recorded Office High: 79
Humidity: 16%

Raw data

2/3/2015 7:3475
2/3/2015 7:5375
2/3/2015 8:1177
2/3/2015 8:2677
2/3/2015 8:4277
2/3/2015 8:5777
2/3/2015 9:1377
2/3/2015 9:2879
2/3/2015 9:4477
2/3/2015 10:0779
2/3/2015 10:2279
2/3/2015 10:3879
2/3/2015 10:5379
2/3/2015 11:0879
2/3/2015 11:4877
2/3/2015 12:2175
2/3/2015 13:0175
2/3/2015 13:1775
2/3/2015 13:3277
2/3/2015 13:5277
2/3/2015 14:0777
2/3/2015 14:2277
2/3/2015 14:3877
2/3/2015 14:5477
2/3/2015 15:0977
2/3/2015 15:2577
2/3/2015 15:4077


There was a huge dive near lunch. You could actually feel a cool breeze running through the office area. It was as if someone enabled some kind of AC cooling unit to cool the place down. Humidity remained constant at  16% for all recorded observations. Thus every time I go to grab my coat I get shocked.

Day 2

Date: 2/5/2015
KC High Temp: 28
Recorded Office High: 79
Humidity: 16%

Raw Data

2/5/2015 7:3373
2/5/2015 7:5073
2/5/2015 8:0673
2/5/2015 8:2175
2/5/2015 8:3675
2/5/2015 8:5275
2/5/2015 9:0875
2/5/2015 9:2377
2/5/2015 9:3977
2/5/2015 9:5277
2/5/2015 10:0777
2/5/2015 10:2477
2/5/2015 10:4477
2/5/2015 10:5977
2/5/2015 11:2477
2/5/2015 11:4077
2/5/2015 12:1277
2/5/2015 12:2777
2/5/2015 12:4477
2/5/2015 13:0777
2/5/2015 13:2177
2/5/2015 13:2377
2/5/2015 13:3877
2/5/2015 14:0177
2/5/2015 14:1977
2/5/2015 14:3477
2/5/2015 14:5379
2/5/2015 15:0879
2/5/2015 15:2479
2/5/2015 15:3479
2/5/2015 15:5679


I was a little more with it on this day in the temp collection. I collected more samples than day 1.  Day 2 was a more typical trend in temperatures in the office. A gradual incline is the norm and is shown here in day 2.

Day 3

Date: 2/6/2015
KC High Temp: 54
Recorded Office High: 79
Humidity: 16%


2/6/2015 6:5273
2/6/2015 7:0575
2/6/2015 7:2175
2/6/2015 7:3675
2/6/2015 7:5977
2/6/2015 8:1777
2/6/2015 8:3377
2/6/2015 8:4777
2/6/2015 9:1477
2/6/2015 9:3777
2/6/2015 9:5779
2/6/2015 10:0179
2/6/2015 10:1779
2/6/2015 12:0177
2/6/2015 12:1677
2/6/2015 12:3177
2/6/2015 12:4777
2/6/2015 13:0377
2/6/2015 13:2877
2/6/2015 13:5477
2/6/2015 14:1177
2/6/2015 14:3777
2/6/2015 14:5377
2/6/2015 15:1177
2/6/2015 15:2677
2/6/2015 15:3877
2/6/2015 15:5379


Interesting dip in temperature on day 3. We reached the high temp right before 10:00AM then had a dip around noon. Stayed consistent throughout the day and we had a last minute spike to 79 right before I left the office for the day.


  • It is warm in the office
  • We reach the high temperature early and typically stay there
  • Sometimes we have a random cooling event that will drop the temperature down. Hard to determine the pattern at this point with the small sample size.
  • The office is pretty dry. 16% humidity was the constant throughout the test. I did see the humidity as high as 20% from time to time.

Next steps

I am planning to build a Arduino temperature recorder to do some longer term automatic trending of temperature. I will base it on a temperature collector I built a while ago. Of course I will post the details on the blog should I actually follow through with the plan.