Vancouver Park Board Swimming Lessons Screen Scraper

I was waiting for the January 2010 swimming times to appear on the Vancouver Park Board website but I got tired of all the clicking and scrolling required to see when the lessons were available. The other problem was that once the lessons appeared for one pool, some of the other pools still hadn't posted their lesson schedule for January 2010. The times that came out for the first pool were not ideal so I wanted to wait and see what came out for the other pool, while making sure that the first pool didn't book up.

Topic: 

eyefiserver2 - A standalone Eye-Fi Server in python, for linux

I recently forked the defunct eyefiserver project. The new project is called eyefiserver2. It is a server for eye-fi cards that runs on Linux using Python, however, it should be possible to run it on any OS, I just haven't tested it on anything other than Linux.

Topic: 

Low Disk Space Warning Script

Have you ever experienced a full disk on a server or a desktop? Not fun. This script would normally be run as a cron job and would notify you by email if any drive's free disk space has passed below a certain threshold. The code could be better; I wrote this one a long time ago when I was a bit of a n00b and I was in a rush as well. I might make take a look at it again and see if I can make some improvements.

Topic: 

Batch JPEG Photo Renamer

I used this script all the time before Picasa finally added this functionality. This script renames a whole bunch of photos, in a directory for example, appending numbers to the end of a base filename according to the EXIF dates stored inside the JPEGs. For example, a directory full of files that looks like this:

IMG_0123.jpg
IMG_0124.jpg
IMG_0127.jpg
IMG_0128.jpg
...
IMG_0248.jpg
IMG_0250.jpg

could be renamed to this:

Camping Photos_001.jpg
Camping Photos_002.jpg
Camping Photos_003.jpg
Camping Photos_004.jpg
...
Camping Photos_112.jpg
Topic: 

Vancouver Sun Parking Tickets Website Screen Scraper

[img_assist|nid=367|title=|desc=|link=none|align=right|width=150|height=146]
When the Vancouver Sun came out with their Vancouver parking tickets database I immediately had some burning questions, like, did the meter maids work on holidays? Do the work less in the evening than during the day? I found it difficult to answer these questions using their interface, so I decided to screen scrape all 1.6 million parking tickets in to my own MySQL database. This was a bit challenging as they made it difficult to screen scrape the data but eventually it could be done simply by first getting an AppKey, a hidden value inside the HTML source and then doing queries using that AppKey as a parameter. It took about a week to get all 1.6 million tickets downloaded. By using Django, it was easy to get them in to a database and view the results. Initially I just put all the data in to one table, then later I decided to normalize the data a bit which was interesting as I decided to do that in pure SQL which I hadn't done before. I did the scraping itself using a combination of BeautifulSoup, lxml, and mechanize.

vancouver-parking-tickets project at GitHub

MySQL SQL dump (42 MB)

Here's some data:

Topic: 

mp3 Sample Clip Creator With Fade-In/Out

This script will create mp3 sample clips. I created it to automatically create sample mp3 clips for a friend of mine's (Will Stroet) website. You can tweak the length of the clip and the fadeout. Unfortunately it can only create samples that start at the beginning, however, it could be easily modified to start anywhere. It depends on the quelcom library, lame, and mpg123.

Topic: 

Concatenate/Combine PDF Files in Linux

I know there are lots of ways to do this. This is not a HOW-TO, but just sharing a script I made for doing this. It's a decent example of how to write a command-line utility in Python.

Since I write my cover letters and resumes in LaTeX I always need to concatenate the two together before sending it to an employer. This is very important so that the person on the other end prints out both and it makes their life easier by not having to worry about two files. For the longest time I just had a simple script that looked like this:

#!/bin/sh
Tags: 
Topic: 

Drupal Module Updater

This script will automatically update a drupal module if your drupal source code is stored in a Subversion respository. It first removes all files except for the .svn directories, then extracts the tarball for the new version of the module. Then it runs an svn status command to see which files are new, which files have been removed in the new version, and which files have changed.

Topic: 

Pages