Commentful and cocomment.com Review

Since co.mments.com shut down I've had to find a new site to track comments to blog article that I have commented on. It turns out that the two alternatives both suck, for different reasons.

Cocomment requires a firefox add-on that is fairly broken. It wants to track every comment page and the only way to not have it track a page is to right click on the cocomment icon in Firefox and disable it, then refresh the page. Annoying. The other problem is that it doesn't work on Drupal sites. When you try to submit a comment on a drupal site it will not work and you will be taken back to the submit comment page. Luckily Firefox won't clear the comment text box so your comment won't disappear. The only solution is to disable Cocomment on that page and refresh. The bigger downside is that my comment won't be tracked. The only upside to Cocomment is that the RSS feed does work. I don't particularly like the web interface though, it's too messy and overly complicated with little icons whose use I always have trouble remembering.

Commentful is nicer and works more like co.mments.com. You're essentially just submitting a URL to a web service that will then notify you when more comments are added. The problem is that the RSS notification feed doesn't work. Not only that, but when you go to your watchlist, you have to click on the little "Check" icon for each item in your watchlist. That's right, Commentful doesn't even check the websites for new comments unless you ask it to. I suspect they didn't have enough resources to be checking remote sites every 15 minutes so they disabled that altogether.

I'm really tempted to write a Django application to do this. The only challenging part is determining how many comments there are in a blog article, and being able to do that across many different blogging platforms (wordpress, blogger, drupal, custom sites, etc...)

Gentoo gcc upgrade...again?

I just upgraded my mythtv backend's gcc from 3.4.6 to 4.1.2. It's still compiling all of world now and will be have been about 24 hours when it's all done. Then I did an emerge --sync and now there is a gcc 4.3! Arghh.

I've always touted gentoo's advantage of allowed me to run a mixed system. For a long time I was running an old version of mysql, and old kernel, and old glibc, and finally an old gcc. I would pull the odd update of certain packages if I felt like it. But the shear magnitude of the output when I would run emerge -uva world got on my nerves. I eventually caved and decided to go back to keeping an updated system. I had originally stopped updating most packages because I wanted my system to be as stable as possible (because it my MythTV machine and it records TV shows for me). At that time I was using it as a backend and as a frontend, but now I'm just using it as a backend which is a bit simpler (less room for breakage) and I'm a bit more careful now when doing upgrades. If any important packages are being updated I make sure to update them then reboot to make sure my system still boots. I also try not to fiddle with anything else. Anyways, in the case of my MythTV box I'm not longer taking advantage of Gentoo's biggest advantage, running system with mixed packages.

Tags: 

Vancouver Needs Real Street Food

Travel to many cities in the world and you will find street food all over the place. Delicious street food. Walk in Vancouver and there is almost no street food. I can only think of 2 hot dog vendors on West 4th or West Broadway west of Granville St. The one near Future Shop just west of Burrard and the one at the corner of Graville and Broadway in front of the Chapters. You can walk a long time downtown before stumbling across any street food, and it's mostly just hot dogs or Crepes. Toronto just announced some new street food offerings downtown. I really hope it is a hit, that the vendors make back their investment and more and I hope that no ones gets sick from the food. Then, this idea just might make it to Vancouver. How awesome would it be to be able to eat some Pad Thai on the street? Or some jerk chicken? or a souvlaki skewer? This is one area of our society where I think we need less regulation.
[img_assist|nid=341|title=Eating street pineapple in Seoul, Korea|desc=|link=none|align=left|width=200|height=150]
[img_assist|nid=343|title=Eating street food in Peru|desc=|link=none|align=left|width=200|height=150]

Tags: 

Django Applications at Work

Today someone at my work just released a simple Django app, the second one now running at my company. Here are the two Django apps running so far:

Server scanner

This is the first Django application that we use. It started out as a script that would scan all the computers on our network and determine what version of our software they were running by inspecting a special file in one of the publicly available SMB shares. The script took about half an hour to run and generated static html that was then served up by Apache. This became cumbersome for a number of reasons. For one, it took about an hour to scan the entire network. There was no persistence whatsoever so my script had no memory of which servers had our software installed on it. These servers are quick to check. The ones that are slow to check are the ones that are actually no longer on the network, or do not have the publicly available share that we are expecting. When the project manager emailed me one day and said "the servers on this page are often out of date" I sprung in to action and converted it to a Django application. There is a script that scans the network and saves Server instances to an sqlite db. I also used a couple generic views to display a list of servers and to edit a server (to provide a description). Converting to Django from my static html generator gave me a few new features right away: 1) full scans were much quicker because I could scan the servers that actually have the software installed with a higher priority. In another thread, I can look for new servers on the network with a lower priority. There's also the "timesince" filter which is awesome, and the automatically filled DateTime fields for "created" and "updated" when I update a model instance. Not to mention ModelForms and generic views which allowed me to get this up and running quickly. The best part is that each server entry has a description entry, where people can put a note like "DO NOT TOUCH THIS SERVER" or "Dave's server, please feel free to play around on it".

Task tracker

This web app is for the software developers to keep track and inform other of what they are working on. Up until we haven't really had anything of the kind, save for e-mail and a bug tracker. Typically software developers send weekly reports to team members. This involves typing one up or constantly opening up an email draft. Either way, other team members can't see what the person is working on in real-time. So one of the members of our Python Club got interested in Django and wrote this up very quickly. He used YUI which is awesome and he just released it to a wider audience today.

Build website - in progress

This is a new one that is in progress. Currently, each build generates a folder structure containing the result of the build along with a bunch of log files and a bunch of other junk. A perl script scans these folders periodically and generates static HTML (at least I think that's how it works, I don't work on it so I don't know the details). My boss is in the process of creating some Django models. The main table is a table called Build. Then his HTML will be generated from Django templates. That is the biggest advantage of moving to Django here, in my opinion, is that he can use templates rather than HTML generation by a perl script. Having the information in a database is nice but not required. He will get some advantages from having the database though. He'll have some other tables, such as BuildServer, Release, and Product that the Build server will contain ForeignKeys to to allow filtering the builds based on these ForeignKeys. Also the build manager (someone other than my boss who is designing the Django part) will be able to use the admin interface to add new Releases, BuildServers, etc...

Conclusion

This apps were implemented extremely quickly and are far better than all the crappy SAP and SharePoint-based web applications used at work and the other simpler web pages around that I think are mostly static HTML or ASP. It will be interesting to see if it catches on. I had this master to plan to re-implement our entire employee performance review site (which is a particularly crappy SAP app) in Django but there are too many hurdles. We'd need to hook in to the existing SAP user tables somehow and also use the existing authentication mechanisms. But a demonstration app using the basic functionality wouldn't be that hard, and then we could worry about the hard bits later after it got approval. The existing employee performance review site is REALLY bad.

Anyways, I just thought it was cool how quickly Django has been adopted for a few simple projects at a company that is for the most part a Microsoft shop. People do use OSS here but there is a bit of a tendency to use Microsoft solutions and a bit of Not-Invented-Here syndrome as well.

Drupal Audio Handling is a Mess

The state of Drupal audio files support in Drupal is a mess right now. The audio module, which is great by the way, has not been fully updated to 6.x. This effort has probably been hampered somewhat by the fact that "audio nodes" is not the way that most drupalers want to treat audio going forward. The new drupal way is CCK module and Views and the new way to do audio in drupal is by creating a CCK type with a File field. The FileField module is mostly ready for 6.x (but still "alpha"), but one other nice addition, XSPF playlist module, which I need, is not even close. I recently tried to update a site I maintain, willmusic.ca, over to Drupal 6.x. Trying to get an album type and then generating an XSPF feed from the associated file fields was next to impossible without actually porting the XSPF module to Drupal 6.x myself. When I tried to use the audio module in 6.x to get an XSPF feed for an album type with attached audio files (using audio_attach), this also failed, because the audio_feeds module has not been ported to 6.x yet. So in the end, I reverted back to the original 5.x site, and got XSPF feeds working for each album and then hooked up a flash media player that accepts an XSPF playlist. All is good except I wish I was using 6.x. The moral of the story is that even though 6.0 was released over a year ago, 6.x is no where near production ready, unless you have time to develop your own modules, or you don't plan on using any modules at all.

Update (2010-02-03): the SWF Tools module looks promising, especially Flow Player which it integrates with. I starting to port the site over to Drupal 6 and I think CCK is the way to go, so I will go with CCK FileField with SWF Tools.

Tags: 

Don't Rewrite Working Black-box Code

At work I am modifying an existing tool to work from the command line instead of a GUI. Currently everything is a bit coupled to the GUI. On Friday, the next problem I encounted was a global variable in Common.py that was not initialized.

""" Common.py """
def initHSCM():
    global hSCManager
    ...
    hSCManager = win32serviceOpenSCManager(None, None, win32con.SERVICE_ALL_ACCESS)
    ...
 
...
def startService(service):
    """ function that uses hSCManager """
    # These functions don't work when hSCManager is set to None
 
def stopService(service):
    """ function that uses hSCManager """
    # These functions don't work when hSCManager is set to None
...

The only place it is getting initialized is when iniHSCM() is called from GUI.py.

""" GUI.py """
Common.initHSCM()
...
startService("blah")
...
stopService("blah")

My new CLI.py does not uses Common.py as well as the other underlying libraries. Instead of just calling initHSCM() inside CLI.py so that calls to Common.py, I decided to rewrite all the functions that use the hSCManager global variable (ie. startService, stopService, and many others). My plan was to rewrite them to not depend on this global variable, and make them a bit cleaner. Instead they would get an hSCManager handle and close it at the end. So service manager objects would be shorter-lived things. Like most of the code in this project, it's a rat's nest and was written by people who knew C and Java better than they knew python. So part of my motivation for re-writing was to clean things up a bit. After I had re-writen a few of the functions, I realized that I had unknowingly creating new bugs (in the functions I was writing). Even though I was writing some unit tests (which was making the development process take longer on the whole) I knew that the code I was writing was going to have more bugs in it than the code that had essentially been working and stable for years (albeit ugly). The last thing I need is bugs in this code. When bugs happen I want to know that they are most likely in my new code.

In the end, I gave up and this is what I did:

""" CLI.py """
Common.initHSCM() # easy hack
...
startService("blah")
...
stopService("blah")

An even easier way is to move responsibility for initializing hSCManager into Common.py. Something like this?

""" Common.py """
hSCManager = win32serviceOpenSCManager(None, None, win32con.SERVICE_ALL_ACCESS)
 
...
def startService(service):
    """ function that uses hSCManager """
    # These functions don't work when hSCManager is set to None
 
def stopService(service):
    """ function that uses hSCManager """
    # These functions don't work when hSCManager is set to None
...

Hmm, that was easy. Moral of the story: don't re-write code that is several years old and works unless you really, really have to. Who cares if it's ugly and a hack. You didn't write it.

Tags: 

Ticket to Ride

[img_assist|nid=333|title=Ticket to Ride: Europe|desc=|link=none|align=left|width=256|height=192]
I recently purchased Ticket to Ride: Europe, an excellent board game by Days of Wonder. I've played it four times with 4 players, and twice with 2 players and I'm still not tired of it. The best part about this game is how simple it is to play and that games don't take very long to play either. I was looking for an alternative for Akham Horror, a game with tons of rules that requires at least 3 hours to play. Ticket to Ride is simple, as there are only a few basic concepts that need to be understood. We even played with my in-laws (who haven't played many board games before) and one of them almost won. The other didn't complete any of their Destination Tickets but they were still able to play and make routes along the way. I think he has figured it out now and if he plays a second time he will know what he is going. He actually ended up breaking some of the longer routes thus preventing my wife and I from getting the extra 10 points for the longest route. The other great thing about this game is that is plays well with 2 players. I highly recommend this game for anyone.

Weight Gain During Pregnancy Progress Tracker Spreadsheet

When woman get pregnant they are supposed to gain weight. Not gaining enough weight can cause problems just as gaining too much weight can cause problems. But when you are 4 months pregnant or when you are 7 months pregnant it is hard to know how much you should weigh at that point in your pregnancy. I made a spreadsheet to help my wife know at any point of her pregnancy whether she is underweight or overweight relative to the end target.

The spreadsheets are available below in Excel and OpenOffice format. I assumed a weight gain of 4-6 lbs in the first trimester followed by steady weight gain all the way until the end of the pregnancy. The defaults are a minimum gain of 25 lbs, a target gain of 27.5 lbs, and a maximum gain of 30 lbs. The minimum and maximum bands are visible on the chart and the goal is to stay within those bands. You only need to enter your starting weight, your date of last menstruation and the low, medium, and high targets on the "Input" sheet and enter your weight weekly (you can skip weeks too and it will still work) on the "Graph" sheet. The defaults are for someone with average BMI.

Supposedly in the U.S., woman with normal BMIs are advised to gain 25-35 lbs whereas in Canada they are advised to gain 25-30 lbs (see "CBC: Pregnancy weight gain guidelines may be too high"). If you have a high BMI you should gain a less weight and if you have a low BMI you should gain a bit more. There are lots of other websites or books with more information on this.

Update: Check out this update from Todd: http://www.epiphanyofthefox.com/private/new_pregnancy_weight_gain.xlsm.

Tags: 

Using XML for Code Documentation is Just Plain Wrong

I was just looking at some C# code at work today and it had XML Documentation (like javadoc or python docstrings, only with XML). Who was the idiot that came up with that idea? It's the most insane thing I've ever seen. Let's look at the predecessors to C#'s XML documentation:

Javadoc:

/**
 * Returns an Image object that can then be painted on the screen. 
 * The url argument must specify an absolute {@link URL}. The name
 * argument is a specifier that is relative to the url argument. 
 * <p>
 * This method always returns immediately, whether or not the 
 * image exists. When this applet attempts to draw the image on
 * the screen, the data will be loaded. The graphics primitives 
 * that draw the image will incrementally paint on the screen. 
 *
 * @param  url  an absolute URL giving the base location of the image
 * @param  name the location of the image, relative to the url argument
 * @return      the image at the specified URL
 * @see         Image
 */
 public Image getImage(URL url, String name) {
	try {
	    return getImage(new URL(url, name));
	} catch (MalformedURLException e) {
	    return null;
	}
 }

Then, doxygen, which looks a lot like javadoc:

      /**
       * a normal member taking two arguments and returning an integer value.
       * @param a an integer argument.
       * @param s a constant character pointer.
       * @see Test()
       * @see ~Test()
       * @see testMeToo()
       * @see publicVar()
       * @return The test results
       */
       int testMe(int a,const char *s);

Unfortunately Genshi doesn't syntax highlight the javadoc comments. But it looks fairly readable. Let's try a python docstring example. There is no one standard. One of the documentation generators for Python, Epydoc understands plaintext, javadoc, epydoc, and reStructuredText.

Python code with epydoc style docstrings:

def x_intercept(m, b):
    """
    Return the x intercept of the line M{y=m*x+b}.  The X{x intercept}
    of a line is the point at which it crosses the x axis (M{y=0}).
 
    This function can be used in conjuction with L{z_transform} to
    find an arbitrary function's zeros.
 
    @type  m: number
    @param m: The slope of the line.
    @type  b: number
    @param b: The y intercept of the line.  The X{y intercept} of a
              line is the point at which it crosses the y axis (M{x=0}).
    @rtype:   number
    @return:  the x intercept of the line M{y=m*x+b}.
    """
    return -b/m

Python code with one example of reStructuredText docstrings (this one includes the types of the parameters but they aren't necessary):

def fox_speed(size, weight, age):
    """
    Return the maximum speed for a fox.
 
    :Parameters:
      size
        The size of the fox (in meters)
      weight : float
        The weight of the fox (in stones)
      age : int
        The age of the fox (in years)
    """
    #[...]

I couldn't find any nice examples for C# XML Documentation. The C# XML Documentation Tutorial has some examples, but conveniently, none that include all the tags that I would need to replicate the javadoc example I showed above. So I'll convert the Java example to C#:

   /// <summary>
   /// Returns an Image object that can then be painted on the screen. 
   /// The url argument must specify an absolute {@link URL}. The name
   /// argument is a specifier that is relative to the url argument. 
   /// 
   /// This method always returns immediately, whether or not the 
   /// image exists. When this applet attempts to draw the image on
   /// the screen, the data will be loaded. The graphics primitives 
   /// that draw the image will incrementally paint on the screen.</summary>
   /// 
   /// <param name="url">an absolute URL giving the base location of the image</param>
   /// <param name="name">the location of the image, relative to the url argument</param>
   /// <returns>
   /// the image at the specified URL</returns>
   /// <seealso cref="Image">
   /// Read more about the Image class</seealso>
 */
 public Image getImage(URL url, String name) {
	try {
	    return getImage(new URL(url, name));
	} catch (MalformedURLException e) {
	    return null;
	}
 }

I followed Microsoft's convention (because they know best) of putting the opening tags on a line on their own.

The javadoc sucks because you have to put a <p> (or <br />?) to make a new line which is stupid. Otherwise it's pretty readable, and same goes for doxygen. Especially the @param and @return tags. The Epydoc-style python docstrings suck. You have to specify the type using a @type tag and the return type using an @rtype tag. The reStructuredText example looks the best to me. No tags at all, except for the :Parameters: heading which should be there anyways. The C# comments are an eyesore. Even if Visual Studio had syntax highlighting for the comments it would suck. Did Microsoft look at the two major previous implementations (doxygen and javadoc) and decide that XML was a better way to document code?

I recently saw an interesting comment in scipy's source about one of scipy's guiding principles in designing the docstring standard for their codebase:

A guiding principle is that human readers of the text are given precedence over contorting docstrings so our tools produce nice output. Rather than sacrificing the readability of the docstrings, we have written pre-processors to assist tools like epydoc_ and sphinx_ in their task.

Microsoft clearly took the opposite route and decided to make code documentation readability by human readers a low priority.

Drupal Module Updater

This script will automatically update a drupal module if your drupal source code is stored in a Subversion respository. It first removes all files except for the .svn directories, then extracts the tarball for the new version of the module. Then it runs an svn status command to see which files are new, which files have been removed in the new version, and which files have changed.

Topic: 

Batch JPEG Photo Renamer

I used this script all the time before Picasa finally added this functionality. This script renames a whole bunch of photos, in a directory for example, appending numbers to the end of a base filename according to the EXIF dates stored inside the JPEGs. For example, a directory full of files that looks like this:

IMG_0123.jpg
IMG_0124.jpg
IMG_0127.jpg
IMG_0128.jpg
...
IMG_0248.jpg
IMG_0250.jpg

could be renamed to this:

Camping Photos_001.jpg
Camping Photos_002.jpg
Camping Photos_003.jpg
Camping Photos_004.jpg
...
Camping Photos_112.jpg
Topic: 

Low Disk Space Warning Script

Have you ever experienced a full disk on a server or a desktop? Not fun. This script would normally be run as a cron job and would notify you by email if any drive's free disk space has passed below a certain threshold. The code could be better; I wrote this one a long time ago when I was a bit of a n00b and I was in a rush as well. I might make take a look at it again and see if I can make some improvements.

Topic: 

mp3 Sample Clip Creator With Fade-In/Out

This script will create mp3 sample clips. I created it to automatically create sample mp3 clips for a friend of mine's (Will Stroet) website. You can tweak the length of the clip and the fadeout. Unfortunately it can only create samples that start at the beginning, however, it could be easily modified to start anywhere. It depends on the quelcom library, lame, and mpg123.

Topic: 

Why I Chose Python Over C#

I was recently tasked with writing a sipmle UI and the choice of what language/framework to use quickly boiled down to two choices: C# or Python. I eventually chose Python and here's why:

XML

When I started sketching out a design for this project in C#, I ended up envisioning lots of XML configuration files, in place of compiled-in, hard-coded configuration. In Python, however, XML is rarely required. Instead you can use python modules and data structure to specify your configuration.

You can notice this difference clearly if you look at C# and Java web frameworks and compare them to Python web frameworks. Django (a Python web framework) has a few configuration files. One is settings.py which is basically just a list of key value pairs. Some of the values, however, are Python tuples (immutable lists). It looks like a hybrid between a .ini file and an xml file. The beauty of this is that you can actually run a program like pychecker or pylint on your program and if you are trying to access a key in your settings file that doesn't exist, it will complain! Try doing that in a compiled language. So Django and other programs just take advantage of the fact that since code isn't compiled, you can put all your config in code, and you can easily tweak it later without needed to re-compile anything.

In web frameworks and other projects for C#/.NET and Java, XML is a huge part of configuration. Spring, Hibernate, Ant, log4j/log4net all rely on either XML configuration or annotations (which just couple settings to your code and bake it into the build). So if you write your own applications in Java/C# you will find yourself also using XML for configuration and writing code to parse and possibly write XML as well. The only time I ever enjoyed using XML was when I was using the ElementTree library for Python, which is now a standard library in Python.

Less code

Python code is generally a lot shorter. This means it's quicker to write, quicker to read, quicker to debug, quicker to modify later. No braces, no semi-colons, terse data structures, list comprehensions.

No compiling

Being able to modify code in the field is huge. Many times we've modified Python code in the field using a text editor after getting some obscure error and then re-ran the script. No re-compilation necessary.

Libraries

I tried porting a simple python script to C# just to see how easy/hard it would be. The first step was porting the command-line options parsing. I used GNU getopt style parsing, which is included in python's getopt library. No such thing is included in .NET. There is a third-party library, CSharpOptParse. Having to download this was a bit of a turn off. Then I looked for an example of CSharpOptParse usage and I found one. Ugh. The python getopt example is much nicer. If you don't like getopt there is also optparse (apparently, "optparse is a more convenient, flexible, and powerful library for parsing command-line options than getopt". I'll have to give it a try!). It looks even simpler than getopt!

The next thing I to find was to look at how to call an external executable in C#/.NET and capture stdout and/or stderr. Talk about annoying. Python's new-ish subprocess library is awesome.

Finally my Python script does some path splitting using os.path.split and os.path.splitext. I did find .NET's Path class to be pretty convenient, although no better than python's os.path.

Documentation

The Python documentation is far better than anything I have seen in the .NET/C# world. Maybe it's because smart people use Python.

Conclusion

I've been a long-time Python user but I do like languages like C# and Java as well, but when I put Python and C# side-by-side for this simple little project, nothing competes with it.

Tags: 

Pages

Subscribe to David Grant RSS