Limit Size of Subversion Commits With this Hook

We have experienced some abuse of our subversion repository at work recently. Someone committed 400 MB of data all at once including many object files, libraries, and executables. I did not get very harsh with the person who did this. Because a) I have no objection to binaries in subversion in the first place, b) I don't really know what he's working on, c) disk space is cheap and we are no where near capacity, and d) his commit was still smaller than a few commits we had a long time ago (which were legit). Still, if you just allow people to commit whatever they want to your subversion repository, in the worst case, you could run out of disk space, necessitating an svn dump-and-load onto a new larger drive (pain). It would suck to have to do that just because some people were committing large binaries (without any legitimate reason to). There are other annoying consequences. Our tarball backups of svn currently fit on a DVD, which is cheap and easy, if we allowed this abuse to continue it would complicate our backup process.

What I wanted was a way to limit the commit size for certain users automatically. There did not seem to be any hooks out there to do this, so I wrote one.

Just paste the following into your pre-commit hook:

/svn/repos/hooks/check_txn_size.py "$REPOS" "$TXN" || exit 1

and paste the following into a check_txn_size.py file in your hooks directory and make it executable.

#!/usr/bin/env python
import sys,os,popen2
 
MAX_BYTES = 1024000
DEBUG = False
SVNLOOK = '/usr/bin/svnlook'
ALLOWED_USERS = ['david', 'ctang', 'vjain', 'mike', 'sbridges', 'tcirip']
ADMIN_EMAIL = '<a href="mailto:admin@company.com">admin@company.com</a>'
 
def printUsage():
  sys.stderr.write('Usage: %s "$REPOS" "$TXN" ' % sys.argv[0])
 
def getTransactionSize(repos, txn):
  txnRevPath = repos+'/db/transactions'+'/'+txn+'.txn'+'/rev'
  return os.stat(txnRevPath)[6]
 
def printDebugInfo(repos, txn):
  for root, dirs, files in os.walk(repos+'/db/transactions', topdown=False):
    sys.stderr.write(root+", filesize="+str(os.stat(root)[6])+"\n\n")
    for name in files:
      sys.stderr.write(name+", filesize="+str(os.stat(root+'/'+name)[6])+"\n")
 
def checkTransactionSize(repos, txn):
  size = getTransactionSize(repos, txn)
  if (size > MAX_BYTES):
    sys.stderr.write("Sorry, you are trying to commit %d bytes, which is larger than the limit of %d.\n" % (size, MAX_BYTES))
    sys.stderr.write("If you think you have a good reason to, email %s and ask for permission." % (ADMIN_EMAIL))
    sys.exit(1)
 
def getUser(repos, txn):
  cmd = SVNLOOK + " author " + repos + " -t " + txn
  out, x, y = popen2.popen3(cmd)
  cmd_out = out.readlines()
  return cmd_out[0][:-1]
 
if __name__ == "__main__":
  #Check that we got a repos and transaction with this script
  if len(sys.argv) != 3:
    printUsage()
    sys.exit(2)
  else:
    repos = sys.argv[1]
    txn = sys.argv[2]
 
  if DEBUG: printDebugInfo(repos, txn)
  user=getUser(repos, txn)
  if DEBUG: sys.stderr.write("User:"+user)
  if DEBUG:
    if (user in ALLOWED_USERS):
      sys.stderr.write(user+" in allowed users")
    else:
      sys.stderr.write(user+" not in allowed users")
  if (user not in ALLOWED_USERS):
    checkTransactionSize(repos, txn)

Palm TX is a Piece of Junk

I got my wife a Palm TX in December 2005. It worked for almost a year, then just as the warranty was about to expire the power button started getting all flaky and only working on 1/3 presses at best. It never really worked great to begin with, often requiring one to hold down the button for a second or two until registering that you pushed it. It seems that others had the Palm TX power button problem as well:

"My second device has worked flawlessly until just a few days ago when the power button problem returned. With a little research (Google: power button problem palm TX) it seems this is a very common problem. The problem is that the button is only soldered onto the board and no other mechanical connection is used. As the power button is used the solder deteriorates and the button eventually becomes useless. Any other button will turn on the palm and there are ways to power it off without the button but I still find this unacceptable. With a broken power button a hard reset is also impossible."

The power button was deteriorating and eventually became completely unbearable and as the warranty period was almost over we decided to return it in December 2006 before it became completely broken.

So we got a new Palm TX in the mail sometime in January 2007. No major problems until May 2007 when the wifi suddenly stopped working. It can't even try to connect anywhere. Anytime the radio is turned on, the Palm locks up. The only cool thing about this was watching the LCD fade slowly to white as the TFTs in the display leaked off charge allowing the backlight to shine through. I'm never seen a Palm lock up so badly that it completly disabled the LCD electronics. A hard reset reset was not able to fix this problem so it is clearly a problem with the hardware itself.

Anyways, I have no idea what we will do now. My wife can live without wifi and in fact did not really use it much before. But she'll be working somewhere this summer where there is no computer although there is probably wifi available nearby. It would be handy if she could send/receive email through GMail on her Palm TX. More than that though, we feel a bit ripped off as (at the time at least) the Palm TX was significantly more expensive than comparable models, largely because of the wifi. We are unsure whether the replacement model they sent us is under warranty (it would suck if they didn't reset the warranty for a reparied devices). It is doubtful that Palm would send us a different more reliable Palm model either, but we'll try anyways.

Tags: 

Better Gmail Extension

I use GMail for email, Google Reader for feeds, and Google Calendar as my organizer. Yeah, I'm Google's bitch. I recently discovered this Better Gmail extension. It doesn't have that much useful stuff. The one thing I find really handy is the integration of Google Reader and Calendar into the GMail sidebar. I no longer need 3 Firefox tabs open for these apps. It kind of takes up a lot of screen real estate opening up Reader and Calendar in an frame within GMail but with more javascript hacking this could be improved even more. Maybe Google will more tightly integrate these apps now that someone else has done it as a proof-of-concept.

Tags: 

Setting Up postfix to send outgoing mail on ubuntu

I don't know why but this is one of those things that I find extremely annoying to set up. Mainly because it often requires installing a full-blown MTA to do a much simpler job than it was designed to do. Unfortunately, ssmtp and exim, the only simpler ones, often suck (especially ssmtp, which even the maintainer gave up on). What follows is how I got postfix working on my Ubuntu Feisty machine (should work on more recent version of Ubuntu as well):

sudo apt-get install postfix
sudo dpkg-reconfigure postfix

Here are the options to answer in order:

"satellite" (when given the list of configuration styles)
<your normal user> (where to forward root's mail to)
<your host name>
<your ISP's smtp server>
accept default (when asked what other domains to accept mail from)
No (default is no synchronous updates on mail queue)
127.0.0.0/8 (accept default unless you know what you are doing)
0 (mailbox limit)
+ (default)
all (default)

Reload or restart postfix, then do some testing:

sudo apt-get install mailx
echo test |mail -s "test mail sent to external" <a href="mailto:me@gmail.com">me@gmail.com</a>
echo test |mail -s "test mail sent to normal user" <normal user name>
echo test |mail -s "test mail sent to root" root

You should get an email to your external address. And if you run

mail

you should see 2 emails there for you, one that was sent to root (but was directed to you, see /etc/aliases for why) and the other that was sent to you.

To have those mails sent to an external address instead (like your me@gmail.com address) do the following:

echo <a href="mailto:me@gmail.com">me@gmail.com</a> > ~/.forward

Update (2007-07-19): When I changed ISPs, my new ISP did not like my From: address being david@centurion (I just named it after the case model). So I had to do the following. Add this to /etc/postfix/main.cf:

smtp_generic_maps = hash:/etc/postfix/generic

Add this to /etc/postfix/generic:

@centurion      <a href="mailto:david@telus.net">david@telus.net</a>

I just made up an email address there to make it happy.
Don't forget to run sudo postmap /etc/postfix/generic

Copy ssh Public key to Server in one line

Update (2011/02/06): you don't really need any of the commands below, you can just use the ssh-copy-id program that comes with ssh.

Here it is:

cat ~/.ssh/id_dsa.pub | ssh <a href="mailto:user@hostname">user@hostname</a> "cat - >> ~/.ssh/authorized_keys"

Apparently some people don't like this one that much. :-)

Here's an alternative:

ssh <a href="mailto:user@hostname">user@hostname</a> "echo `cat ~/.ssh/id_dsa.pub` >> ~/.ssh/authorized_keys"

It doesn't require the first cat or the pipe.

Tags: 

Green Party Leader May Looking for Co-Operation from Layton

I wanted to stay out of this but I just saw a news article today that was just way too similar to my last blog post for me to not write about. The Globe & Mail's headline was "May blasts Layton over lack of party co-operation." She actually made reference to our first-past-the-post system in this criticism of Layton's refusal to talk with her about collaboration: "The door, as far as I'm concerned, is still open to discuss if there's some way that, despite our first-past-the-post system, leaders who care about their country and are willing to put the future of their planet first can't find some way to communicate," she said. Well said. Why shouldn't the two parties who both claim to care about the environment the most work together to get more seats? Seems like a win-win situation.

May went on to say that "'I felt a clear signal was needed that the Harper Conservatives still represent a grave threat to any future action on climate, as well as on a large number of social policy issues. ...This was more about putting principle ahead of partisanship.' She added that, under Canada's electoral system — 'which Mr. Layton claims he wants to reform' — 660,000 Canadians voted Green in the last election and the party was still shut out of Parliament."

She could have added that, like the Greens, the NDP also regularly receives a larger percentage of the popular vote than it does in seats. (9% vs 17% in the last election). Then it would sound like it was lifted directly from my previous post.

Liberals and Greens Swapping Ridings

Back in August, I suggested the Greens and NDP should work out some sort of riding swapping arrangement. I suggested this as a way for both parties to gain seats by reducing vote-splitting, reducing the number of "wasted votes", and making the parliament more representative of the popular vote. In the absence of true proportional representation (which is a long ways away in Federal politics) this is a great idea.

Today a similar arrangement was made made between the Liberal party and the Green party that "is being touted as a historic turn in Canadian politics" according to this CTV article, "Critics charge May, Dion made backroom deal." The Liberals will not run a candidate in Peter Mackay's Nova Scotia riding (the riding in which Green leader Elizabeth May is running in). In turn, the Greens will not run in Stephane Dion's riding of Saint-Laurent-Cartierville in Quebec. Rather than describe this move as a way to "team up to get Harper" or a ploy to reduce vote-splitting (and it is reasonable to try to reduce it), Ms. May said it came out of their "shared commitment to a greener Canada." Fair enough. Harper is the weakest on the environment, so that's another of saying "we want anyone other than a Conservative to win either of these seats. Just based on the 2006 election's results, it turns out Peter Mackay's seat is probably a sure win for him, with or with the Liberal candidate, and Dion probably has a lock on his riding as well so this whole thing won't really make a difference. There's nothing that Dion or Mackay have done (or not done) in the past year to really change those numbers too much. Which makes me wonder why they bothered? It starts to look like two party leaders trying to secure their own seat (well, in this case just Dion) in which case I can understand when John Baird said "it's a bit surprising and I think what it amounts to is an incredible example of bad judgement on Stephane Dion's part". An alternative with much better optics would be have to swap a riding where the Liberals lost previously by a narrow margin (or won by a small margin) and the Greens "stole" more votes from the Liberals than the other parties, with a riding where the Greens garnered lots of support (like the Saanicih-Gulf Islands riding where Greens grabbed 17% of the vote in 2004).

The replies from the other parties ranged from laughable to logical. One of John Baird's comments sort of makes sense: "To have to make a deal with the leader of a fifth party to try to save his own seat and prop her up just leaves you scratching your head." Again, Dion probably had a lock on his riding already with or without the swap and I don't think May stands a chance of winning the riding she is running in with or without the swap. She would have to pick up all of the Liberal votes and half the NDP's.

Jack Layton said it denies Canadians true choice in the next election: “We’re going to make sure that we offer a choice to vote for a New Democrat candidate in every riding because we think that’s Canadians’ right . . Why should people in some riding be denied the choice that other Canadians have in other ridings?"

There's that riding thinking again...as if who you elect in your own riding really matters. I agree that if there were only a Right-wing Christian party candidate and a Marxist-Leninist candidate running in my riding I would be disappointed. But we are only talking about a couple ridings here and what's stopping the parties from gaming the system a bit? The only thing we should all care about at the end of the is whether or not your vote is represented in the allocation of seats. The NDP is always the most under-represented part in parliament compared to the percentage of the popular vote it gets. They are the ones that could benefit the most from riding-swapping. The NDP is clearly in favour of proportional representation (PR). Yet, in the absence of an actual implementation of PR you have Jack Layton proudly saying that the NDP is running a candidate in every riding in Canada. If he partnered up with another party and did some riding-swapping (hence not running a candidate in every riding) that would most probably lead to a government representing the popular vote better, just like it would if there were run-offs in each riding. So Jack is contradicting himself to a certain extent. He's basically saying, we want proportional representation but only if its done right (by passing a bill). Ironically, if the NDP engaged in riding swapping with another party in key ridings it might get get people talking about the negative effects of vote-splitting, and so on, possibly leading to proportional representation at the federal level sooner, rather than later.

Hometown Baghdad Videos

I just discovered an interesting video blog created by chattheplanet.com and a group of Iraqi filmmakers. It is "a series about life in Baghdad." As it is filmed by Iraqis, they do a far better job at showing the real Iraq, as it's not just footage from inside the Green Zone or from "embedded reporters" tagging along with US troops. Great personal stories and video footage.

You can see all the videos on the Hometown Baghdad home page or on the hometown baghdad youtube page.

Tags: 

Sure Ways to Get Your Resume in the "No" Pile

We were recently looking to fill a "Senior Java Developer" position. We got some interesting applications that went straight into the "no pile". I'm sure there are tons of list out there like this one but I couldn't resist listing a few of the things I saw:

  • Your resume looks like it was written using Microsoft Works for Windows. It kind of document I might have made in 1990 with my old 386.
  • Your resume is 7 pages long. This guy thought "don't make your resume 3-4 pages" meant that it should be even longer, not shorter. Sorry, if you can't summarize your experience in 1-2 pages, or, if you can't put in the effort to tailor your resume for the specific position, you're probably incapable of doing a lot of other things.
  • The subject line of your email says that you are applying for the Senior Software Developer position and then in the closing line of your cover letter you say "I'd like to apply my skills and experience in working as a Scientific programmer." Never let on what you really want to be doing. Pretend that you actually want the job you're applying for. It is possible that this particular applicant copied and pasted from a different cover letter. So double-check your letter for typos. But there's still no way we're going to higher someone who applied for a Scientific Programmer position at a different company for a Senior Developer/Architect type position at ours.
  • You used Word's highlight tool to highlight certain words in your resume using red, green, and yellow. Yeah that's right, I'm not talking about using the "font colour" tool, but the "highlight" tool in Word. Someone actually did that.
  • You didn't attach any resume. Only a cover letter contained bulleted lists and a transcript. You're wasting your time (and mine).
  • You put things like "Watching movies & TV", "computer games", "listening music" (is the music doing the listening?), "internet surfing", and "driving" in the hobbies section of your resume. Better yet, don't even put a hobbies section. What you do in your spare time is your own business. Well, put a few interesting or unique things in if you want.
  • Your resume lists the 3 Indian languages you are fluent in. Ask yourself: is that important for this job with a North American company?
Tags: 

Finally Joined Facebook

I finally joined Facebook. I kind of like it. It looks a hell of a lot better than MySpace, doesn't suck as much as Orkut, and is similar to LinkedIn in its clean interface and simplicity. I was on Orkut for a while when it first started and then deleted my account some time later. There just wasn't much too it and was too focused on forums. Facebook actually seems kind of cool. I really like the ability to tag people in photos. Because true image search/matching is probably still years away, it's a nice workaround. It's nice to be able to "write on someone's wall" especially if that someone from high school that you would never have take the time to email because you just aren't that close to them.

Tags: 

Bacon vs. Bears

I'm going up to Bella Coola for a few days, so when I saw the fiece-looking Grizzly bear on the front page of Saturday's Vancouver Sun and the headline "Grizzly attacks plague central coast" I had to pick it up. Bella Coola is definitely central coast and as it turns out the entire article focused on Bella Coola. I hope you are as shocked as I was when you read the following story:

"Instead of going after the bears, he (Gary Shelton, a long-time Bella Coola Valley resident and author of three best-selling books on bear attacks) said, the government tells people what to do. 'I'm not supposed to feed my dog on my porch, we can't have bird feeders, you can't cook bacon on Saturday morning with the window open.'

'This is bullshit. People have a right out here to live as they want to live. At no time in history have grizzly bears and humans ever lived together in any kind of complacency. It's ridiculous.'"

I really felt for him. Not being able to feed one's dog on the porch has got to be a major PITA. And I could never live without my bird feeder, so I totally know where he's coming from on that one. But just imagining not being able to cook bacon on Saturday morning with the window open, that's when I lost. How dare those Grizzly bears confine the smell of bacon to our homes! Bullshit is right.

Tags: 

Maven is Amazing

After 2 senior developers recommended maven over ant and another told me he was planning on switching to it (from ant) for his next major project, I knew I would eventually try it when I had the chance. Today I was forced to try it because building my project with ant started becoming a total nightmare. I had added a dependency to my project and now the ant script failed. Time to modify the ant script. Said dependency had some dependencies of it's own. Not only that but one of the dependencies was an internal project and did not have an ant build of it's own so no Jar. So it looked like I was going to have to write an ant build for the dependency or put a hard link to it's output class path (created by Intellij IDEA) or use IDEA's jar maker utility. All of those are crappy workarounds. I like automated one like calls like "ant deploy" that will build, test, package and deploy my app to a tomcat server. I was fed up; there has got to be a better way.

So I tried Maven and it is amazing. I just converted a bunch of projects to use maven and it is so much better than ant. It follows the "convention over configuration" and DRY (don't repeat yourself) philosophies very nicely. It just works. No need to tell Maven where your source is and where you want it to build to. Just use the standard maven directory structure. No need to tell Maven what your classpath is for unit tests or compilation. Deploying builds to a remote or local repository is easy. Running unit tests and generating reports is easy, with no extra code in the build file required.The best part is that it fetches dependencies from a server (or the local repository cache) and fetches dependencies recursively. Building projects will be so much easier now and will save so much time. No more writing ant builds and no more copies of junit or log4j all over the place. In fact, never download any jars, ever again. Another bonus: it can generate an IDEA module file or modify an existing one. I've tried this for all the projects and it works perfectly.

So Maven is awesome, and Ant, well...Ant is dead to me.

Geocoding for Canadian Addresses Now in Drupal

What a pleasant surprise to find that my patch for geocoding of Canadian addresses for the location module was committed to CVS only a couple days after being submitted. So now geocoding for Canadian addresses works with the new location and gmap modules in Drupal 5.x. bdragon, the maintainer of the gmap module also threw in geocoding of Canadian addresses using Google as well.

Tags: 

Pages

Subscribe to David Grant RSS