David Grant's blog

How I increased my Nexus 4 battery life to 48 hours on one charge

I did several things to increase the battery life on my Nexus 4 phone. Here's a list of all the things I did. I am not sure exactly what led to the biggest improvement in battery life because I didn't do extensive testing at each step. Here they are:

Unlock, custom ROM, packages

Configuration

  • If you can, delete the Facebook app and use Facebook in your browser instead.
  • Disable notifications and auto-updating/auto-syncing in as many apps as you can.
  • Look at what is using the most battery under "Setings->Battery" and remove any apps that you don't actually need.
  • If you can, disable Google Now and Google Location Reporting (found in "Settings->Location").
  • Put "Location->Mode" into "Battery saving" mode
  • Under "Settings->Sound" disable "Touch sounds", "Vibrate on touch", and "Screen lock sound"
  • Under "Settings->Language & Input->Android Keyboard (AOSP)" disable "Sound on keypress" and "Vibrate on keypress"
  • In the Hellscore Kernel Manager, set the CPU "Min Freq" to 94 MHz.
  • After recharging your phone and using it for a whole discharge cycle, look at Partial Wakelocks in the BetterBatteryStats app.
  • Using the Greenify app, Hibernate as many of the apps that show up in the "App Analyzer" as you can.
  • In "Settings->Display & lights->Brightness", set to "Auto" and then click the gear to configure. Enable "adjust to sunrise and sunset". Click "Adjust" and move all the sliders to the left by about 5% or so.
  • Get a watch so you don't have to wake-up your phone to check the time.

Now I can get about 48 hours on one charge with minimal usage (checking email once in a while, sending a few messages, checking a few web pages.

Tags: 

Technical documents should be reviewed by technical people

Just recently, I was implementing a protocol at work. The protocol was handed to me in the form of a Word document and it appeared to have already cleared some approval stages. The protocol seemed over-engineered to me. It used HTTP POST with XML in the request and response bodies. There was actually only one message type and it seemed like it could have easily been implemented with a simple GET request and no response body (just an HTTP response with a proper response code). The document was 24 pages long when it only required 1. It had about 10 error response message types, about half of which had to do with XML errors. I looked at who reviewed the document and it was as follows:

  • Author: Engineering Manager
  • Reviewer: VP Engineering
  • Reviewer: Technical Director
  • Reviewer: Director of Engineering, Client Engineering
  • Reviewer: Director of Engineering, Host and Infrastructure Engineering
  • Reviewer: Engineering Manager, Quality Engineering

Missing are Principal Engineers and most importantly, Senior Developers, like myself, who will be implementing the protocol. It's not even necessary to consult the people who will be implementing the protocol, but you need to consult with people who understand HTTP, have experience with using web protocol, and an understanding of programming. One of those listed above qualifies in that respect, but that is still just one person. I am sure that if this was shown to three other people who aren't managers, directors, or VPs, they would say this protocol is over-engineered. If even one person thinks it's over-engineered, and over-engineering things is costly. It adds complexity on both sides of the protocol which means more development, more bugs, more testing. There's no excuse for over-engineering things and not excuse not to have technical documents revealed by those who will implement, the hands-on technical people.

Burning CDs/DVDs/Blu-Ray in Debian/Ubuntu

For long time I've been wondering why I can't seem to burn DVDs or Blu-Rays properly using k3b in Linux (or any other graphical burning software for that matter). The problem is cdrkit (a fork of the original cdrtools).

You can get the latest version of cdrtools, which actually supports burning to Blu-Ray and also should create DVDs properly, by installing cdrecord from this PPA. You might as well also install mkisofs from there too.

Prior to finding this PPA, I had compiled cdrtools from source, which is pretty easy. That also involved installing smake, also from the cdrtools author.

Logging in to a Django site with a magic token

I have a simple video website for my kids and each kid has a separate login. This is so they can each have their own videos, but also so that some videos can be private (ie. hidden from the outside world, or other logged in users). Typing in a username and password is impossible for my kids to do, as they are almost 5 and 2 years old, and they use this website on Google TV. So, with a magic token-style login, all they need to do is navigate to their bookmark on the Google TV homepage and press OK on the remote control.

(I don't need crazy security--it wouldn't be the end of the world if somehow someone guessed the magic token and saw some private videos, which are basically just home videos uploaded to Youtube. Videos that I really wouldn't want the public to see don't get uploaded to Youtube in the first place.)

I couldn't find how to do this easily, although one person on stackoverflow suggested "logging in the user in the view by calling 'login'". The tricky part was figuring out that I had to set the User object's backend to 'django.contrib.auth.backends.ModelBackend'. It's a bit of a hack, but it works, and it's simple.

models.py:

class MagicToken(models.Model):
    user = models.OneToOneField(User)
    magictoken = models.CharField(max_length=128, unique=True)
 
    def __unicode__(self):
        return unicode(self.user)

views.py:

from django.http import HttpResponse, HttpResponseRedirect, Http404
import django.contrib.auth.login
 
class MagicTokenLogin(View):
    def get(self, request, token):
        try:
            magic_token_obj = MagicToken.objects.get(magictoken=token)
        except MagicToken.DoesNotExist:
            raise Http404
 
        user = magic_token_obj.user
        user.backend = 'django.contrib.auth.backends.ModelBackend'
        django.contrib.auth.login(request, user)
        if request.user.is_authenticated():
            # login successful
            return HttpResponseRedirect(reverse('some-view-for-logged-in-users'))
        else:
            # login failed
            return HttpResponseRedirect(reverse('some-view'))

Be careful renaming folders with copy.com

I recently renamed a folder in my copy.com folder and caused Copy.com to think that I had deleted it. For example, I had the a folder called "Photos 2013." I installed the copy.com agent on a new computer and it began syncing. Midway through the sync, I wanted to change the folder's name to "Photos 2013-2014". This was interpreted as "Delete 'Photos 2013' folder and all sersver-side contents" and create a new folder called Photos 2014 with the photos in it. So let's say there were 100 photos, and 10 were were synced when I did it. The 90 photos that had not yet been synced would be flagged for deletion on the copy.com server. The 10 photos that actually got synced will remain, in the new "Photos 2013-2014" folder.

All I can suggest to avoid this is don't touch the folders while they are being synced.

Tags: 

Wiping out a hard drive in Linux

To wipe a hard drive using zeros:

dd if=/dev/zero | pv -s 250000000000 | dd of=/dev/sdX bs=10M

To wipe a hard drive with random data (more secure):

dd if=/dev/urandom | pv -s 250000000000 | dd of=/dev/sdX bs=10M

For the "pv" progress bar, do "apt-get install pv" or similar. I highly recommend using "pv" so you have some idea of when the dd job is going to finish.

Tags: 

First thoughts about Django-CMS

I just set up a site using Django-CMS over at Pomme d'Api Preschool. Overall, I am pretty impressed. I really like the Django admin interface enhancements as well as the site overlay editing feature. One of the best things it he use of django-filer, a file management tool to manage all the uploads to the site. It's awesome and it makes me want to throw out this home page completely and switch over to Django-CMS. Image handling in Drupal has always sucked and still does. The fact that there was no official media/image handling in Drupal 6 sucks, and it's made worse by the fact that there is no real upgrade path to Drupal 7 for media/images, especially if you're like me and you used the "Image" and "image-assist" modules, two of the most popular modules dealing with images.

Django-CMS is pretty powerful and so far it has allowed me to do pretty much anything I have wanted to do.

svn_load_dirs: E200009

If you ever get something like this:

Running /usr/bin/svn add -N --targets /tmp/svn_load_dirs_ErUQhjXmWG/targets.00001
/usr/bin/svn_load_dirs: /usr/bin/svn add -N --targets /tmp/svn_load_dirs_ErUQhjXmWG/targets.00001 
failed with this output:
svn: warning: W150002: '/tmp/svn_load_dirs_ErUQhjXmWG/my_import_wc/scripts/drupal.sh' is already under version control
svn: warning: W150002: '/tmp/svn_load_dirs_ErUQhjXmWG/my_import_wc/scripts/run-tests.sh' is already under version control
svn: warning: W150002: '/tmp/svn_load_dirs_ErUQhjXmWG/my_import_wc/scripts/password-hash.sh' is already under version control
svn: E200009: Could not add all targets because some targets are already versioned
svn: E200009: Illegal target for the requested operation

when running svn_load_dirs, try running svn_load_dirs with the -no_auto_exe option. That fixed it for me.

Django's select_related and prefetch_related

I was working on a personal website for my kids where I can post videos for them and I noticed that for each video link that was displayed on the page there were 2 additional SQL queries. One to get the username of the user that added the video link, and another to get the list of categories for the link.

The model looks like this:

class Link(models.Model):
    ...
    # There was an extra query per link to get the categories for a link
    category = models.ManyToManyField(Category)
    # There was an extra query per link to get the user.username
    user = models.ForeignKey(User)
    ...

The reason that an extra query was required to get the username is because the query to get all the links does not look at the user table at all. It doesn't do any joins. So it has to look up the username by doing something like "select username from users where users.pk == link.user_id". You get the idea. And it has to do this for every link on the page. This is easily fixed by doing

Link.objects.select_related().all()

or something similar. Django will then do a JOIN on the User table so it will now have all the user-related fields in the result set.

I was still getting a query for the category names on each link. To resolve this, I did:

Link.objects.select_related().all().prefetch_related('category')

This causes Django to do a fetch on the Category_Link table (the table that describes the many-to-many relationship) to get all the categories that match the primary keys for the links displayed on the current page. Then a join is done in Python, essentially mapping a list of category objects with each link object.

The number of queries for my home page was reduced from 44 down to 5. Page load time has been decreased by 1-2 orders of magnitude.

So, the naive rule-of-thumb might be to always use select_related() if you have a ForeignKey and you are going to reference fields in the ForeignKey in your template, and to always use prefetch_related if you have a ManyToManyField and you need information in the related table. For now I am going to NOT use these commands unless I first use the django-debug-toolbar and figure out what is going on. Then if it's necessary to use select_related or prefetch_related I will.

Tags: 

Pomme d'Api website hacked

I've been maintaining and old Joomla 1.5-based website for my daughter's preschool, Pomme d'Api. It got hacked, I'm not sure how, via FTP or PHP but they basically managed to get 2 .php files on the server that were causing search engine crawlers to get a different version of the site that contained all sorts of links to various pages. It also changed the meta-information such as keywords and description. I've now removed the hack and instructed Google to re-index the site via Google's Webmaster tools, but frustratingly, Google's crawler still hasn't re-indexed the site! It's been over a week since Google last visited it... hopefully providing this link to the website will cause Google to re-index it.

Tags: 

I hate hash-tags

I hate hashtags. At least they way that people use them. People seem to like using them to say things they wouldn't normally say in plain English. For example:

Why won't my kid take a nap? #damnkid #brat #nomorekids

This doesn't translate exactly into the following:

My damned kid won't nap

My kid is a brat

I don't want any more kids

So why do people use hashtags like those above? Hash tags have become a way to make a stupid one-word joke or sarcastic comment. For sarcasm, the classic goes something like this:

Forgot to strap my kid into his car-seat today. FAIL. #parentoftheyear

It's a pretty lame joke, but the root cause is that you probably shouldn't be telling the world about your bad parenting in the first place, but you did, and so you're trying to make light of the situation so you don't feel like the idiot that you are.

Java exceptions guide

This is intended as a dumping ground for all the tips on using exceptions that I accumulate over the years.

Don't swallow exceptions

Never, ever do this:

try {
  ...
} catch (IOException ioe) {
  // do nothing here
}

Always log a message, or print a stacktrace, or re-throw the exception, or throw a new exception (and wrap the caught exception).

Allow your checked exceptions to wrap other exceptions

class MyException extends Exception {
  public MyException(String message, Throwable cause) {
    super(message, cause);
  }
}

This is almost always useful. You will often find yourself catching an IOException and re-throwing it as your own exception. The main point is DON'T DISCARD THE ORIGINAL EXCEPTION. If you do, you won't see it any stack traces as the "caused by:" exception.

Don't log an exception that you are re-throwing or wrapping

This is a minor detail but it makes for much cleaner logs.

try {
  ...
} catch (IOException ioe) {
  log.error("Got an IOException", ioe); // not necessary
  throw new MyException("Had a problem contacting the server on port " + port, ioe);
}

In the above, assuming the logger is log4j or something similar, it is going to print the stacktrace for the IOException exception. Then whatever catches MyException up the stack will also log the IOException. This makes for message logging. Either log the IOException and do not wrap the IOException, or don't log it and just wrap it.

Create your own top-level exception classes

I've often found it useful to create two top-level exception classes, one that extends Exception and one that extends RuntimeException. For example, ABCCompanyException and ABCCompanyRuntimeException. Sometimes at the top-level of your application (in the main() method or the HTTP request handler, for example) you want to catch everything that hasn't been dealt with yet. It can be useful to handle "ABCCompanyException" and it's sub-classes separately from Java's "Exception" or "ABCCompanyRuntimeException" separately from Java's RuntimeException.

Don't forget to catch run-time exceptions in callback methods

Let's say you're implementing an event handler to be used in a third-party event bus framework, or implementing an "onOpen/onClose/onMessage" type interface for a web socket client. You should wrap the entire body of the method in a try...catch (Throwable t) otherwise a run-time message might be thrown somewhere, bubble up into the third-party library, and be swallowed there. Do this any time you are implementing a method that you don't actually call yourself. The same thing goes for the run() method in the Runnable class.

Social Networking Bullshit

I'm tired of all this social networking bullshit. I mean the social networking bullshit that is always asking me to click something, creating visual noise, and generally just pissing me off because I'm not interested in sharing everything I do with the world. Today I purchased some books at Chapters and after my purchase went through I got this screen asking me if I want to "tell my friends about my purchase" on Facebook, LinkedIn, or Twitter.

[img_assist|nid=403|title=Chapters - Share Your Purchase|desc=|link=none|align=left|width=501|height=233]

The only good thing to come of this is that as more and more people share useless information on social networks that no one cares about (like your lastest high score in some game that no ones care about but you, or your recent check-in at a local business, or something that you liked), more are more people will get pissed off and stop using them. I barely go to Facebook anymore because I find the ratio of crap to quality is monotonically increasing. Google+ is a bit refreshing because they are so few people that use it, there is therefore less crap; however, I can still use it for the things that matter to me, like sharing photos and videos with close friends and family (not the public), even those that aren't actually signed into Google+.

It's probably because I'm getting old, and being older and wiser, I can see things for what they are. I have no illusions that my friends will care what I buy at Chapters, and I know that Chapters just wants to sell more books. It's a little thing, but when combined with other things (the new Plum points crap), it's another reason to shop somewhere else instead.

I Just Tried Windows 8

I just tried Windows 8, and to be honest it's pretty awful. It looks like it's meant for running on a touch screen but if you are a desktop user it's horribly awkward to use. I'm not sure if it's a bug but getting the task bar thingy (what's it called?) to show up on the left-hand side of the screen is very difficult. I seem to have to hover on the top-left corner and wait for an icon to appear, then move downward, then the whole task bar thingy will appear. Now that I am in the Maps App, I have to go to the bottom-left corner and a "Start" thingy appears but I have to move very fast to it before it disappears (and for some reason, clicking on it right now doesn't do anything, so I seem to be stuck in the Maps app). Ok, I seem to be able to get the Start menu button to appear at the bottom-left but clicking on it doesn't always work.

There does seem to be an app called "Desktop" but it doesn't seem possible to launch any apps from it. It just has a Windows Explorer and an Internet Explorer icon. The menu in Windows Explorer also seems horribly complicated (I guess this is the whole "ribbon" thing). Shudder. It's just awful.

All the apps seem to be full-screen, at least all the apps accessible from the "Start screen" (is that called "Metro"?). Not really sure what the deal is with that. How are we supposed to work with two applications at the same time? I can't seem to launch apps from the "Desktop" I have to launch them from the "Start screen".

I didn't think it was possibly to make something worse than Ubuntu's Unity, but it seems that Microsoft has managed to do that. BTW, I am starting to like Unity more and more (although at the moment I am taking a short break and using KDE4 instead).

Moving Data from One Database to Another with Django

So I have a django database I'm working on and I decided I wanted to do the development in sqlite3 instead of mysql. I decided to do this because it makes it easier, for example, to have someone else work on HTML/CSS if I can just give them a directory, tell them to run a bash script and go to http://localhost:8000, rather than them having to do all that AND setup a mysql server. Sure, that can also be done with a script, but with sqlite things are just a hell of a lot easier in some ways.

I decided I wanted to take the data out of my mysql database and load it into a sqlite db. Not so simple, you can't just take an SQL dump from mysql and import it into sqlite3. I started Googling around for "mysql to sqlite" and didn't really get anywhere. I then realized that Django can already talk to sqlite and mysql transparently without me having to know any SQL. So I thought about writing my own python modules to do this, but it turns out someone has already done it.

The django-extensions project has a manage.py command called "dumpscript" that "Generates a Python script that will repopulate the database using objects. The advantage of this approach is that it is easy to understand, and more flexible than directly populating the database, or using XML." This was exactly what I wanted.

Tags: 

Pages

Subscribe to RSS - David Grant's blog