Maintaining Vendor Sources With Subversion

Here is how I upgrade my Drupal sites (which have their own customizations) to new versions of drupal core when they become available, as I mentioned in a previous post.

This section of the subversion book pretty much explains it. Here is specifically what I did for Drupal, assuming you already have some directories set up as follows:

/var/svn/repos/www/vendor/drupalcore
/var/svn/repos/www/vendor/drupalcore/4.7.2
/var/svn/repos/www/vendor/drupalcore/4.7.3
/var/svn/repos/www/vendor/drupalcore/4.7.4
/var/svn/repos/www/vendor/drupalcore/current
/var/svn/repos/www/drupal4.7

The /var/svn/repos/www/drupal4.7 is like my own drupal4.7 "trunk". The /var/svn/repos/www/vendor/drupalcore is where I track all the vendor sources.

Download the new drupal sources:

cd ~/downloads
wget <a href="http://ftp.osuosl.org/pub/drupal/files/projects/drupal-4.7.5.tar.gz

">http://ftp.osuosl.org/pub/drupal/files/projects/drupal-4.7.5.tar.gz
[/ge...
Run svn_load_dirs to load the new sources into current and create a tag of current called 4.7.5:

svn_load_dirs file:///var/svn/repos/www/vendor/drupalcore current ~/downloads/drupal-4.7.5 -t 4.7.5

Now go to a checked-out copy of /var/svn/repos/www/drupal4.7

cd /var/www/localhost/htdocs/sandbox
svn st
svn ci -m "commit some uncommitted changes that I had (might as well get these in)"

Merge in the changes between 4.7.4 and 4.7.5. This is the magic (almost as magical as svn-load-dirs):

svn merge file:///var/svn/repos/www/vendor/drupalcore/4.7.4 file:///var/svn/repos/www/vendor/drupalcore/current .

Now you will see a bunch of output similar to what you would get if you did an update. Look through the changes with svn diff |less. Especially the changelog, just to make sure you are actually upgrading from 4.7.4 4.7.5 and not something different (due to a typo in the previous svn commands above).
Make sure to test the codebase first. I backed up my database first, then went to <a href="http://localhost/update.php">http://localhost/update.php[/geshifilter-code] and made sure the upgrade worked. Very important step!
Commit it when you are happy:

svn ci -m "Merged differences between drupal core 4.7.4 and 4.7.5"

After this I logged in to my 3 drupal sites, backed up the db, ran svn up, ran <a href="http://localhost/update.php">http://localhost/update.php[/geshifilter-code], and checked that there were no errors and that the site loaded some pages with no trouble. If I notice a problem in a few days I can always revert to the database backup I made.

That's all there is to it to upgrade a Drupal site when a new Drupal tarball is released! As long as you track vendor sources in your own subversion repository, the rest is easy.

The advantage of this technique is that I can make changes to drupal core but I can still easily merge in changes from new drupal releases. I can also do this same thing with modules. I track the module sources in a directory structure like:

/var/svn/repos/www/vendor/drupalmodules/gmap
/var/svn/repos/www/vendor/drupalmodules/gmap/09_09_2006
/var/svn/repos/www/vendor/drupalmodules/gmap/10_24_2006
/var/svn/repos/www/vendor/drupalmodules/gmap/10_25_2006
/var/svn/repos/www/vendor/drupalmodules/gmap/current
/var/svn/repos/www/vendor/drupalmodules/flash_gallery
/var/svn/repos/www/vendor/drupalmodules/flash_gallery/11_01_2006
/var/svn/repos/www/vendor/drupalmodules/flash_gallery/current
/var/svn/repos/www/vendor/drupalmodules/location
/var/svn/repos/www/vendor/drupalmodules/location/10_20_2006
/var/svn/repos/www/vendor/drupalmodules/location/current

I made some changes to the gmap and location module. I have had to merge in upstream changes in the gmap module 3 times. You can perform the merge by providing the location of the old tag and the new tag (or just use current) and the destination of the merge is a working directory of the module in question's directory (within the drupal modules directory of your trunk).

I have also used this technique several times for the mediawiki site at work. I highly recommend doing this any time you modify upstream source code and want to get updates from upstream when they become available.

Comments

This is not working the way I expect it to at all. I am upgrading from 5.5 to 5.7, so I have those copies of Drupal core in two directories under vendor/. When I run this

 svn merge <a href="http://svn2.assembla.com/svn/myproject/vendor/drupal-5.5">http://svn2.assembla.com/svn/myproject/vendor/drupal-5.5</a> <a href="http://svn2.assembla.com/svn/myproject/vendor/drupal-5.7">http://svn2.assembla.com/svn/myproject/vendor/drupal-5.7</a> .

svn vaporizes absolutely all of my contrib modules and my custom theme. Those are not in Drupal core, so why should a diff of 5.5 and 5.7 have any effect on those files??

Checkout http://svn2.assembla.com/svn/myproject/vendor/drupal-5.5 and http://svn2.assembla.com/svn/myproject/vendor/drupal-5.7 two to temp directories and make sure those contain what you think they contain. Do a diff of the two directories as well. I suspect that they don't actually contain 5.5 and 5.7 vanilla sources.

I suspect that your vendor/5.5 isn't actually pure drupal 5.5 but is actually drupal-5.5 + your modifications (additional modules, etc...). That would explain the behaviour you are seeing.

Next time leave your email address so I can actually reply back to you directly as well. Hopefully you are watching for responses to this post.

along that line of thinking I'd bet that svn_load_dirs is deleting those folders. You may want to make sure you are running the commands in the directory you think you are / should be.

Hi David!

Thanks for the great post! Based on your ideas, we have released a subversion repository that will be updated with every Drupal release, so that Drupal developers don't need to maintain their own vendor folder and can skip straight to the merge step. I would really appreciate your feedback on our instruction page at http://www.transmachina.com/en/drupal-subversion-repository

Best,
Nic.

Sorry for the delay in publishing your comment. It got stuck in the Akismet spam queue. It would be nice if the drupal project had an official svn repository like this, even if it was just a mirror of their CVS repository.

David,
I have been using svn for about a year and was recently trying to learn to use vendor branches. the red book helped me to cause great harm to my working dir. your step by ste instructions have helped me upgrade my skills. thank you. I have used this for way more then just zen cart.

JOsh