From The Light Point Security Blog

Advanced Subversion – Open Source Merging

With Light Point Web, we use a lot of open source code. However, some of this code is released only as a zip, not the ability to pull directly from their version control system. This can cause a lot of problems if you are trying to use this code to build a product of your own. Some problems you might encounter:

• keeping track of which version of their code you are using
• how to update your source to their newest version
• how to merge their updates with your local changes

This guide should be able to help you get around all these issues.

Setting Up Your Repository Structure

To start off, lets make up a sample project. Lets say you have a subversion repository for your project, called wowza, checked out to C:\projects. In your trunk somewhere, you have a mix of your own code, and open source code. Lets say you are using 2 open source libraries to provide some functionality. One is called perfect, and the other is called almost. Purely coincidentally, the perfect library requires no changes by you, but almost needs a few source code changes to be right for your needs. Here is how I recommend you set up your repository:

• C:\projects\wowza\trunk\
  • code\
    • mainappcode\
      • <your own code here>
    • opensource\
      • perfect\
        • <all the perfect source code>
      • almost\
        • <the almost source code, with your mods>
  • externalCode\
    • opensource\
      • VERSIONS.txt
      • almost\
        • <the almost source code, without your mods>

So what have I done here? First, try and keep all the related open source code together, and away from your own code. Most importantly, keep a pristine copy of the code of any open source project that you have altered, before you have made any changes to it. You don’t have to keep a second copy of open source code that you haven’t made changes to. Since you haven’t altered it, they should be exactly the same anyway. Finally, in the directory that stores the pristine copy of almost, put in a VERSIONS.txt. It should contain the version of both perfect, and almost. Something like this:
perfect: 1.2.0.3
almost: 4.0.1.1

Updating a Project Which You Haven’t Altered

So everything is going great with your project. You find out that the project perfect has released version 1.3.0.1. Now you need to update your code to match it. Because you haven’t made any local changes, this one is easy. Download the new code for perfect, and put it somewhere on your hard drive (not in the repo just yet). The first thing we need to do is find out if there are new files, or deleted files. If the code base is small enough, you can do this manually. However if it is a large project, I recommend using a recursive diff utility. There is diff on linux machines, and you can use it on windows if you install cygwin. Run a recursive diff on your code in trunk and the new code somewhere else on your hard drive(this will show you what all has changed between the 2 versions). Look for new files, and removed files, and make a note of them. Next, just copy the new code, right over your old code. Finally, remove any files that the diff indicated should be removed from your repo. Likewise, add any new files. As a last step update your VERSIONS.txt in the externalCode directory:
perfect: 1.3.0.1
almost: 4.0.1.1
Thats it, perfect is up to date. If the interface to it has changed, you may have to alter whatever code is using it so it still works. Test it out, and check it in.

Updating a Project You Have Altered

Now you find out the project almost is up to version 4.3.0.1. This one is a little harder, but here is how to do it. Repeat all the steps listed for updating project perfect above, except, do it for the pristine copy of almost in externalCode, instead of the copy with your changes in code. Don’t update VERSIONS.txt just yet though. Once the pristine copy is up to date, check it in and make note of the revision number of that check in. For an example, lets say that check in was revision 108.

This means change set 108 contains all the changes necessary to bring almost from version 4.0.1.1 to 4.3.0.1. This is exactly what we want to add to our almost code in code. To merge this in, open a command prompt to C:\projects\wowza\trunk\code\opensource. From there, execute this command:
svn info
This will print some information about your repository. Somewhere in here will be the URL of your repository’s server. The line will look like this:
URL: <the project url>/wowza/trunk/code/opensource
What we want is the part after URL: and before /wowza. Once you have this, still from the opensource directory perform the merge:
svn merge -c 108 <the project url>/wowza/trunk/externalCode/opensource
At this point, all the changes in 108 will be applied to your modified version of almost. You may have conflicts here if your mods are in the same places as the changes in 108. Just take your time, and find all the conflicts, and make them right. Once you have resolved the conflicts, and told subversion you fixed them with a svn resolved <filename>, test it out. Finally, update the VERSIONS.txt and check it in. You are now all up to date!

Related posts:

• Managing Projects with Subversion and Trac: Free eBook
• Can a Project Have Too Much Support?