Subversion Master With Git Mirrors

Subversion has proved itself to be a strong centralised version control system, establishing itself as the major corporate version control system. But the times are changing, people are collaborating with others around the world, often working offline while travelling, wanting something with a little more features locally or just something more performant. Distributed version control systems (DVCS) address all of these issues and have been proven by the vast numbers of open source developers using them to develop critical software. Dropping subversion and adopting a DVCS such as git is not an easy step for most corporate organisations, the tool chain is slightly different and there needs to be a change in approach to attain the increase in productivity that can be achieved.

This article looks at not replacing subversion but continuing to use it as the master source repository and use one or more git mirrors to provide git for those that are familiar or wish to try it. Having subversion as the master repository can also alleviate management worries around authoritative sources, learning curves and any lost productive during upskilling, and other barriers to change. Those users using git can interact with both the subversion master and git mirrors, committing any work to be shared with all users back to subversion as and when appropriate. To reduce any issues, commits should always be pushed back to the subversion repository, creating a circular flow of changes.

Repositories involved including traffic flow

Note the use of the fetch repository. This is the recommended approach to creating a mirror as git svn clone will create a local working area which we do not want if hosting as a mirror. Therefore we have to use two repositories, a standard git bare repository for the mirror and a git svn “fetch” repository for updating the mirror.

This guide makes the following assumptions:

  • master subversion repository is available at http://servername/svn/reponame/ with standard conventions for trunk, tags and branches
  • you are able to set up access to a remote git repository, ideally using something like gitosis or gitolite or even github
  • you are able to create cron jobs on the server hosting the fetch repository
  • svn, git and git-svn are installed and available on the command line
  • optional: gitk is installed and available on the command line

Updates 2010-10-06:

  • Fixed a number of mirror issues with the approach using master. Now uses trunk and HEAD is set to refs/heads/trunk using symbolic ref.
  • sync script is now a standalone script allowing manual and automated (via cron) updates.

Creating a git mirror

A git mirror can be created on the git repository server of your choice.

Note: It is important to set HEAD to refs/heads/trunk. github allows this through the repository admin, and you can do this manually with gitosis or gitolite by running the following command on a bare repository:

GIT_DIR=/path/to/reponame.git git symbolic-ref HEAD refs/heads/trunk

For example, using gitolite, clone the admin repo:

git clone git@hostname:gitolite-admin

Modify the conf/gitolite.conf file to also contain:

@mirrors    = reponame

repo        @mirrors
            R       =   @all
            RW+     =   @admins

Add and commit the changes:

git add -A
git ci -m "added reponame mirror"
git push

Now ssh to the server and run the following:

GIT_DIR=/home/git/repositories/reponame.git git symbolic-ref HEAD refs/heads/trunk

The rest of the article assumes that the git mirror is available at git@hostname:reponame.git.

Setting up a fetch repo for the mirror

git svn clone will create a clone of a subversion repository including the entire history. Unfortunately, for repositories with a lot of history, this can take a long time to run as git-svn has to replay each revision during the clone.

Ideally, the fetch repo will be setup on a server so that it cron can be used to provide regular updates. If you are hosting your own mirror then it is a good idea to also put the fetch repository on the same server.

To create the fetch repo on the server as the git user from a particular revision (e.g. 12345), run the following:

ssh username@hostname
su - git
mkdir -p ~/fetchers
cd ~/fetchers
git svn clone -s --no-minimize-url -r12345:HEAD http://servername/svn/reponame/ reponame-fetch
cd reponame-fetch
git branch -m master trunk
git remote add mirror git@hostname:reponame.git
git config --unset remote.mirror.fetch
git config remote.mirror.push 'refs/remotes/*:refs/heads/*'
git push mirror

Create a script to sync the repository. Edit ~/bin/sync-reponame and add the following:

cd ~/fetchers/reponame-fetch
echo Syncing at `date`
git svn rebase
git push mirror

Ensure the script can be run:

chmod +x ~/bin/sync-reponame

Create a suitable crontab by running crontab -e and entering the following:

*/5 * * * * ~/bin/sync-reponame >> ~/sync-reponame.log 2>&1

This will run a sync script every 5 minutes as the git user, appending the output to a log file. Change the timings as appropriate for your setup.

Git clone

Each user can create a clone using the git mirror and create the necessary svn references:

git clone -o mirror git@hostname:reponame.git
cd reponame
git svn init --prefix=mirror/ -s http://servername/svn/reponame/
git svn rebase

A user can keep up to date by pulling from the git mirror:

git pull --rebase

Or they can update directly from subversion:

git svn rebase

The git mirror may lag behind subversion depending on the timings of any cron jobs to sync repos, although this is not an issue usually. A user may wish to update from svn if they are waiting on a change that they need quickly for example. Also, in order to commit to subversion, a user should have the latest version so it is usual to update from subversion just prior to the subversion commit.

A user can now perform the usual workflows with git, committing any changes and when happy using git svn dcommit to commit the changes to the subversion master. For example [comments in square brackets]:

git pull --rebase               [ensure up to date with mirror]
....                            [modify a file]
git add -A                      [add any changes in files to the index]
git ci -m "updated a file"      [commit the changes in the index]
git svn rebase                  [ensure up to date with svn]
git svn dcommit                 [push changes back to subversion]

Working with branches

When adding features it is good practice to use a feature branch. The master branch can be kept up to date with subversion using git svn rebase and then the feature branch can be rebased onto master as and when you want to align with the current stream of development. When you are ready to push back to the subversion master repository, rebase the feature branch from the master branch and merge the branch using a fast forward merge onto the master branch. You can then dcommit the changes as normal. A sample workflow would be:

git checkout -b feature         [create a feature branch and checkout]
....                            [modify a file]
git add -A                      [add any changes in files to the index]
git ci -m "updated a file"      [commit the changes in the index]

occasionally rebase from mirror and rebase branch from master:

git checkout master             [switch back to the master branch]
git pull --rebase               [ensure up to date with mirror]
git checkout feature            [switch back to the feature branch]
git rebase master               [replay branch changes on top of master and fix any problems]
....                            [do some work, until you are ready to commit back to svn]
git checkout master             [switch back to the master branch]
git svn rebase                  [ensure up to date with svn]
git checkout feature            [switch back to the master branch]
git rebase master               [replay branch changes on top of master and fix any problems]
git checkout master             [switch back to the master branch]
git merge feature --ff-only     [merge feature branch ensuring fast forward merge]
git svn dcommit                 [push changes back to subversion]

It is important to be careful with merges due to the way git svn works, see CAVEATS from git svn man page.

Useful aliases

Git allows aliases to be created to reduce the amount you have to type for common commands and their options. Run the following:

git config --global alias.svnsync "pull --rebase"
git config --global alias.svnup "svn rebase"
git config --global alias.svnci "svn dcommit --rmdir"

You can then run the following commands, with an explanation of what they do:

  • git svnsync will do the equivalent of svn up using the git mirror
  • git svnup will do the equivalent of svn up using the svn server
  • git svnci will do the equivalent of svn ci using the svn server

Other tools

gitk (and gitx, its more beautiful counterpart for OS X) can be used to view the git repository. Its is a useful gui tool for reviewing commits. Alternatively, git log and friends will give you details on the command line.


blog comments powered by Disqus
Fork me on GitHub