Two way git mirror

This articles describes how to setup a two way mirror of git repositories. I hope this article will be helpful as it works around a number of caveats for two way git mirrors.

Setup

We create a clone from either of the upstream repositories. It is important that the repository is bare.

<source lang="bash"> cd /srv/gitsync git clone --bare git@github.com:[account]/[repository].git mv [repository].git [repository] </source>

Now we delete remote "origin" and configure a remote setting for each upstream repository instead: <source lang="bash"> cd [repository] git remote remove origin git remote add github git@github.com:[account]/[repository].git git remote add sourceforge ssh://[account]@git.code.sf.net/p/[repository]/code </source>

The mirroring should happen right after changes have been pushed to one of the repositories. Therefore a webhook is required to trigger the mirror script.

On Sourceforge a project admin needs to enable it at Admin -> Tools -> Repository -> Webhooks. On Github it is at Settings -> Webhooks & Services -> Add Webhook.

Mirror scripts

We use the following script to do the actually mirroring. It can be invoked with the name of a known repository as parameter or with the "--all" flag.

<source lang="bash">

1. !/bin/bash

function sync_repo {

   cd /srv/gitsync
   cd $1
   echo $1
   # fetch all known remotes
   git fetch --all -p
   # push branches from sourceforge to github and via versa.
   git push github "refs/remotes/sourceforge/*:refs/heads/*" "refs/tags/*:refs/tags/*"
   git push sourceforge "refs/remotes/github/*:refs/heads/*" "refs/tags/*:refs/tags/*"

}

cd /srv/gitsync if [ "$1" == "" ]; then

   # no command line parameters, print help message
   echo "gitsync [report]|--all"

elif [ "$1" == "--all" ]; then

   # "--all": for all known repositories
   for D in *; do
      if [ -d "${D}" ]; then
            sync_repo $D
            cd /srv/gitsync
      fi
   done

elif [ -d "$1" ]; then

   # sync only the specified repository
   sync_repo $1

else echo "gitsync [report]|--all" fi </source>

In function sync_repo the branches from remotes named github and sourceforge are pushed to each other. You can more mappings here.

As discussed in the previous section, we use webhooks to initiate a mirroring run on every push. This is achieved by the following php script:

<source lang="php"> if ($_SERVER['REQUEST_METHOD'] !== 'POST') { die('POST required'); }

if (!isset($_REQUEST['repository'])) { die('repository not specified'); }

// validate repository name to prevent injection and traversing attacks $repo = $_REQUEST['repository']; if (!preg_match('/^[a-zA-Z0-9]+$/', $repo)) { die('invalid repository name'); }

header('HTTP/1.0 204 Found'); system('sudo -Hu gitsync /usr/local/bin/gitsync '.$repo); </source>

Security

• It is important to validate the repository name in order to prevent shell command injection, git parameter injection and directory traversing
• We use sudo to execute gitsync as user gitsync. Thus the webserver process does not require write access to the repositories
• Both Sourceforge and Github support a secret to authorize webhook invocations. In the above example this is not verified, so anyone can trigger a sync.

Deleting branches

There is one caveat: Deletion of branches is not mirrored, but deleted branches are resurrected by the mirror script.

To delete a branch for good, the following commands need to be executed in quick succession.

<source lang="bash"> git branch -d [branchName] git push github --delete [branchName] git push sourceforge --delete [branchName] </source>