Using git subtree merging, while also merging in a

I'd like to use a popular, open source issue tracker (Redmine) that offers git integration. Unfortunately, each project in the tracker can only be associated with one git repo. Creating multiple projects in the tracker is not my ideal setup.

With that in mind, I've attempted to use git subtree merging (explained here, and here). I've created an "umbrella" repo which has merged in each of the numerous other repos that I'm working with.

Unfortunately, the examples given only pull in the master branch of each subtree. Since I have development going on in multiple branches of each subtree, I need to learn how to have this umbrella repo reflect each branch of each subtree.

Is this possible?

Extra Credit: What if 2 subtrees each have a branch with the same name?

For those of us not familiar with Redmine, please extend your description to include answers to the following questions: What kind of access into the repository does the tracker need? Will it need to make its own commits? Or, does it just need certain kinds of read access (perhaps to validate commit hashes and scan commit logs for keywords)?

If your tracker only needs read access, you may not need any subtree merging at all. It is perfectly acceptable to have multiple initial commits (allowing multiple, independent histories) in a single repository. The Git project itself does this for some ‘extras’ (man, html, todo) that share no (commit) history with, but are published alongside the main set of branches for the source code (maint, master, next, pu). For your purpose, it may be enough to setup a remote for each sub-repository and fetch their branch tips into your aggregating repository. Maybe the automatic ‘remote tracking branches’ would be enough, or maybe you need to take the extra step to create (and update) local branches based on the remote tracking branches.

The subtree merging scheme you describe is probably not meaningful in the general situation where the branches in the source repositories are unrelated or only semi-related. But, if all the source repositories share a set of branches where each branch has a given purpose that is the same across all the repositories, you could probably meaningfully merge them into a kind of super-repository.

But the interesting question is not “what if two repositories have branches with the same name?”, but “how do you handle the case where a repository is missing a branch from the shared, ‘global’ set?”.

If all the sub-repositories have the same set of branches, you just do what you did with master, but once for each branch. The problem comes when a particular branch is missing from a repository. You could substitute its master, but that may not always be the right answer. It depends on why you are aggregating the repositories together in the first place and what you expect to ‘see’ in that subtree of that branch in the super-repository.

If the sub-repositories are not closely related, then I really have my doubts about the reasonableness of this subtree approach. Such an approach for unrelated repositories feels like it would be ‘going against the grain’. It is probably still possible, but I doubt there is any tool to help, and you will need to spend some time planning out the corner cases.

If you end up sticking with subtree merges, you might look at the third-party git subtree command. It might help in keeping your myriad repositories synchronized.

Collecting Branches, Without Merging

If Redmine specifies --mirror clone, the implication is that it expects local branches and may not be able to directly read the ‘remote tracking braches’, so you will probably need to create and update some local branches.

Local Branches Updated From ‘Remote Tracking Branches’

Initial Setup

mkdir $COLLECTION_REPO && cd $COLLECTION_REPO &&
git init
git remote add alpha <url/path-to-alpha-repo>
git remote add bravo <url/path-to-bravo-repo>
git remote add charlie <url/path-to-charlie-repo>
for r in $(git remote); do
    git config --add remote.$r.fetch \
      "$(git config remote.$r.fetch | sed -e 's.heads.tags.;s.remotes.tags/all.')"
    git config remote.$r.tagopt --no-tags
done

Periodic Update

git remote update
git for-each-ref --shell --format \
  'git branch --force --track -l all/%(refname:short) %(refname:short)' refs/remotes \
  | sh

Local Branches That Directly Receive Fetched Branch Tips

Initial Setup

mkdir $COLLECTION_REPO && cd $COLLECTION_REPO &&
git init
git remote add alpha <url/path-to-alpha-repo>
git remote add bravo <url/path-to-bravo-repo>
git remote add charlie <url/path-to-charlie-repo>
for r in $(git remote); do
    git config remote.$r.fetch \
      "$(git config remote.$r.fetch | sed -e 's.remotes.heads/all.')"
    git config --add remote.$r.fetch \
      "$(git config remote.$r.fetch | sed -e 's.heads.tags.g')"
    git config remote.$r.tagopt --no-tags
done

Periodic Update
```
git remote update
```

Both methods end up collecting branches under refs/heads/all/<remote-name>/<branch-name-on-remote>, but the first also has a duplicate set of refs under refs/remotes/<remote-name>/<branch-name-on-remote>. The first uses a normal fetch refspec and uses git branch to duplicate the ‘remote tracking branches’ (refs/remotes/…) into normal, local branches (refs/heads/all/…). The second one uses a custom refspec to store the fetched refs directly into the destination ref hierarchy.

Because of the way updates are blindly fetched into this combined repository, no one should ever try to directly use it: no commits made directly on its branches, no pushes from outside. If someone were to make commits locally or to push onto one of the branches those commits would be wiped out when the next update is done.

If Redmine can handle a bare repository, I would recommend using one. Use git init --bare and a repo name that ends with .git. Also git config core.logAllRefUpdates true might be a good idea (since this defaults to false in a bare repository).

Besides the all/ prefix in the namespaces, another difference between this approach and a full --mirror clone is that refs outside refs/heads and refs/tags will not be collected. Most of the other common refs are considered ‘local’ to a repository (which is why they are not copied by a normal clone). Some of the other refs are ‘remote tracking branches’ (refs/remotes), some ‘bisect’ record keeping (refs/bisect), git filter-branch ‘original’ ref backups (refs/original), et cetera. Probably none of these other things are important for Redmine. If they are, they can also be included with additional refspecs.

Creating Extra Initial Commits

To arrange for a branch with an new initial commit, see GitTips page under How to create a new branch that has no ancestor. Two of the recipes involve another repository from which you push or fetch a branch after going through the usual init/add/commit step (exactly what the above recipes do in an automated way).