How to move files from one git repo to another (no

Question

Our Git repositories started out as parts of a single monster SVN repository where the individual projects each had their own tree like so:

project1/branches
        /tags
        /trunk
project2/branches
        /tags
        /trunk

Obviously, it was pretty easy to move files from one to another with svn mv. But in Git, each project is in its own repository, and today I was asked to move a subdirectory from project2 to project1. I did something like this:

$ git clone project2 
$ cd project2
$ git filter-branch --subdirectory-filter deeply/buried/java/source/directory/A -- --all
$ git remote rm origin  # so I don't accidentally the repo ;-)
$ mkdir -p deeply/buried/different/java/source/directory/B
$ for f in *.java; do 
>  git mv $f deeply/buried/different/java/source/directory/B
>  done
$ git commit -m "moved files to new subdirectory"
$ cd ..
$
$ git clone project1
$ cd project1
$ git remote add p2 ../project2
$ git fetch p2
$ git branch p2 remotes/p2/master
$ git merge p2 # --allow-unrelated-histories for git 2.9
$ git remote rm p2
$ git push

But that seems pretty convoluted. Is there a better way to do this sort of thing in general? Or have I adopted the right approach?

Note that this involves merging the history into an existing repository, rather than simply creating a new standalone repository from part of another one (as in an earlier question).

Answer 1

I wanted something robust and reusable (one-command-and-go + undo function) so I wrote the following bash script. Worked for me on several occasions, so I thought I'd share it here.

It is able to move an arbitrary folder /path/to/foo from repo1 into /some/other/folder/bar to repo2 (folder paths can be the same or different, distance from root folder may be different).

Since it only goes over the commits that touch the files in input folder (not over all commits of the source repo), it should be quite fast even on big source repos, if you just extract a deeply nested subfolder that was not touched in every commit.

Since what this does is to create an orphaned branch with all the old repo's history and then merge it to the HEAD, it will even work in case of file name clashes (then you'd have to resolve a merge at the end of course).

If there are no file name clashes, you just need to git commit at the end to finalize the merge.

The downside is that it will likely not follow file renames (outside of REWRITE_FROM folder) in the source repo - pull requests welcome on GitHub to accommodate for that.

GitHub link: git-move-folder-between-repos-keep-history

#!/bin/bash

# Copy a folder from one git repo to another git repo,
# preserving full history of the folder.

SRC_GIT_REPO='/d/git-experimental/your-old-webapp'
DST_GIT_REPO='/d/git-experimental/your-new-webapp'
SRC_BRANCH_NAME='master'
DST_BRANCH_NAME='import-stuff-from-old-webapp'
# Most likely you want the REWRITE_FROM and REWRITE_TO to have a trailing slash!
REWRITE_FROM='app/src/main/static/'
REWRITE_TO='app/src/main/static/'

verifyPreconditions() {
    #echo 'Checking if SRC_GIT_REPO is a git repo...' &&
      { test -d "${SRC_GIT_REPO}/.git" || { echo "Fatal: SRC_GIT_REPO is not a git repo"; exit; } } &&
    #echo 'Checking if DST_GIT_REPO is a git repo...' &&
      { test -d "${DST_GIT_REPO}/.git" || { echo "Fatal: DST_GIT_REPO is not a git repo"; exit; } } &&
    #echo 'Checking if REWRITE_FROM is not empty...' &&
      { test -n "${REWRITE_FROM}" || { echo "Fatal: REWRITE_FROM is empty"; exit; } } &&
    #echo 'Checking if REWRITE_TO is not empty...' &&
      { test -n "${REWRITE_TO}" || { echo "Fatal: REWRITE_TO is empty"; exit; } } &&
    #echo 'Checking if REWRITE_FROM folder exists in SRC_GIT_REPO' &&
      { test -d "${SRC_GIT_REPO}/${REWRITE_FROM}" || { echo "Fatal: REWRITE_FROM does not exist inside SRC_GIT_REPO"; exit; } } &&
    #echo 'Checking if SRC_GIT_REPO has a branch SRC_BRANCH_NAME' &&
      { cd "${SRC_GIT_REPO}"; git rev-parse --verify "${SRC_BRANCH_NAME}" || { echo "Fatal: SRC_BRANCH_NAME does not exist inside SRC_GIT_REPO"; exit; } } &&
    #echo 'Checking if DST_GIT_REPO has a branch DST_BRANCH_NAME' &&
      { cd "${DST_GIT_REPO}"; git rev-parse --verify "${DST_BRANCH_NAME}" || { echo "Fatal: DST_BRANCH_NAME does not exist inside DST_GIT_REPO"; exit; } } &&
    echo '[OK] All preconditions met'
}

# Import folder from one git repo to another git repo, including full history.
#
# Internally, it rewrites the history of the src repo (by creating
# a temporary orphaned branch; isolating all the files from REWRITE_FROM path
# to the root of the repo, commit by commit; and rewriting them again
# to the original path).
#
# Then it creates another temporary branch in the dest repo,
# fetches the commits from the rewritten src repo, and does a merge.
#
# Before any work is done, all the preconditions are verified: all folders
# and branches must exist (except REWRITE_TO folder in dest repo, which
# can exist, but does not have to).
#
# The code should work reasonably on repos with reasonable git history.
# I did not test pathological cases, like folder being created, deleted,
# created again etc. but probably it will work fine in that case too.
#
# In case you realize something went wrong, you should be able to reverse
# the changes by calling `undoImportFolderFromAnotherGitRepo` function.
# However, to be safe, please back up your repos just in case, before running
# the script. `git filter-branch` is a powerful but dangerous command.
importFolderFromAnotherGitRepo(){
    SED_COMMAND='s-\t\"*-\t'${REWRITE_TO}'-'

    verifyPreconditions &&
    cd "${SRC_GIT_REPO}" &&
      echo "Current working directory: ${SRC_GIT_REPO}" &&
      git checkout "${SRC_BRANCH_NAME}" &&
      echo 'Backing up current branch as FILTER_BRANCH_BACKUP' &&
      git branch -f FILTER_BRANCH_BACKUP &&
      SRC_BRANCH_NAME_EXPORTED="${SRC_BRANCH_NAME}-exported" &&
      echo "Creating temporary branch '${SRC_BRANCH_NAME_EXPORTED}'..." &&
      git checkout -b "${SRC_BRANCH_NAME_EXPORTED}" &&
      echo 'Rewriting history, step 1/2...' &&
      git filter-branch -f --prune-empty --subdirectory-filter ${REWRITE_FROM} &&
      echo 'Rewriting history, step 2/2...' &&
      git filter-branch -f --index-filter \
       "git ls-files -s | sed \"$SED_COMMAND\" |
        GIT_INDEX_FILE=\$GIT_INDEX_FILE.new git update-index --index-info &&
        mv \$GIT_INDEX_FILE.new \$GIT_INDEX_FILE" HEAD &&
    cd - &&
    cd "${DST_GIT_REPO}" &&
      echo "Current working directory: ${DST_GIT_REPO}" &&
      echo "Adding git remote pointing to SRC_GIT_REPO..." &&
      git remote add old-repo ${SRC_GIT_REPO} &&
      echo "Fetching from SRC_GIT_REPO..." &&
      git fetch old-repo "${SRC_BRANCH_NAME_EXPORTED}" &&
      echo "Checking out DST_BRANCH_NAME..." &&
      git checkout "${DST_BRANCH_NAME}" &&
      echo "Merging SRC_GIT_REPO/" &&
      git merge "old-repo/${SRC_BRANCH_NAME}-exported" --no-commit &&
    cd -
}

# If something didn't work as you'd expect, you can undo, tune the params, and try again
undoImportFolderFromAnotherGitRepo(){
  cd "${SRC_GIT_REPO}" &&
    SRC_BRANCH_NAME_EXPORTED="${SRC_BRANCH_NAME}-exported" &&
    git checkout "${SRC_BRANCH_NAME}" &&
    git branch -D "${SRC_BRANCH_NAME_EXPORTED}" &&
  cd - &&
  cd "${DST_GIT_REPO}" &&
    git remote rm old-repo &&
    git merge --abort
  cd -
}

importFolderFromAnotherGitRepo
#undoImportFolderFromAnotherGitRepo

Answer 2

If your history is sane, you can take the commits out as patch and apply them in the new repository:

cd repository
git log --pretty=email --patch-with-stat --reverse --full-index --binary -- path/to/file_or_folder > patch
cd ../another_repository
git am < ../repository/patch 

Or in one line

git log --pretty=email --patch-with-stat --reverse -- path/to/file_or_folder | (cd /path/to/new_repository && git am)

(Taken from Exherbo’s docs)

Answer 3

If the paths for the files in question are the same in the two repos and you're wanting to bring over just one file or a small set of related files, one easy way to do this is to use git cherry-pick.

The first step is to bring the commits from the other repo into your own local repo using git fetch <remote-url>. This will leave FETCH_HEAD pointing to the head commit from the other repo; if you want to preserve a reference to that commit after you've done other fetches you may want to tag it with git tag other-head FETCH_HEAD.

You will then need to create an initial commit for that file (if it doesn't exist) or a commit to bring the file to a state that can be patched with the first commit from the other repo you want to bring in. You may be able to do this with a git cherry-pick <commit-0> if commit-0 introduced the files you want, or you may need to construct the commit 'by hand'. Add -n to the cherry-pick options if you need to modify the initial commit to, e.g., drop files from that commit you don't want to bring in.

After that, you can continue to git cherry-pick subsequent commits, again using -n where necessary. In the simplest case (all commits are exactly what you want and apply cleanly) you can give the full list of commits on the cherry-pick command line: git cherry-pick <commit-1> <commit-2> <commit-3> ....

Answer 4

I found this very useful. It is a very simple approach where you create patches that are applied to the new repo. See the linked page for more details.

It only contains three steps (copied from the blog):

# Setup a directory to hold the patches
mkdir <patch-directory>

# Create the patches
git format-patch -o <patch-directory> --root /path/to/copy

# Apply the patches in the new repo using a 3 way merge in case of conflicts
# (merges from the other repo are not turned into patches). 
# The 3way can be omitted.
git am --3way <patch-directory>/*.patch

The only issue I had was that I could not apply all patches at once using

git am --3way <patch-directory>/*.patch

Under Windows I got an InvalidArgument error. So I had to apply all patches one after another.

Answer 5

The one I always use is here http://blog.neutrino.es/2012/git-copy-a-file-or-directory-from-another-repository-preserving-history/ . Simple and fast.

For compliance with stackoverflow standards, here is the procedure:

mkdir /tmp/mergepatchs
cd ~/repo/org
export reposrc=myfile.c #or mydir
git format-patch -o /tmp/mergepatchs $(git log $reposrc|grep ^commit|tail -1|awk '{print $2}')^..HEAD $reposrc
cd ~/repo/dest
git am /tmp/mergepatchs/*.patch

Answer 6

This answer provide interesting commands based on git am and presented using examples, step by step.

Objective

• You want to move some or all files from one repository to another.
• You want to keep their history.
• But you do not care about keeping tags and branches.
• You accept limited history for renamed files (and files in renamed directories).

Procedure

1. Extract history in email format using
   git log --pretty=email -p --reverse --full-index --binary
2. Reorganize file tree and update filename change in history [optional]
3. Apply new history using git am

---

1. Extract history in email format

Example: Extract history of file3, file4 and file5

my_repo
├── dirA
│   ├── file1
│   └── file2
├── dirB            ^
│   ├── subdir      | To be moved
│   │   ├── file3   | with history
│   │   └── file4   | 
│   └── file5       v
└── dirC
    ├── file6
    └── file7

Clean the temporary directory destination

export historydir=/tmp/mail/dir  # Absolute path
rm -rf "$historydir"             # Caution when cleaning

Clean your the repo source

git commit ...           # Commit your working files
rm .gitignore            # Disable gitignore
git clean -n             # Simulate removal
git clean -f             # Remove untracked file
git checkout .gitignore  # Restore gitignore

Extract history of each file in email format

cd my_repo/dirB
find -name .git -prune -o -type d -o -exec bash -c 'mkdir -p "$historydir/${0%/*}" && git log --pretty=email -p --stat --reverse --full-index --binary -- "$0" > "$historydir/$0"' {} ';'

Unfortunately option --follow or --find-copies-harder cannot be combined with --reverse. This is why history is cut when file is renamed (or when a parent directory is renamed).

After: Temporary history in email format

/tmp/mail/dir
    ├── subdir
    │   ├── file3
    │   └── file4
    └── file5

---

2. Reorganize file tree and update filename change in history [optional]

Suppose you want to move these three files in this other repo (can be the same repo).

my_other_repo
├── dirF
│   ├── file55
│   └── file56
├── dirB              # New tree
│   ├── dirB1         # was subdir
│   │   ├── file33    # was file3
│   │   └── file44    # was file4
│   └── dirB2         # new dir
│        └── file5    # = file5
└── dirH
    └── file77

Therefore reorganize your files:

cd /tmp/mail/dir
mkdir     dirB
mv subdir dirB/dirB1
mv dirB/dirB1/file3 dirB/dirB1/file33
mv dirB/dirB1/file4 dirB/dirB1/file44
mkdir    dirB/dirB2
mv file5 dirB/dirB2

Your temporary history is now:

/tmp/mail/dir
    └── dirB
        ├── dirB1
        │   ├── file33
        │   └── file44
        └── dirB2
             └── file5

Change also filenames within the history:

cd "$historydir"
find * -type f -exec bash -c 'sed "/^diff --git a\|^--- a\|^+++ b/s:\( [ab]\)/[^ ]*:\1/$0:g" -i "$0"' {} ';'

Note: This rewrites the history to reflect the change of path and filename.
      ^{(i.e. the change of the new location/name within the new repo)}

---

3. Apply new history

Your other repo is:

my_other_repo
├── dirF
│   ├── file55
│   └── file56
└── dirH
    └── file77

Apply commits from temporary history files:

cd my_other_repo
find "$historydir" -type f -exec cat {} + | git am 

Your other repo is now:

my_other_repo
├── dirF
│   ├── file55
│   └── file56
├── dirB            ^
│   ├── dirB1       | New files
│   │   ├── file33  | with
│   │   └── file44  | history
│   └── dirB2       | kept
│        └── file5  v
└── dirH
    └── file77

Use git status to see amount of commits ready to be pushed :-)

Note: As the history has been rewritten to reflect the path and filename change:
      ^{(i.e. compared to the location/name within the previous repo)}

• No need to git mv to change the location/filename.
• No need to git log --follow to access full history.

---

Extra trick: Detect renamed/moved files within your repo

To list the files having been renamed:

find -name .git -prune -o -exec git log --pretty=tformat:'' --numstat --follow {} ';' | grep '=>'

More customizations: You can complete the command git log using options --find-copies-harder or --reverse. You can also remove the first two columns using cut -f3- and grepping complete pattern '{.* => .*}'.

find -name .git -prune -o -exec git log --pretty=tformat:'' --numstat --follow --find-copies-harder --reverse {} ';' | cut -f3- | grep '{.* => .*}'