How can I fix the SVN import line endings error?

问题:

I have to import an huge SVN repository that I have to transfer from one server to another. So I exported it from the old server:

svnadmin dump . > archive.svn

and imported it on the new one:

svnadmin load . < archive.svn

In the middle of the import process I got this error:

Cannot accept non-LF line endings in 'svn:ignore' property

How can I fix this? I have full control of both servers.

回答1:

Have you changed the server version? This is a known issue in 1.6, and causes problems when going from 1.4 or 1.5.

Subversion 1.6 no longer accepts carriage returns (^M) in property files. You'll need to fix the line breaks in your svn:ignore file(s), or recreate if that's easier.

Alternatively, you could go for Subversion 1.7, or use uberSVN.

回答2:

You have 2 options, repair the source or disable prop validation.

Repairing the source (svn:log and svn:ignore):

sed -e '/^svn:log$/,/^K / s/^M/ /' -e '/^svn:ignore$/,/^PROPS-END$/ s/^M/\n/' archive.svn > repaired-archive.svn

svnadmin load . < repaired-archive.svn

Where ^M is a control character, that means 0D in hex. To get it use ^V^M (control V control M) instead ^M (circumflex M or control M)

Disabling prop validation:

svnadmin load --bypass-prop-validation . < archive.svn

回答3:

The first option of the answer of @ventura10 sounded good but didn't work for me. The sed command changed some versioned content outside the properties section resulting in md5 mismatches while loading the dump.

Because my repository has no properties with binary content I changed the sed command to correct all properties and not only svn:log and svn:ignore. I was also sure that no versioned file contained a line starting with Prop-content-length:. Otherwise I would have got an error when loading the dump.

sed -e '/^Prop-content-length: /,/^PROPS-END$/ s/^M/ /' svn.dump > svn.dump.repaired

It's important to replace ^M with [blank], because the size of the property value must not change.

The note of @ventura10 is still valid:

^M is a control character, that means 0D in hex. To get it use ^V^M (control V control M) instead ^M (circumflex M or control M)

It's hard to believe that upgrading an existing repository from svn 1.4 to svn 1.7 makes this step nessecary, but I've found no other way to get rid of the carriage returns that are no longer accepted since svn 1.6.

回答4:

With a little gymnastics you can work around this using svnsync, which has the ability to fix the EOLs. Let's say that your repository is dumped in archive.svn.

First create the repository to load the repo back, ignoring the EOL problems:

svnadmin create repo
svnadmin load repo < archive.svn --bypass-prop-validation

Now create a new repository for copying into:

svnadmin create repo-fixed

svnsync requires some pre-commit hook, even if you don't use it, so just use your editor to create an empty one in repo-fixed/hooks/pre-revprop-change:

#!/bin/sh
exit 0

Initialize the destination repository for svnsync:

svnsync init file:///path/to/repo-fixed file:///path/to/repo

Now copy the entire repository over:

svnsync sync file:///path/to/repo-fixed

Whew! svnsync will even give you good news: NOTE: Normalized svn:* properties to LF line endings (Why the Subversion team didn't update svnadmin to do the same normalization is a mystery to me.)

Once that's done, dump the new repository:

svnadmin dump repo-fixed > archive-fixed.svn

You now have archive-fixed.svn, which should be identical to archive.svn except the EOLs have been fixed as needed.

(Optional) You can now remove the temporary repository you used for svnsync:

rm -rf repo-fixed

Update It turns out if you load this new dump, your Subversion client gets an error: Repository UUID does not match expected UUID. You'll have to use svnadmin setuuid ... to change the UUID ID to what it used to be.

(This post is a culmination of a multitude of snippets and partial solutions I found around the web. Thanks to all the people who knew more than I did; I just put it all together.)

See also:

• Copying subversion repositories with history using svnsync
• https://stackoverflow.com/a/28472420/421049
• https://svn.apache.org/repos/asf/subversion/trunk/notes/svnsync.txt

回答5:

I ran into this error when upgrading a 1.6 repo to 1.8. I found a number of "Fixes" on the net.

The --bypass-prop-validation did not appeal to me because it postpones the problem, the next time you need to restore the repo you will hit the same promblem.

I found a bash script to loop through the revisions getting the comments and setting them again but this did not work.

An improvement on ventura10's solution did the job. Because of the size of the dump I ended up using the single command to remove the unwanted characters while restoring the dump

    sed -e '/^svn:log$/,/^K / s/^M/ /' -e '/^svn:ignore$/,/^PROPS-END$/ s/^M/\n/' /path/to/svn.dump | svnadmin load /path/to/repo

NOTE:

Where ^M is a control character, that means 0D in hex. To get it use ^V^M (control V control M) instead ^M (circumflex M or control M)

回答6:

Use svndumptool:

svndumptool.py eolfix-prop svn:ignore svn.dump svn.dump.repaired

@tangens solution works too for me, but it's not perfect as I'm getting an extra space character as the replacement of the carriage return characters. I did however test that the svn:ignore still works with that extra space, but I didn't test the other SVN properties.

The downside of using the svndumptool is it can only work on one svn property at one time, which is time consuming if your dump files are big.

---

Some findings

You might be curious on why @tangens didn't replace the ^M with empty character. If you try to replace it with empty character, you will be getting this error:

svnadmin: E140001: Dumpstream data appears to be malformed

The dump file stores Prop-content-length property which will be matched against the content of the actual content of the property. Replacing ^M to an empty character will reduce the property content length, hence the error.

svndumptool will change the Prop-content-length and Content-length respectively.

回答7:

I just repaired a svn dump file successfully. It also had a CRLF in the properties, which caused an Exception in SVN Edge dump import (they have a really bad import routine). Then I "installed" svnserve for testing, which was much easier than expected (instructions for Windows OS):

1. download and install TortoiseSVN or another software that includes svn command line tools. I already had it installed
2. start cmd.exe, run svnserve -d. This cmd window is busy now.
3. start another cmd.execreate a repo: svnadmin create d:\svn_repos\test12
4. load the dump into the repo: svnadmin load d:\svn_repos\test12 < d:\temp\svn\backup_test.dump

svnserve will give you detailed info what failed.

And here's what you have to repair (original on left, repaired on right side):

回答8:

My problem was that the commentary of the commit was too big.