[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: EOL problems in cvs2svn converted repository

From: Johan Corveleyn <jcorvel_at_gmail.com>
Date: Fri, 9 Feb 2018 13:40:43 +0100

On Fri, Feb 9, 2018 at 11:44 AM, Bo Berglund <bo.berglund_at_gmail.com> wrote:
> On Thu, 11 Jan 2018 09:51:54 -0600, Ryan Schmidt
> <subversion-2018_at_ryandesign.com> wrote:
>
>>> I found a section in the conf file named [auto-props] but it seems
>>> only to be applicable for commit and import, not for checkout AFAICT.
>>> And all of that section is commented out by default.
>>
>>If you want Subversion to perform EOL translation on a file, you must set
>>the svn:eol-style property on that file to the desired value.
>>There is no other way.
>
> Really? I imagine that a lot of people have been bit by this and
> someone must have come up with a way to fix the problem without having
> to manually go over all of the files in a repository matching the file
> type (by extension) and set the property.
> Is there no script or such available?
>
>>
>>If you want all files with a particular name or extension to get this
>>property when they are added to a working copy, set that up in the
>>auto-props section of your Subversion client's file.
>>This will apply to that client only and would need to be repeated for
>>all clients / all users.
>
> I have put a Readme.txt file into the repo that describes the way to
> set up the svn client side conf file.
> But it will help only for new files...
>
>>
>>If using Subversion client 1.8 or later, use a repository-dictated
>>auto-props configuration:
>>
>>https://subversion.apache.org/docs/release-notes/1.8.html#repos-dictated-config
>>
>>Auto-props only cause the property to be added to the file when the file
>>is added to the working copy. A user can subvert auto-props, and a user
>>can also delete the property after it was added, before or after the file
>>was committed.
>>
>>If you want to enforce that this property is used on files that need it,
>>and/or prevent users from removing it, write a pre-commit hook that does that.
>
> But this only affects files added to the repo, right?
> I need something that can apply to *all* existing files of the
> affected types.
>
>>If you want all of the files you're importing from CVS that match the criteria
>>to have the property, adjust your cvs-to-svn conversion process to arrange for
>>that to be the case.
>>
>
> This will be a hurdle since the conversion is already done and the
> repository is in use...
> But *how* would one do this if a new conversion has to be done?
> I am not fluent in Python, so any such coding is beyond me...
>
> There is one repository holding a number of projects prepared for use
> by an external contractor, which has so far not been used for
> development. So in theory I can repeat the conversion cvs2svn for this
> with some cvs2svn settings tweak that will set the svn:eol-style
> conversion correctly.
> But I need to know how this can be done in that case.
>
> Is it possible to use dump/load?
> --------------------------------
> Is it possible to somehow edit an svn dump file (with a script) to
> look for all the target files and add the eol-style property setting
> for each file so it will be effected in the svn load?
> When I look at a dumnp file load it looks like what it does is
> automatically performing all the commit operations that was done in
> the source repository....
>
> Or maybe simpler:
> -----------------
> Create dump file of the repo.
> Create a new repo and set its automatic property settings as discussed
> above.
> Then load the dump file.
> Will this make the new repository contain the automatically set eol
> properties? I.e. will the load be affected by the global repository
> automatic property settings?
>
> Note that our repos are each containing a number of "projects" in the
> top level of the repo and all of these have the structure of a svn
> repo with trunk, branches and tags subdirectories.
>
> We use VisualSVN with svn version 1.9.7 on the server

Standard dump/load cannot do this.

However, you might be able to do it using this tool:
https://github.com/jwiegley/svndumptool

(it can manipulate dump files by setting properties and stuff like
that, and performing various eol fixes)

I'm wondering how robust this is though ... make sure to doublecheck
the result (for instance by checking out / exporting some of these
files from different points in history).

The reason why I'm wondering is that, internally (in the repository) a
text file is stored in "normalized form" (which is LF-terminated) if
you have the svn:eol-style=native property set. But if it's not set,
svn doesn't alter the incoming file in any way (it could be binary for
all svn knows), so if that file was committed from Windows, it will
probably be in CRLF format, so that's what will be stored in the
repository. So changing this in historic revisions (in a dump file) is
not just a question of adding the property, it also involves
normalizing the corresponding file contents in the dump (and the dump
format contains checksums and offsets, so manipulating actual contents
in not easy).

-- 
Johan
Received on 2018-02-09 13:41:10 CET

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.