[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Migrating repository from Windows to UNIX and eol-style

From: Ryan Schmidt <subversion-2006q2_at_ryandesign.com>
Date: 2006-06-29 19:27:22 CEST

Joe,

Just up front, I want to make sure I'm not coming across as
adversarial in this thread. I certainly don't mean to argue with you
on any points. I just want to try to explain how Subversion is, and
give you recommendations for things I'd do and things I'd avoid. As
long as you're aware of what you're doing, I don't want to prevent
you from following any strategy you'd like. Just want to make sure
you make informed decisions. :-)

Now, on to business:

On Jun 29, 2006, at 16:43, Joe wrote:

> Ryan Schmidt wrote:
>
>> You seem to be under the impression that the presence of CRs in
>> the dumpfile would prevent it from loading on a Subversion server
>> running on BSD. But that should not be the case; there should be
>> no problem loading the dumpfile, on BSD or any other OS.
>
> I'm not sure why you think that because I've mentioned that I *did*
> reload the dump file on BSD and everything was fine, except for the
> EOLs on checkout.

Oh. Ok.

>> I don't think that's a good idea. Your repository is supposed to
>> represent historical data, and you're proposing to rewrite history.
>
> "Rewriting history" is where we disagree. I think of it this way.
> A developer creates a source file on Windows. It's mostly
> transparent to him that each line in the file has a CR-LF. He
> checks the file into a version control system and when he checks it
> out, naturally, he don't see any differences. However, with an
> auto-prop of eol-style native set in his config file, underneath
> the covers, the VCS has mapped each CR-LF to an LF on the check-in
> and vice versa on the check out. Another developer (or the same
> one) creates another source file (perhaps a .h instead of .c and he
> doesn't have auto-prop for .h). Check-in, check-out, everything
> appears to work the same (but the VCS is now storing the .h with CR-
> LFs).
>
> Now the repository is moved to UNIX. The administrator does a dump
> on Windows and a load on UNIX and everything looks good. Then the
> developer does a check out of both files from UNIX. The first file
> is fine, but when he opens the second file in an editor there's all
> these ugly ^M's at the end of each line. To the developer, that
> doesn't look right: he didn't change his source file! Yes, he
> forgot to set auto-prop for the second file (or he didn't know
> about auto-props or whatever), but as an administrator I don't
> think I'd be "rewriting history" if I tried to mask the effects of
> the migration (and the lack of foresight and/or knowledge about
> auto-props) from the developer.

Here's again where I'm getting that impression. Because you again
mention a repository migration from Windows to Unix, and it sounds
like you're implying that this migration would be the cause of the
problems you're describing. But that can't be the case. A Subversion
repository is a Subversion repository is a Subversion repository and
behaves identically regardless of whether the server runs on Mac OS
X, Unix, Linux, FreeBSD, Sun OS, Windows, pocket lint or good
intentions. In the context of the svn:eol-style property, the
component that behaves differently on different platforms is the
client, not the server.

>> What if something breaks because of this change? What if you've
>> inadvertently programmed in a dependency on DOS linefeeds at some
>> point in your application, and your app now breaks due to this
>> conversion? You wouldn't be able to go back in the repository to
>> the last version where you know it worked to compare the changes.
>> All your old versions would be broken too.
>
> I'm sorry but I have trouble imagining how could I program such a
> dependency in the source files. If I do code that, say in C, I'd
> use literal strings like "\r\n" in the statements. Any
> "conversion" that I would be looking for, would leave those
> literals untouched and would only strip the CRs from the end of lines.

Well, you've got it exactly. In the source code of the script, I
might be looking for the literal "\r\n" within a file, let's say
foo.dat. What if foo.dat is also a checked-in file? If Subversion
does some automatic line ending transformation as you suggest, and I
now check out foo.dat on Unix, the file does not contain DOS
linefeeds anymore, and the script, which is looking for DOS linefeeds
in it, fails.

>> I don't see why this problem is any more acute for Windows users
>> vs. users of any other OS.
>
> If you're on UNIX, a text file is stored with LFs on your local
> copy *and* in the repository. If you're on Windows it's stored
> with CR-LF's on your local copy and depending on how you set your
> auto-props it may end up as is or with LFs in the repository.

I'll try to construct a scenario that doesn't involve a repository
migration (since it should not be relevant).

You create a text file on Windows with DOS linefeeds and check it in
without setting svn:eol-style to anything. The repository stores the
file as received, with DOS linefeeds. You check it out on Windows and
it looks right, because it has DOS linefeeds. If you check it out on
Unix or Mac OS X, it looks wrong, because it has DOS linefeeds.

Other way around: you create a text file on Unix with Unix linefeeds
and check it in without setting svn:eol-style to anything. The
repository stores the file as received, with Unix linefeeds. You
check it out on Unix or Mac OS X and it looks right, because it has
Unix linefeeds. If you check it out on Windows, it looks wrong,
because it has Unix linefeeds.

So there's no particular disadvantage to Windows users here. If you
check in a file with one kind of line ending, without setting svn:eol-
style, and then check it out to an environment where another line
ending style is needed, then you've got a problem.

This assumes you do not have an editor that can deal with any kind of
line ending. I would expect anyone working in a cross-platform
environment to have a text editor that can edit files of any line
ending. I use TextEdit on the Mac; my colleagues on Windows use
UltraEdit. There are many alternatives, free and otherwise.

There are two problems that can come up with svn:eol-style=native.

Problem 1. In our setup, we check out working copies from Mac and
Windows workstations to a Samba share served from a Linux server.
Suppose we have a file in the repository with svn:eol-style=native.
We check it out using TortoiseSVN on Windows and write it to the
Samba share. When viewed on Windows, the file has DOS linefeeds,
which are correct. When viewed through the shell on the Linux server,
the same file looks incorrect, because it still has DOS linefeeds.

Other way around: say you check the same directory out while logged
into the Linux server, and you put it in the directory that's shared
via Samba. On the Linux server, the file looks correct with its Unix
linefeeds. If you now look at the file from the Windows machine
through the Samba share, it still has Unix linefeeds, only now that's
incorrect.

Point is: "native" is only native according to the Subversion client
that checked it out. It says nothing about the editor later wanting
to use that checked-out file. What if your working copy is on a Zip
disk which might be inserted into either a Mac or a Windows PC?
"native" is quite confusing in this scenario.

For this reason, we never use svn:eol-style=native. We standardize on
all text files having Unix linefeeds, even when used it on Windows.
Our Windows text editor can handle this, so this is not a problem.
(Editing DOS text files with vi on Unix is a bit of a problem, with
^M appearing at the end of each line as you said, which is why we
didn't standardize on DOS line endings.)

Problem 2. I believe the cygwin environment under Windows considers
svn:eol-style=native to mean Unix linefeeds. So you run into the same
problem if you have a working copy checked out under cygwin and want
to use it within the normal Windows environment, or vice-versa. So in
this scenario, I grant that Windows users are at a disadvantage --
but only if svn:eol-style=native is used.

>> As you know, auto-props are described in the book. Was there some
>> part of that section that was unclear? Or do you feel it should
>> have been mentioned earlier in the book? Version control is a
>> complicated topic, which is why there's a big book describing it,
>> and not everything can be taught at once. You really have to start
>> with the basics and then introduce additional concepts as you go
>> along. Auto-props is not central to the function of what
>> Subversion does, which is probably why it's not mentioned in
>> chapter 1. But if you have suggestions for how the book can be
>> improved, I believe they have their own mailing list.
>
> No, it's not unclear and I agree that it's not something that needs
> to be mentioned up front. However, just like there's an appendix
> for CVS users, perhaps there needs to be a section of gotchas for
> users that share repositories across platforms or that have the
> potential of migrating from one platform to another.

And again it doesn't matter if the repository moves across platforms,
it matters if the working copy moves across platfroms.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Jun 29 19:29:24 2006

This is an archived mail posted to the Subversion Users mailing list.