Feature to force svn:eol-style native to LF or any other valid value.

680 views
Skip to first unread message

Visenri

unread,
Jun 10, 2021, 6:33:12 PM6/10/21
to TortoiseSVN
Hello everyone.

While working with TortoiseSVN I have sometimes an small problem with eol-style property set to native.

I'm not the only one, the problem is described here by another user:

It happens when working with projects where you don't want to change the eol-style property from "native", because it's ok for other people (or they don't want you to change it).
When working with mixed environments, for example windows using MINGW, some tools expect only LF instead of CRLF, other tools work fine with both CRLF and LF.

So it should be useful to force the transation value for "native" to whatever value we need.
It should be possible to set it to:
"native": taking the value from the system (current behavior).
"Any other value": for example forcing the value to be LF (like in unix), so tools like MINGW could be used without issues.

It should be possible have different settings for different local projects (repositories).

Thank you very much for this great tool. I hope someone can fix this little problem.


Stefan

unread,
Jun 11, 2021, 1:17:02 PM6/11/21
to TortoiseSVN
use the svn:eol-style property:

Visenri

unread,
Jun 12, 2021, 9:23:30 AM6/12/21
to TortoiseSVN

Please, take another look at the description of the problem.
I can't change the eol-style property of the affected files, because the value 'native' is the right one for pure windows and unix users.

The problem is, in windows, sometimes, I work in a mixed environment, with some linux-like tools included in MINGW, so, it is necessary to translate eol-style 'native' to 'LF' not to 'CRLF', what I am asking for is a way to convince TortoiseSVN that I am using unix tools despite running TortoiseSVN in windows.
So, when it will see a eol-style 'native' it will use 'LF' like in a unix environment.

Stefan

unread,
Jun 15, 2021, 12:29:27 PM6/15/21
to TortoiseSVN
On Saturday, June 12, 2021 at 3:23:30 PM UTC+2 Visenri wrote:

Please, take another look at the description of the problem.
I can't change the eol-style property of the affected files, because the value 'native' is the right one for pure windows and unix users.

The problem is, in windows, sometimes, I work in a mixed environment, with some linux-like tools included in MINGW, so, it is necessary to translate eol-style 'native' to 'LF' not to 'CRLF', what I am asking for is a way to convince TortoiseSVN that I am using unix tools despite running TortoiseSVN in windows.
So, when it will see a eol-style 'native' it will use 'LF' like in a unix environment.

you can set the svn:eol-style property to LF for those files you work with linux tools with.
However if you want different eol-styles depending on what tools you use, then that's not possible. I mean how would svn know which tools you like to use today and which ones tomorrow???
 

Visenri

unread,
Jun 15, 2021, 4:29:34 PM6/15/21
to TortoiseSVN
I am telling you, I can't just:
"set the svn:eol-style property to LF for those files you work with linux tools with."
As I said this property is ok as it is for other people and I can't commit this change in svn properties.

My suggestion is very simple:
TortoiseSVN surely has a function that processes each occurrence of 'native' and converts it to CRLF in Windows.
I just need a per project (repository) property to allow me to change the behavior of that translation function when I need Unix tools, so: 'native' -> LF.

I just want to make TortoiseSVN behave like I am in a Unix machine.

If anytime I need to use windows tools that need 'native' translated to 'CRLF', I just change the global setting and next time I update the files I will have files with that conversion done.


Niemann, Hartmut

unread,
Jun 16, 2021, 1:42:24 AM6/16/21
to TortoiseSVN on behalf of Visenri

What I don’t understand:

Are these files of the kind that needs LF style everywhere and the colleagues just use only machines where LF is native,

or are these files needed in LF style on unixoid machines, CRLF style on Windows and you are the only one

who has a use case where he needs them LF although the host is Windows?

 

I believe there can not be a per-file-and-user property svn:eol-style=native_except_for_Visenris_MINGW_workingcopy_where_it_must_be_LF.

What happens if you need a working copy on a linux machine?

Or this would be a per-file-and-workingcopy property – this could probably be done, but I can hardly imagine a good use case for that.

 

(Well actually, I had the use case that I wanted to disable an svn:external locally because you can exclude a file or directory

from a working copy, but not an external – different story.)

 

I would fix that in the build script. Leave the file svn:eol-style=native, have the editors do with it whatever they are used to and

create a temporary file with the needed linefeed style as part of the build system.

 

If, on the other hand, these files are normally used in unix environments, and everybody uses them LF style but

your colleagues refuse to make that explicit by svn:eol-style=LF, then you have a people problem, not a technical one.

 

Hartmut

--
You received this message because you are subscribed to the Google Groups "TortoiseSVN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tortoisesvn...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tortoisesvn/513a116f-186d-4ded-856e-9be24ac58569n%40googlegroups.com.

Niemann, Hartmut

unread,
Jun 16, 2021, 2:19:55 AM6/16/21
to TortoiseSVN on behalf of H. Niemann

Addendum:

If I recall correctly, the cygwin svn command line clients think that native is LF.

Maybe you investigate in that direction and just use mingw or Cygwin tools for managing the mingw WC.

 

Hartmut

Bruce C

unread,
Jun 16, 2021, 1:14:20 PM6/16/21
to TortoiseSVN
As I understand it, the aim is to to checkout a project. Then, depending on tasks that are to be performed, different tools will be used to modify the files. Some tools need Unix line endings and other tools need Windows line endings. All tools are to be used on the same system [presumably Windows since the request is being made for TortoiseSVN that is a Windows only tool].

The observation is that Subversion can checkout the files differently, using different line endings. The suggestion is to use this capability to perform the line ending translations that you require, but this isn't supported as it stands.

Rather than try to change the Subversion, or TortoiseSVN, functionality to fit these specific requirements, perhaps consider using a tool dedicated to the task of changing line endings (e.g. unix2dos). Where the task requires different line endings, use the tool to translate the files, perform the task, and then run the tool to translate the line endings back again. I would expect that the translations could be scripted.

Alternatively, you have suggested that TortoiseSVN "has a function that processes each occurrence of 'native' and converts it to CRLF in Windows". I doubt that's how it works. I would expect that TortoiseSVN simply uses the Subversion functionality. [Stefan provided a link to a description of the Subversion functionality.] If that is the case, the request would need to be directed at Subversion - not TortoiseSVN. Perhaps it might be possible to request a feature that ignores the Subversion EOL property and, instead, uses an override setting. Personally, I don't see that as a great feature but perhaps this is an under-rated use case. However, it seems to me that it'd still be better to perform the conversions directly and this is something you can do without the need to await a change to Subversion.

Hope this helps.

Visenri

unread,
Jun 17, 2021, 2:17:06 PM6/17/21
to TortoiseSVN
I found what I am asking for, but for export:

Causes svn export to use a specific end-of-line sequence as if it was the native sequence for the client platform. ARG may be one of CR, LF, or CRLF.

But it's only available for export, What I am asking for is for update, so when I update I get the unix version of the file.

Using external tools (e.g. unix2dos), I have to keep track of which files need the conversion, and I also lose the original file date.

At least using the svn export all is handled automatically.

I tried it and it works, but it generates an unwanted side effect, all files will have a changed date.

I have not tried to overwrite the working dir (just to update the files with native-eol):
svn export --force ./  --native-eol LF

What I need in crystal clear words:
-Do the update simulating my machine is a UNIX one.
-Do it using the TortoiseSvn GUI if possible.

What I don't want is:
-Keep track of which files need this process.
-Change date of files.

Daniel Sahlberg

unread,
Jun 17, 2021, 3:20:16 PM6/17/21
to TortoiseSVN
As someone already said, the translation occurs deep in the Subversion code used by TSVN. The function svn_subst_translation_required detects if translation is required and then the translation is done by wrapping the read()/write() calls (in the svn_wc__internal_translated_stream function). Thus you might be better off by asking at us...@subversion.apache.org (see http://subversion.apache.org/mailing-lists.html), but I think the answer will be the same as you have already received here. I checked the code quickly and I couldn't see any options related to this.

What you are asking for has been discussed before, for example:
(which was posted to the predecessor of us...@subversion.apache.org). The conclusion of that thread was the same as you have been advised here: Use two separate WCs and use a "local" tool for each WC - TortoiseSVN (GUI or CLI) for the "Windows-wc" and the Cygwin/WSL command line client for the "Unix-wc"

Kind regards,
Daniel Sahlberg


lorenz

unread,
Jun 18, 2021, 1:38:50 AM6/18/21
to torto...@googlegroups.com
Visenri via TortoiseSVN wrote:

>Hello everyone.
>
>While working with TortoiseSVN I have sometimes an small problem with
>eol-style property set to native.
>
>I'm not the only one, the problem is described here by another user:
>https://stackoverflow.com/questions/22901144/error-from-gawk-about-backslash-line-ending-when-running-configure-script
>
>It happens when working with projects where you don't want to change the
>eol-style property from "native", because it's ok for other people (or they
>don't want you to change it).
>When working with mixed environments, for example windows using MINGW, some
>tools expect only LF instead of CRLF, other tools work fine with both CRLF
>and LF.
>[...]

perhapse you could use a combination of manually converting line
endings and a pre-commit hook to prevent checkin of files not matching
the line ending style defined in the repo?.
--

Lorenz

Niemann, Hartmut

unread,
Jun 18, 2021, 3:29:48 AM6/18/21
to TortoiseSVN on behalf of Visenri

Hi Visenri,

 

is this

  • What I don't want is:
  • -Keep track of which files need this process.

the core of your problem?

 

Subversion is a version control *tool*. If I understand you, you do not want a tool and advice how to solve your problem.

You want magic that solves your problem for you, and you do not even want to know what your problem exactly is.

 

I doubt you will find a magic solution in this mailing list.

You were presented different approaches, and we might come up with some more, and you could choose the one that fits your problem best – but only if you want to understand your problem in detail first.

 

You *need* an understanding which files are affected and why. IMHO this is crucial for finding a solution that fits.

 

Hartmut

 

 

Von: Visenri via TortoiseSVN <torto...@googlegroups.com>
Gesendet: Donnerstag, 17. Juni 2021 20:17
An: TortoiseSVN <torto...@googlegroups.com>
Betreff: Re: Feature to force svn:eol-style native to LF or any other valid value.

 

I found what I am asking for, but for export:

 

Causes svn export to use a specific end-of-line sequence as if it was the native sequence for the client platform. ARG may be one of CR, LF, or CRLF.

 

But it's only available for export, What I am asking for is for update, so when I update I get the unix version of the file.

 

Using external tools (e.g. unix2dos), I have to keep track of which files need the conversion, and I also lose the original file date.

 

At least using the svn export all is handled automatically.

 

I tried it and it works, but it generates an unwanted side effect, all files will have a changed date.

 

I have not tried to overwrite the working dir (just to update the files with native-eol):

svn export --force ./  --native-eol LF

 

What I need in crystal clear words:

-Do the update simulating my machine is a UNIX one.

-Do it using the TortoiseSvn GUI if possible.

 

-Change date of files.

Visenri

unread,
Jun 18, 2021, 7:36:49 PM6/18/21
to TortoiseSVN
These comments are exactly what I like about internet, people telling someone that he has no Idea of what he wants when they are the ones who have not understood the problem. (no hard feelings! I am just surprised sometimes by other people answers! :D)

Other people in contrast, understood precisely what I meant, and even provided links to other people requesting the same:
https://svn.haxx.se/users/archive-2009-07/0107.shtml (Thank you very much, Daniel)

I precisely know what I need, because I am doing it manually, we, as programmers, (should) like automate, not to do things manually.
I have precisely expressed what I need:

What I need in crystal clear words:
-Do the update simulating my machine is a UNIX one.
-Do it using the TortoiseSvn GUI if possible.

What I don't want is:
-Keep track of which files need this process (I mean manually, I know which files need this process, the ones with native-eol attribute, I just don't want to check it  manually).
-Change date of files.

If this could be done my process would be:
-TortoiseSvn - Change project settings to do UNIX conversion for files that have --native-eol attribute (1 setting per project or even global option for the program). Do this ONCE each time I need to change the operating mode.
-Just work normally - Update - Commit.
-DONE!!!.

If this was possible, I would not have use another tool: SVN in MINGW command line, export command in SVN or a custom script, and all would be done from the same user interface (TortoiseSvn).

I agree that, after seeing that SVN functionality is provided to TortoiseSvn by the SVN command line utility, maybe, I have asked at the wrong group.

I will ask in SVN or even check the code myself (that normally takes much more time because I am not familiar with the code base, and I have to first understand someone else code, then modify it, deal with the configure scripts and potential building errors, test...You all know). People involved in the project can do this kind of change much quicker.

If SVN ever implement this functionality I may come back to gently ask if it is possible to add this functionality to the GUI.

Tony Rietwyk

unread,
Jun 19, 2021, 12:08:32 AM6/19/21
to TortoiseSVN
Visenri said:

> -TortoiseSvn - Change project settings to do UNIX conversion for files that have --native-eol attribute
> (1 setting per project or even global option for the program).
> Do this ONCE each time I need to change the operating mode.
> -Just work normally - Update - Commit.

When you change this project setting, what do you expect TSVN to do?
- Alter the line endings of all the files in the working copy, without changing their timestamps, and without considering them to be modified.
- Or leave the files as is and end up with a mix of Windows and Unix line endings, after subsequent Updates/Merges/etc.

A quick google shows dos2unix and unix2dos have a -k or --keepdate option.  Have you tried that?

Regards

Niemann, Hartmut

unread,
Jun 21, 2021, 4:13:21 AM6/21/21
to TortoiseSVN on behalf of Visenri

Hi Visenri,

 

well said : „ people telling someone that he has no Idea of what he wants when they are the ones who have not understood the problem. (no hard feelings! I am just surprised sometimes by other people answers! :D)

“

If you understand my post this way I must apologize. I don’t claim that you do not have an idea what you want. But I do claim that you did non express your problem in a way that I can understand. Currently you are trying to discuss a solution, and I am still trying to understand whether you are trying to solve a complex problem where solving a different problem would make solving much easier. Yes, I believe that there is an easy solution that does not involve using SVN the way you are suggesting at the moment and I would love to find it. At a class about root cause analysis I learned that asking “why is this” about five times gives a good chance to find the real core of the problem. I don’t know whether we are at round two or three at the moment 😊

 

You obviously meant something different than I thought when you wrote “Keep track of which files need this process”, sorry for the misunderstanding on my side.

 

 

How many files are we talking about? How big are they? What are they for?

 

What is the detailed use case, if you on your machine need the files stored CRLF locally one day and LF the other day (as you write

“Do this ONCE each time I need to change the operating mode.”)

 

What are these operating modes? What happens if you need both modes simultaneously or within 10 minutes?

What makes you think that switching back and forth is faster, easier und giving correcter results  than having a working copy in each style (TSVN for CRLF style and Cygwin for LF, for example)

and doing a “commit here - update there” cycle when you need to switch operating mode (which can be a PITA – been there, done that)?

What makes you think that converting the files to LF style on the fly when you need them that style is not a viable solution?

 

I have the gut feeling that switching file style from LF to CRLF and back should not be done on the file storage layer – TSVN is (in my point of view) more a versioning file system than a part of my build toolchain. If you dropped TSVN in favor of the old “have a CIFS/SMB/NFS with zip files or subdirectories for each version of the project” or a file versioning system/tool that treats all files as binary and never converts eol styles – how would you handle the EOL problem then?

 

 

But I do not want to convince you to do anything in a certain way. I am trying to discuss and understand a problem as a way to learn more about software engineering and how to solve interesting problems. And if I ask “What …” this is not a rhetorical phrase. I do want to know your reasons and then I want to question and discuss those.

 

 

Hartmut

 

 

Von: Visenri via TortoiseSVN <torto...@googlegroups.com>
Gesendet: Samstag, 19. Juni 2021 01:37
An: TortoiseSVN <torto...@googlegroups.com>
Betreff: Re: Feature to force svn:eol-style native to LF or any other valid value.

 

These comments are exactly what I like about internet, people telling someone that he has no Idea of what he wants when they are the ones who have not understood the problem. (no hard feelings! I am just surprised sometimes by other people answers! :D)

Visenri

unread,
Jun 27, 2021, 5:56:16 PM6/27/21
to TortoiseSVN
Here we go again.
Thank you for apologizing, as I said, no bad feelings.

As I said, I can say it louder, but not clearer:

-I need to emulate a Unix behavior with TSVN.
-When I get files I need to get files using LF for the ones that have the EOL flag set to native.
-No need to check manually which files have the native EOL flag to change all of them to LF (we are talking of tens if not hundreds of files).

I will answer your questions and I hope to clarify the subject (answers in blue):

How many files are we talking about?
-As I said, big projects, not doable by hand.
How big are they?
-Not very big, a few from 1k to a few 500k.
What are they for?
-Source code, makefiles, scripts, I don't care.

What is the detailed use case, if you on your machine need the files stored CRLF locally one day and LF the other day (as you write “Do this ONCE each time I need to change the operating mode.”)

The use case is as I said at least 3 times now:
-Use my windows system with linux like tools with TSVN, no more worry about file conversions, just work normally, update-commit.

-I do not change on a daily basis from one environment to the other, in fact I can keep one folder for each system if needed.
-I just ask for the basic, being able to emulate the unix behavior when I update the folder.
-I really don't know the details of how SVN work, I don't know if the conversion happens in the server or locally, but it does not matter.
-If the server is the one that does the conversion, ok, change the information supplied to it to let it think I work on a linux machine.
-If the conversion happens locally, ok, change the function that does the conversion and emulate the linux behavior.

Doing a “commit here - update there” cycle when you need to switch operating mode (which can be a PITA – been there, done that)?
-In dreamland:
-It would also be useful to have also the feature to change all affected files if I change the option anytime, but this is dreamland:
-I mean change the option, the folder & subfolders get inspected for files with EOL native and a list is presented with a confirmation to do the change.

What makes you think that converting the files to LF style on the fly when you need them that style is not a viable solution?

It is a viable option as long as:

-It is done in an AUTOMATED way, I do not want to waste my time doing this process manually (looking for what files need conversion, the ones with the flag set to native).

-It does not change the file date, flagging a change when i use some external tools (maybe I can live with the date changed, and not changing it may arise other problems, the important part is the automation).


I have the gut feeling that switching file style from LF to CRLF and back should not be done on the file storage layer – TSVN is (in my point of view) more a versioning file system than a part of my build toolchain. If you dropped TSVN in favor of the old “have a CIFS/SMB/NFS with zip files or subdirectories for each version of the project” or a file versioning system/tool that treats all files as binary and never converts eol styles – how would you handle the EOL problem then?

I don't know if I fully understand your point here.

I am using TSVN as a versioning file system, I don't want to go back to files with subdirectories/zips for each version. 

In fact I just want the opposite, 1 folder with version control, forget about the #*$#! EOL and just keep working.

And I can do that if I have the automated tools to do the conversion back and forth.

Whenever I update the folder I need the program to rely on my setting to know what should it do with those files.


If this is not enough to give you a very good Idea of the use case, I give up, I will create my own script using SVN to check the properties, create a log with that, open that file and search for the files with the property set to native and call recursivelly the conversion programs needed to change the files.

But I think it would be much easier to to the simple change to allow the user to decide in TSVN what EOL do you need and forget about the rest.

Visenri

unread,
Jun 27, 2021, 6:13:58 PM6/27/21
to TortoiseSVN
I FORGOT to repeat this:
-I can't change the EOL property on the server/project, it works fine for other build environments, so I can't commit the change in the property.

For the files I meant:
-Hundreds to thousands of files, but not all of them need this conversion, but we have to check them all to check its properties.
-Not very big, from a few KB to some hundreds of KB.

Bruce C

unread,
Jun 28, 2021, 6:08:08 AM6/28/21
to TortoiseSVN
At this point, it seems fairly clear that Subversion (and hence TortoiseSVN) doesn't do exactly what you'd like. If the capability you desire doesn't exist then, it seems to me, all you can do in the short-term is to determine the preferred workaround. This is simply another suggestion as a potential workaround that is offered to provide assistance.

The suggestion seems plausible, to me, but is not one I've tried so you'll need to do some work to confirm that the behaviour operates correctly and as you desire.

Use a WSL session for the checkout, using the Subversion command line tools in WSL. As those tools are effectively operating in a Unix environment, the native EOL will be set for Unix style. Now, as I understand it, that will also cause the pristine copies that Subversion creates to have Unix line endings, so all comparisons will be to those. Having created a working copy on the local file system (albeit from a WSL session), I'd expect TortoiseSVN to also be able to interact with that working copy. That would allow you to use the TortoiseSVN GUI to commit changes. However, it's possible (and perhaps likely) that the plan may fail when TortoiseSVN performs any updates as the native EOLs may get set to Windows style for files that are updated. You'd need to do some testing for the different scenarios to ensure that it worked as you required. If it were me, I'd particular check the behaviour for files that are updated or added (by an update) from TortoiseSVN.

As I recall, WSL2 is much more comfortable sharing files between the WSL and Windows sessions than WSL1. Again, this would be something for you to investigate but I think WSL2 would be a better choice.

Alternatively, if you are OK with using the Subversion command line tools then performing all the Subversion interactions (commits, updates) in WSL would seem to be a safer option. Of course, that doesn't offer the benefits of using the TortoiseSVN GUI and since you have chosen to ask the question here that may be a deal breaker. However, I think the problem is that the underlying Subversion doesn't support your requirement, so it seems unlikely that TortoiseSVN is going to support it until that changes.

I also found another proposal, for the capability you desire, on the Subversion dev list, that is a little newer than the previous reference - 2011 rather than 2009. It is a request but with no feedback or suggestion that anything changed. However, it might be a starting point for a discussion on that Subversion dev list. Once the capability exists in Subversion, I'd expect TortoiseSVN to support that capability soon afterwards. [Isn't Stefan great? :-)]
Reply all
Reply to author
Forward
0 new messages