[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [RFC] str versus bytes in subversion/tests/cmdline

From: Greg Stein <gstein_at_gmail.com>
Date: Fri, 3 Apr 2009 15:12:25 +0200

On Fri, Apr 3, 2009 at 15:04, Arfrever Frehtes Taifersar Arahesis
<arfrever.fta_at_gmail.com> wrote:
> 2009-04-01 14:20 Greg Stein <gstein_at_gmail.com> napisał(a):
>> On Wed, Apr 1, 2009 at 15:04, Arfrever Frehtes Taifersar Arahesis
>> <Arfrever.FTA_at_gmail.com> wrote:
>>>...
>>>> 2. Internally store the output of commands in str type, decode output
>>>> of commands quickly after obtaining it from subprocess.Popen, convert
>>>> significantly smaller number of strings to bytes type and *ban invalid UTF-8
>>>> characters* in subversion/tests/cmdline.
>>>>
>>>> In this case merge_tests.py 4 test would have to be changed to no longer
>>>> set invalid UTF-8 characters in some properties.
>>>>
>>>> See the attached patch (subversion-svntest-decode_subprocess_output.patch)
>>>> for trunk which implements decoding of outpuf of commands.
>>>
>>> I have decided to implement the improved version of the second
>>> solution. svntest will try to store output of commands in str type,
>>> but will use bytes type for strings with invalid UTF-8 characters.
>>> bytes type will have to be used also when writing to files opened in
>>> binary mode.
>>
>> "bytes type" ?? there is no b"foo" syntax in 2.4, so I don't even know
>> how you're going to start on this.
>>
>>> subversion/tests/cmdline/svntest/wc.py:StateItem.tweak() will contain
>>> workaround for merge_tests.py 4. The properties set by merge_tests.py
>>> 4 (simple_property_merges()) will have bytes type. The expected output
>>> of error message with property values with invalid UTF-8 characters
>>> will depend on Python version.
>>>
>>> See the attached, unfinished patch (subversion-svntest-python-3-v2.patch).
>>
>> It relies on the b"foo" syntax, so how could this be applied?
>
> This part will be applied only on the "python-3-compatibility" branch.

Ah. Fair enough!

> (Python 2.6 supports b"string". In this version b"string" == "string".)
> Changes to function attributes (e.g. func_name -> __name__) will be
> applied also on this branch.
>
> The following changes will be applied on trunk:
>  - os.tempnam() -> tempfile.mkstemp()
>  - file() -> open()
>  - Addition of some calls to variable.encode() / variable.decode()
>  - sys.path.append(os.path.dirname(__file__))
>   It's a workaround for changed behavior of 'import' statement.
>   Changes generated by 2to3 seem to not work.
>  - try:\n  bytes\nexcept NameError:\n  bytes = str
>  - Some other improvements
>
> Workaround for problems (pristine_url) caused by global variables and
> circular imports won't be applied anywhere. Support for the --verbose
> option is still broken.
>
> Anyway, with the newest version of that patch, all tests pass with
> Python 2.6 and Python 3.0!

Cool. Cleanups in there are most welcome! I've been spending some time
in there recently :-( ... and have noticed some pretty ugly stuff. At
some point, I'll take a break from wc-ng again and keep cleaning that
up. Not that it *needs* it (non-core code doesn't have to be Perfect),
but more to demonstrate patterns and set precedent for when people
write more tests. I also have plans for all those globals which should
help to reduce some of the circular dependencies and whatnot.

The problem is in making serious architectural changes. There is just
*so much* code to change.

Cheers,
-g

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1532176
Received on 2009-04-03 15:12:45 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.