[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [swig-py3][patch] interfacing bytes object instead of str

From: Yasuhito FUTATSUKI <futatuki_at_poem.co.jp>
Date: Tue, 25 Dec 2018 03:10:37 +0900

On 12/17/18 5:50 AM, Yasuhito FUTATSUKI wrote:
> On 12/17/18 2:08 AM, Troy Curtis Jr wrote:

>> Ah yes, I suppose this also has bearing on the discussion in the other
>> thread relating
>> to returning property values.  With that in mind, perhaps your suggestion
>> of effectively
>> defaulting to always being Bytes for char * , svn_string_t, and
>> svn_stringbuf_t unless there
>> is a specific circumstance not to makes the most sense as the general
>> principle.
>>
>> Yasuhito, I suppose that means we should probably tweak the typemaps to
>> follow this
>> principle.  Have you already started down that path based on the properties
>> discussion?
>
> Yes, but there is almost nothing to output, so you don't need to care :)
> I've also read swig document again, especially about default typemaps,
> and also there is no result :)

Status update: I've done to make patch that default to map bytes
from char *, svn_string_t, and svn_stringbuf_t, for swig-py3_at_1849037.

buildi system change:

* build/ac-macros/swig.m4: add "-DPY3" option to SWIG_PY_OPTS for py3
  build to define symbol to switch typemap between py2 and py3

typemap changes:

* subversion/bindings/swig/core.i
  (%typemap(argout) (char *buffer, apr_size_t *len):
   map buffer arg to Bytes insted of Str
  (%typemap(in) (const char *data, apr_size_t *len) ($*2_type temp):
   + map data arg to Bytes instead of Str
   + passing Py_ssize_t variable as length argment of
   PyBytes_AsStringAndSize for for explicit type conversion.
  (%typemap(in) const void *value):
   map value arg to Bytes instead of Str (for svn_auth_set_parameter())

* subversion/bindings/swig/include/svn_global.swg:
  x define SWIG_PYTHON_STRICT_BYTE_CHAR symbol for swig context for
   Python3
  x switch "in" type typemap for char *, char const *, char * const,
   and char const * const "", to use parse parameter "y" for py3 and
   "s" for py2

* subversion/bindings/swig/include/svn_string.swg
  (%typemap(argout) RET_STRING, %typemap(in) svn_stringbuf_t *,
   %typemap(out) svn_stringbuf_t *, %typemap(in) const svn_string_t *,
   %typemap(out) svn_string_t *, %typemap(argout) const char **OUTPUT):
   map args to Bytes instead of Str

* subversion/bindings/swig/include/svn_swigcompat.swg
  (%set_constant(name, value): use PyDict_SetItem and PyBytes_FromString
  instead of PyDict_SetItemString, for SWIG <= 1.3.27 (not tested)

* subversion/bindings/swig/include/svn_types.swg
  (%typemap(in) const char *MAY_BE_NULL): don't use "z" conversion
   format and convert to Bytes if it is not NULL
  (%typemap(in) (const char *PTR, apr_size_t LEN): use PyBytes_Check and
   PyBytes_AsStringAndSize instead of PyStr_Check and PyStr_AsUTF8AndSize

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c
  (make_string_from_ob(), make_svn_string_from_ob(),
   svn_swig_py_make_file(), svn_swig_py_get_commit_log_func()):
   use PyBytes_Check and PyBytes_AsString instead of
   PyStr_Check and PyStr_AsString
  (convert_hash()): use Dict_SetItem and PyBytes_FromString instead of
   DictItemString
  (convert_svn_string_t(), svn_swig_py_proparray_to_dict(),
   ra_callbacks_get_wc_prop()):
   use PyBytes_AsStringAndSize instead of PyStr_AsUTF8AndSize
  (cstring_to_pystring(), convert_string(),
   svn_swig_py_propinheriteditemarray_to_dict(),
   svn_swig_py_proparray_to_dict(),
   svn_swig_py_locationhash_to_dict(),
   svn_swig_py_c_strings_to_list(),
   svn_swig_py_changed_path_hash_to_dict(),
   svn_swig_py_changed_path2_hash_to_dict(),
   svn_swig_py_unwrap_string(),
   svn_swig_py_array_to_list()): use PyBytes_FromString instead of
   PyStr_FromString
  (delete_entry(), add_directory(), open_directory(), change_dir_prop(),
   add_file(), open_file(), apply_textdelta(), change_file_prop(),
   close_file(), parse_fn3_uuid_record(), parse_fn3_set_revision_property(),
   parse_fn3_set_node_property(), parse_fn3_delete_node_property(),
   svn_swig_py_notify_func(), svn_swig_py_status_func(),
   svn_swig_py_delta_path_driver_cb_func(), svn_swig_py_status_func2(),
   svn_swig_py_fs_lock_callback(), svn_swig_py_repos_authz_func(),
   svn_swig_py_repos_history_func(), svn_swig_py_proplist_receiver2(),
   svn_swig_py_log_receiver(), svn_swig_py_info_receiver_func(),
   svn_swig_py_client_blame_receiver_func(),
   svn_swig_py_changelist_receiver_func(),
   svn_swig_py_auth_gnome_keyring_unlock_prompt_func(),
   svn_swig_py_auth_simple_prompt_func(),
   svn_swig_py_auth_username_prompt_func(),
   svn_swig_py_auth_ssl_server_trust_prompt_func(),
   svn_swig_py_auth_ssl_client_cert_prompt_func(),
   svn_swig_py_auth_ssl_client_cert_pw_prompt_func(),
   svn_swig_py_config_auth_walk_func(), ra_callbacks_get_wc_prop(),
   ra_callbacks_push_or_set_wc_prop(), ra_callbacks_invalidate_wc_props(),
   svn_swig_py_commit_callback(), svn_swig_py_ra_file_rev_handler_func(),
   svn_swig_py_ra_lock_callback(), reporter_set_path(),
   reporter_delete_path(), reporter_link_path(),
   wc_diff_callbacks2_file_changed_or_added(),
   wc_diff_callbacks2_file_deleted(), wc_diff_callbacks2_dir_added(),
   wc_diff_callbacks2_dir_deleted(),
   wc_diff_callbacks2_dir_props_changed(),
   svn_swig_py_config_enumerator2(),
   svn_swig_py_config_section_enumerator2()): use 's' for py2 and use 'y'
   for py3 in argment format string
  (ra_callbacks_push_or_set_wc_prop()): use PyBytes_FromStringAndSize
  instead of PyStr_FromStringAndSize
  (ra_callbacks_get_client_string()): use PyBytes_FromString instaed of
  PyStr_AsString

* subversion/bindings/swig/svn_client.i
  (%typemap(argout) apr_array_header_t **props):
   use PyBytes_FromStringAndSize() instead of PyStr_FromStringAndSize

helper python codes changes:

* subversion/bindings/swig/python/svn/core.py
  (svn_path_compare_paths): not to use cmp for py3
  (Stream::read()): chunks is now list of bytes, not list of str

* subversion/bindings/swig/python/svn/fs.py (FileDiff::get_pipe()):
  x make sure self.tempfile1 and self.self.tempfile2 are bytes
  x pass bytes to header_encoding arg in _svndiff.file_output_unified4()

* subversion/bindings/swig/python/svn/repos.py (Callbacks::__doc__):
  treat path as bytes in sample code

* subversion/bindings/swig/python/svn/repos.py
  (ChangeCollector::_make_base_path()): treat path as bytes
  (ChangeCollector::open_root(): path and basepath should be bytes in
   dir_baton
  (RevisionChangeCollector::_make_base_path()): treat path as bytes

test unit changes:

* subversion/bindings/swig/python/tests/auth.py: replace all str literals
  into bytes literals except docstring and module name

* subversion/bindings/swig/python/tests/checksum.py
  (ChecksumTestCases::test_checksum()): check if type of check_val is
   bytes instead of str

* subversion/bindings/swig/python/tests/client.py:
  x replace all str literals into bytes literals except mode arg of open(),
   args to utils.Temper methods, docstrings, and module name
  x (SubversionClientTestCase::test_uuid_from_url): check if return value
   type client.uuid_from_url() is bytes instead of str
  x (SubversionClientTestCase::test_uuid_from_path): check if return value
   type client.uuid_from_path() is bytes instead of str
  x (SubversionClientTestCase::test_merge_peg3()): open result file in
   raw mode
  x (SubversionClientTestCase::test_update4()): convert os.path.sep into
   bytes if it is str in py3

* subversion/bindings/swig/python/tests/core.py:
  x replace all str literals into bytes literals except exception messages,
   args to utils.Temper methods, docstring and module name
  x add new tests for svn_stream_*()
   (new)(SubversionCoreTestCase::test_stream_from_stringbuf()):
    test for svn_stream_stringbuf()
   (new)(SubversionCoreTestCase::test_stream_read_full()):
    test for svn_stream_read_full()
   (new)(SubversionCoreTestCase::test_stream_read2()):
    test for svn_stream_read2()
   (new)(SubversionCoreTestCase::test_stream_write_exception()):
    test for svn_stream_write() to raise exception if try to write unicode
   (new)(SubversionCoreTestCase::test_stream_write()):
    test for svn_stream_write()
   (new)(SubversionCoreTestCase::test_stream_readline()):
    test for svn_stream_readline()

* subversion/bindings/swig/python/tests/delta.py
  (DeltaTestCase::estTxWindowHandler_stream_IF(),
   DeltaTestCase::estTxWindowHandler_stream_IF()):
   x use bytes literals to make stream by using
    svn.core.svn_stream_from_stringbuf()
   x use bytes file name to make stream by using
    svn.core.svn_stream_from_aprfile2()

* subversion/bindings/swig/python/tests/fs.py
  (SubversionFSTestCase::log_message_func(): log_msg_func3 callback now
   expect bytes to return
  (SubversionFSTestCase::setUp()): pass bytes path and url to
   client.import2()
  (SubversionFSTestCase::test_diff_repos_paths_internal(),
   SubversionFSTestCase::test_diff_repos_paths_external()): pass bytes
   path1 to fs.FileDiff::__init__()

* subversion/bindings/swig/python/tests/mergeinfo.py
  (SubversionMergeinfoTestCase::TEXT_MERGEINFO1,
   SubversionMergeinfoTestCase::TEXT_MERGEINFO2,
   SubversionMergeinfoTestCase::MERGEINFO_SRC): use bytes instead of str
  (SubversionMergeinfoTestCase::test_mergeinfo_get()):
   x pass list of bytes for paths arg to repos.fs_get_mergeinfo()
   x replace str in expected_mergeinfo into bytes

* subversion/bindings/swig/python/tests/ra.py
  x replace all str literals into bytes literals except assersion message,
   args to utils.Temper methods, docstring and module name
  x (SubversionRepositoryAccessTestCase::test_get_file): compare file
   content as bytes

* subversion/bindings/swig/python/tests/repository.py
  x replace all str literals to pass into Subversion API or to compare
   with their return values into bytes literals except exception message.
  x (SubversionRepositoryTestCase:: test_dump_fs2()): treat dump and
   feedback as bytes as they are, and compose expected_feedback as bytes

* subversion/bindings/swig/python/tests/utils.py
  (Temper::alloc_empty_dir(),alloc_empty_repo) convert return value of
   tempfile.mkdtemp() into bytes
  (Temper::file_uri_for_path()): return uri as bytes

* subversion/bindings/swig/python/tests/wc.py
  x replace all str literals to pass into Subversion API or to compare
   with their return values into bytes literals except exception message.
  x (SubversionWorkingCopyTestCase::test_get_adm_dir()): check type as
   bytes
  x (SubversionWorkingCopyTestCase::test_get_pristine_copy_path(),
     test_diff_editor4()): open file as raw mode
  x (SubversionWorkingCopyTestCase::test_commit()): use bytes literal
    instead of encoded str

* subversion/bindings/swig/python/tests/trac/versioncontrol/main.py
  (Node::DIRECTORY, Node::FILE): replace to bytes
  (Node::__init__): not to convert path to str
  (Node::get_name): treat self.path as bytes
  (Changeset::ADD, Changeset::COPY, Changeset::DELETE, Changeset::EDIT,
   Changeset::MOVE): replace to bytes

* subversion/bindings/swig/python/tests/trac/versioncontrol/svn_fs.py
  (SubversionRepository::__init__(), get_oldest_rev(), youngest_rev(),
   next_rev()): treat self.path and self.scope as bytes
  (SubversionRepository::normalize_path(), get_node()): treat path as
   bytes
  (SubversionRepository::get_path_history()): replace 'unknown' Changeset
   value to bytes
  (SubversionRepository::get_deltas()): replace 'entry' arg as bytes for
   repos.svn_repos_dir_delta()
  (SubversionNode::__init__()):
   x treat self.path and self.scope as bytes
   x decode path into str for exception message on py3
  (SubversionNode::get_entries()): treat self.path as bytes
  (SubversionNode::get_properties()): property values are bytes, not str
  (DiffChangeEditor::open_root()): return bytes value

* subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py
  x replace all str literals to pass into Subversion API or to compare
   with their return values into bytes literals except exception message.
  x (REPOS_URL) build REPOS_URL as bytes

Cheers,

-- 
Yasuhito FUTATSUKI

Received on 2018-12-24 19:11:40 CET

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.