Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C

On Tuesday 04 July 2006 10:59, Malcolm Rowe wrote:
> On Tue, Jul 04, 2006 at 10:16:41AM +0200, Denis Vlasenko wrote:
> > On Monday 03 July 2006 18:33, Malcolm Rowe wrote:
> > > > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
> > > > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='5.Р╖Ь(Д╖╪JЦ╞ .С╖^M'
> > >
> > > ... could you get a backtrace from the svn_io_file_open() call that opens
> > > this file? Yes, if we're trying to open a file with a garbage name,
> > > that's probably a bug.
> >
> > The call chain is:
> >
> > svn_io_file_open():
>
> I was actually more interested in a backtrace - i.e. where the call was
> coming from. But it occurs to me that there's no way we can tell that
> the string above is actually supposed to be a filename, so that probably
> doesn't help.
>
> > The filename is passed to svn_error_wrap_apr() in third argument,
> > but svn_error_wrap_apr() is a variadic function and:
> >
> > svn_error_t *
> > svn_error_wrap_apr (apr_status_t status,
> > const char *fmt,
> > ...)
> > {
> > svn_error_t *err, *utf8_err;
> > va_list ap;
> > char errbuf[255];
> > const char *msg_apr, *msg;
> >
> > err = make_error_internal (status, NULL);
> >
> > if (fmt)
> > {
> > /* Grab the APR error message. */
> > apr_strerror (status, errbuf, sizeof (errbuf));
> > utf8_err = svn_utf_cstring_to_utf8 (&msg_apr, errbuf, err->pool); // <---- HERE
> > if (utf8_err)
> > msg_apr = NULL;
> > svn_error_clear (utf8_err);
> >
> > /* Append it to the formatted message. */
> > va_start (ap, fmt);
> > msg = apr_pvsprintf (err->pool, fmt, ap);
> > va_end (ap);
> > err->message = apr_psprintf (err->pool, "%s%s%s", msg,
> > (msg_apr) ? ": " : "",
> > (msg_apr) ? msg_apr : "");
> > }
> >
> > return err;
> > }
> >
> > See? we call svn_utf_cstring_to_utf8() but it has no chance in hell
> > to get a filename, because va_start() is not yet called,
> > and therefore it has no way to obtain 3rd argument.
> >
>
> No, that's fine. The generated error message is in the form
> "<fmt,variadic arguments>: <APR error message>". In our example, it
> might be something like "Can't open file 'foo': File not found".
>
> The first section (including the line you highlighted) only looks up
> the APR error message, which doesn't contain any format strings, and
> doesn't reference the filename.
>
> > I inserted a fprintf(stderr, "%s: errbuf='%s'\n", __FUNCTION__, errbuf)
> > directly after apr_strerror (status, errbuf, sizeof (errbuf))
> > call and I see this:
> >
> > svn_io_file_open: fname='busybox/.svn/dir-props'
> > svn_error_wrap_apr: errbuf='ФЁ╧╖юдд╖.╗Ю╖..╟╞h.╟╞A.Ю╖юдд╖.╗Ю╖Л╖Ю╖a.'
>
> That is interesting though. I'd be interested to know what 'status' is at
> this point, and whether errbuf is just what was in the buffer originally,
> or whether it's been filled by apr_strerror() [and in that case, is it
> possible that it's the native error message in the native codepage?].

Added a bit more debugging. In ru locale:

svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=6 outbytes_left=0 translated=-1 errno=7 inbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=64 translated=0 errno=7 inbuf='л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=39 translated=0 errno=7 inbuf='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='╖л.Б╖a.'
...infinite loop...

NB: in C locale it _also_ has some garbage in errbuf, but iconv()
returns different errno and therefore we do not loop.

svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=38 outbytes_left=76 translated=-1 errno=84 inbuf='Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=39 translated=0 errno=84 inbuf='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=38 outbytes_left=76 translated=-1 errno=84 inbuf='Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=35 translated=0 errno=84 inbuf='busybox/.svn/props/AUTHORS.svn-work'

and it continues!

My native_strerror currently looks like this:

static char *native_strerror(apr_status_t statcode, char *buf,
                             apr_size_t bufsize)
{
    if (strerror_r(statcode, buf, bufsize) < 0) {
fprintf(stderr, "%s: strerror_r()<0, return stuffbuffer('APR does not understand this error code')\n", __FUNCTION__); //vda
        return stuffbuffer(buf, bufsize,
                           "APR does not understand this error code");
    }
    else {
fprintf(stderr, "%s: return '%s'\n", __FUNCTION__, buf); //vda
        return buf;
    }
}

Hmm.... "man strerr_r" says:

       #include <string.h>
       char *strerror(int errnum);
       char *strerror_r(int errnum, char *buf, size_t buflen);
                               /* GNU-specific strerror_r() */

       #define _XOPEN_SOURCE 600
       #include <string.h>
       int strerror_r(int errnum, char *buf, size_t buflen);
                               /* XSI-compliant strerror_r() */
...
       The XSI-compliant strerror_r() is preferred for portable applications.
       It returns the error string in the user-supplied buffer buf of length
       buflen.

       The GNU-specific strerror_r() returns a pointer to a string containing
       the error message. This may be either a pointer to a string that the
       function stores in buf, or a pointer to some (immutable) static string
       (in which case buf is unused). If the function stores a string in buf,
       then at most buflen bytes are stored (the string may be truncated if
       buflen is too small) and the string always includes a terminating null
       byte.

Can this be the reason?

--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org