Philip Martin <philip@codematters.co.uk> writes:
> Looking at the python code there is a list that is modified in
> multiple threads without locking. I don't know much about python
> threading, but the following patch makes it "work for me":
>
> The tests list needs to be protected by a mutex when running tests in
> parallel.
>
> * subversion/tests/cmdline/svntest/main.py
> (SpawnTest.__init__): Add lock parameter.
> (SpawnTest.run): Use lock when modifying list.
> (run_one_test): Add lock parameter.
> (_internal_run_tests): Create lock, use lock when acessing list.
Having looked at the testsuite code more closely I don't see how the
parallel code can ever work reliably. The problem is that the code
uses spawn_process to run each test in a separate process, and
spawn_process uses a open3/wait sequence. That might work in a single
threaded program[1] but it won't work in a multithreaded program that
starts multiple processes simultaneously as there is no guarantee that
the status obtained by wait corresponds to the process started by
popen3. The python documentation states that the process exit status
is not available when using popen3 and that the Popen3 class should be
used, although I think Popen3 is not available on Windows.
My "works for me" patch doesn't fix the problem, I have realised that
it doesn't even "work for me" all the time. I'm not sure why the
program is losing track of child processes, but since it is attempting
to do something explicitly not supported by Python I suppose we can't
really complain.
I suppose the current code might work if it was switched to use Popen3
on those platforms that support it. Another, perhaps better, approach
would be to dispense with the subprocess and just run the test
directly in the thread, although this might require some additional
code to handle the stdout/stderr output.
Does anyone use --parallel or PARALLEL=1?
1. It could fail in a single threaded program if there is a subprocess
running when the popen3/wait combination is attempted, say if the
program had earlier called popen3 without calling wait.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Sep 11 02:32:00 2007