On Mon, Oct 26, 2009 at 6:44 AM, Bhuvaneswaran A <bhuvan_at_collab.net> wrote:
> On Thu, 2009-10-22 at 14:11 -0500, Hyrum K. Wright wrote:
>>
>> That does seem rather quick. I just ran 'make check' on a separate
>> working copy on the same machine, and got something different:
>> real 32m51.756s
>> user 6m40.321s
>> sys 6m27.504s
>>
>> I don't know that this is a problem with the junit converter script,
>> or perhaps the test running reporting bogus ELAPSED times.
>
> This seem to be a problem with combination of both scripts. As there is
> no way to calculate the time taken for EACH TEST, we calculate it based
> on total time taken. However it is not done correctly.
>
> For instance, consider a test suite that has 10 tests taking 15 seconds
> to complete. We don't know the time taken for 10 tests individually. We
> only know that it took 15 seconds for all 10 tests together. We
> calculate the time taken for individual tests as 1 instead of 1.5. It's
> due to a bug in gen_junit_report.py script.
>
> On the other hand, since we do not account for milliseconds, those tests
> are recorded as 0 ms for individual tests.
>
> The Hudson as such, does not recognise the time specified for each test
> suite in the junit report. Instead it calculates the time taken for each
> test suite based on time taken for each test individually. Due to this
> bug, in the above use case, the Hudson would report 10 seconds, because
> according to the junit report, each test has taken 1 second.
>
> Bottom line, I see two issues to be fixed here:
> a) Include milliseconds in the tests log file, tests.log.
> b) Do not round the result while calculating the time taken for each
> test.
I do not understand how this accounts for such a big discrepancy in
the time reported? For example, as I pointed out, the entire
commit_tests.py reports as taking 0ms. How is including and not
rounding the milliseconds going to suddenly make that reports as the
several minutes that it in fact does take?
As someone else speculated, is it possible that the test setup and
teardown is not being included in the times?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2411373
Received on 2009-10-26 13:54:45 CET