Phil Holmes | 8 Aug 2012 10:59
Favicon

make test

I've been looking at how the regression test comparison works.  The first 
thing I find is that we have 2 effectively duplicate, but different, pages 
on running regtest comparisons:

http://lilypond.org/doc/v2.15/Documentation/contributor/verify-regression-tests
http://lilypond.org/doc/v2.15/Documentation/contributor/regtest-comparison

I think the latter is probably more accurate.  I think it would be best to 
delete one and point to the other?

I've also been benchmarking.  For example, I know that make CPU_COUNT=9 test 
is _much_ faster than make test, but the make -j9 test isn't worth doing - 
most of the time is spent building the single regtest document, which 
lilypond parallelises much better than make.  I have had errors using -jX so 
am slightly suspicious of that option.  I would like to add the best way to 
parallelise the test process to the CG.

I've also been looking at how output-distance works.  Does anyone now 
understand what this actually does?  From following the code, it looks to me 
like it doesn't actually compare images - it compares the .signature files, 
and if there's a difference over the threshold, it creates an image of the 
original and changed file, and then makes a "ghost" version of the change to 
overlay on the original.  Does this seem correct.  Worth documenting?

--
Phil Holmes

Gmane