Re: Overview of ETS 3.0 tests

Robert Kern wrote:
> On Sat, Jul 19, 2008 at 16:58, Ilan Schnell <ischnell@...> wrote:
>> Hi Prabhu,
>> The reason why test_all.py does not pick up the integration tests
>> is because nose does not pick them up.  test_all.py just runs nose
>> in integrationtests as a subprocess.
> 
> Note that things in integrationtests/ directories are not supposed to
> be part of the main test suite. "python setup.py test" should not run
> them. It is a place for tests that need to be manually run for visual
> verification or take very long or other such things that should not be
> automated.

In the strict sense of the term, most of the mayavi2 tests are 
integration tests.  They do however test various modules/filters and I 
see no reason why they shouldn't be automated.  The tests do take a 
while with about 20 of them each taking about 10 seconds.  There isn't 
any visual verification required.  There is one image based test but 
that does not need user intervention either.

So I am unsure what exactly I am to do about these tests.  I can make 
them play nicely with nose which I will do.  However, as it stands, the 
tests also support command line arguments which sometimes conflict with 
nosetests' command line options.

I also have code (run.py) to run these faster by running them all in one 
go rather than individually but that isn't important and can be removed.

I'd like to fix these so the tests are easier to run and follow standard 
operating procedures.  Can you comment on what I need to do.

Right now I am inclined to just make them play nicely with nosetests so 
you can run nosetests in the directory and it runs all the tests.  If 
people need to run it with certain command line options they may do so 
either as python [options] test_something.py or as nosetests [args] 
where the args may be re-interpreted by the test (for example -s will 
run the test standalone but is also nosetests option to not capture stdout).

Comments?

cheers,
prabhu