Re: Updating Tktest suite - GSoC project

Ania Pawelczyk wrote:
> I'm Ania Pawelczyk, a student from Poland who participates in Google 
> Summer of Code and works on updating Tk test suite.

Thanks for taking on the project.

I think you should establish for yourself and for the sake of others
trying to offer help a clear sense of what you want to achieve.  The
Tk test suite is large, and the summer is short, so it's best if
you focus efforts that most effectively lead toward your goals.

 From my perspective, the most important problems with the Tk test
suite is that it fails its primary purpose - helping find and fix
bugs in Tk.  This is because 1) There are tests that fail, where
that failure does not imply an actual bug in Tk; and 2) There are
tests that fail, which do imply an actual bug in Tk, but don't
clearly point to just where that bug is.  I think the most valuable
"updating" for the Tk test suite would try to tackle that.  I would
also aim for these corrections to get into releases of Tk starting
with future patch releases of Tk 8.5.

However, it's your project, so you choose the goals.  Just communicate
clearly to others what they are.

> 1. Failing tests
...
> I usually work by assuming that the test that passes on one machine but 
> not on the others has probably "good" - I mean required - settings and 
> the failing ones lack something or take wrong assumptions. I compare 
> configurations and try to find what p.e.
> - wasn't set, but matters in the result and may have different default 
> values
> - was somewhere wrongly assumed (like in 
> http://www.assembla.com/flows/show/brq6hCsrar3BB4ab7jnrAJ)
> Is this alright?

I think it's fair to assume every test probably passed for the
developer who first added it in their usual development environment.

If a test is failing now, that can be for many reasons.  A common reason
is that there was an unstated assumption made by the test which is no
longer true, or not true on a different platform, etc.  Test constraints
can deal with that situation, assuming the test is sensible in the first
place.

That's really the most difficult part of test suite maintenance, especially
many years after the fact: trying to discern just what was being tested, and
whether it was a sensible test in the first place.  As we often observe in
the chatroom, the test suites include both tests and experiments.  Only
the tests are really useful.  There's little point in tidying up the
experiments.  Much effort for little value.

> 3. Individual test's structure
> Furthermore, should I also restructure tests? So that they'd follow the 
> pattern that is presented in man tcltest page?

Here is where clarity of your goals is important.  I would love to see
this task done.  Getting the test suite(s) updated to use the features
of tcltest 2 would be great!  (Including tcllib even better!).  But in
the end, this is a semi-mechanical conversion process that doesn't make
the test suites fundamentally better than they are now.

So, I say do this if you decide this is your goal.  Or if it's not your
goal, do it if you can do it with reasonably small effort so that progress
toward your goals isn't impeded.  Otherwise, don't worry about it, and just
be sure to use modern conventions in any new tests you create.

> 4. Wiki page
> I added my project's wiki page where I put my ideas (marked(i)) and 
> questions (marked(?)).

I think it's a mistake to set this work aside somewhere.  If you're
going to work on updating the Tk test suite, let's get you registered
as a Tk maintainer, and you can start committing to the HEAD and
core-8-5-branches.  You'll get a lot more and better review of your
work if you're hacking the actual Tk sources than you will over in
your own space somewhere.

--

-- 
| Don Porter          Mathematical and Computational Sciences Division |
| donald.porter@...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08