Re: [JGIT PATCH 8/6] Fix zero context insert and delete hunk headers to match CGit

[Robin Rosenberg <robin.rosenberg.lists@dewire.com>]

söndag 03 maj 2009 10:31:30 skrev "Ferry Huberts (Pelagic)" <ferry.huberts@pelagic.nl>:
> > 3) We should have a convention like C Git for marking known breakages.
> > One option is FIXME, another it so go JUnit 4 and abuse the expected exception 
> > annotation (using it for declaring OK exceptions is pretty bad use anyway I think,
> > so we might use it for something better), or perhaps the @Ignore annotation which
> > is meant specifically for this and other cases. A FIXME can be implemented right
> > away.
> 
> standard pratice for junit would be to write a test case on what you would 
> expect to be _correct_ behaviour. obviously that test would then fail.
> it would be a know failure in the test suite. do not go ignoring it. it's 
> better to keep being reminded that stuff doesn't work :-)

What I've see so far is that people start ignoring almost any failure, including new ones, when the test suites contains fails with "known" failues. The assumption is that the failed tests were the same as before.

Worse, automated tests have a hard time telling the difference. Currently I ran
the jgit tests as part of the Eclipse plugin build and I want it to stop if there is a problem that we don't know of. 

"Annotation" of different kinds can be "grepped" for so we can find the broken
cases separately and even refuse completion of release builds if we decide
on that. 

Our primary UI right now is the Eclipse JUnit tests runner and I don't want
to be remined of Shawn's or whoever's bugs when trying to make sure I don't
break anything. Red = *I* broke something or found something new. 

TestNG has a nice way of classifying tests so, we could mark failures as "known failures" and specifically exclude/include them when invoking the
JUnit tests.

Best is to fix before we apply the patches as happened this time. So this problem still remains theoretical :)

-- robin


-- robin