What makes me believe that there is something to gain in the statistical approach in general for software is the idea that software components simply aren’t cars.
The idea of a “unit test” is a lovely thing. If it fails, we know for sure something is broken. If it passes, we know for sure, that part works. But the key point is, we do not know whether we’ve covered all areas with tests. So at the end of the day, we just know that with 95% confidence, we have covered everything important with tests and that these tests pass in 100% of the cases we defined. Which makes us know with 95% confidence that the software works.
So why do we insist on having a test that is 100% correct and possibly takes up quite a bit of time to run, if we could probably find a simple way of getting to 95% confidence by some other measure?
The difference about the cars is pretty clear: There we simply do not have the luxury of 100%. No test on a car is a 100% guarantee that something works, ever.
…
So, have you seen this “fastCI” already out there?