Unit tests are not enough (Part 2)

In a previous post I've explained why integration tests alone will not be enough to create a high quality product. However assuming that unit tests on their own will be enough is also a Mistake I don’t intend to repeat.

There is great value in adding integration tests (and specifically doing writing them before coding is started) that is not gained by writing unit test.

Desired Behavior

Unit tests are testing the actual behavior of a single small component. A good set of unit tests will assure that a unit does exactly what its programmer has intended it do to. However when a programmer does not fully understood the DESIRED behavior, his misunderstanding will not be caught by any unit test HE writes. Writing some integration tests will not only make sure that all units behaved as desired, it will also do wonders to the programmer understanding of the desired behavior.

Real Life

Unit tests are usually executed in a very sterile environment. One of the more common practices when writing unit tests is to isolate the tested unit from all external dependencies by using stubs or mocks. However, stubs have a weird habit of behaving exactly as they are told by the developer, which reflect HIS understanding of how the real component behaves. From time to time this understanding does not reflect the ACTUAL behavior of the component. One cant be sure until the component is tested against the real thing.

None Functional Requirements

By nature unit tests usually focus on functional requirements. It is usually much harder to unit test none functional requirements. For example not only it is hard to test performance on the unit level, in most cases it is kind of pointless. It’s very hard to translate the performance needs of the system into a single testable criteria for a given component. Testing system Performance make much more sense on the system level.

Actually the fact that unit tests are not enough is more or less understood by everyone I've encountered. However, after doing TDD for a long period of time its very easy to forget that simple fact. I was caught (and caught others) more then once falling into the false assurance that a passing suite of unit tests gives.

Posted in: AUT,Integration Tests,TDD,UAT,Unit Tests