Painful Unit Testing? What went wrong?

Let’s talk about unit tests, do you hate it? The enormous amount of stubbing, verifying, and assertions that are just indecipherable, the incomprehensible scenario titles, those stupid gigantic old legacy unit test codes that are either ignored or no longer maintained, or maybe those useless assertions (or even the opposite, the enormous assertions on a test), it clutters the unit test class making it much bigger than the production code. Admit it, sometimes you only wrote unit tests intending to pass the code coverage check on your build, while the verifies or assertions are defined by the mock objects, making it a false positive.

I have a love and hate relationship with unit testing. It is a way to verify your code correctly without having to start your application and preparing the requests needed to test that code manually. It helps you detect flaws in your code while maintaining the quality. It (should have) makes coding easier… Yet it seems the concept is much more beautiful than most of the execution.

So… what went wrong?

What makes those unit tests painful to maintain or even used? Why does it have so many stubbing, verifies, or even assertions? Why is it not maintained properly? Are unit tests worth the cost of maintaining? What went wrong?

Complex Production Code

Never blame the unit tests, blame the production code. Unit tests reflect the complexity of your production code, which means the more messed up your unit tests are, the more complex your code is.

“But, what if the processes are complex? what can I do?”

Well, first you must ask, what makes it so complex in the first place. Is it the result of patchworks by numerous deadline-driven development? Is it the lack of considerations on the code implementation? Or is it really because it is indeed complex.

There is no fool-proof solution that could solve all your problems at once. Each problem only can be solved by a certain solution with some trade-off in mind.

The Lack of Care on Unit Testing

The Art of Unit Testing is a hard knowledge to master. It is often underestimated and hated sometimes… by those who don’t fully understand the benefits, and sees it as a mere chore for code coverage. How many scenarios do we need? Which data needs to be asserted? How to keep it simple and readable? Does this title describe what scenario it fulfill? Will I be able to understand this code after a few weeks, months, or even years?

These questions are often not answered when creating unit tests. Too often, the lack of care on these small things which then leads to messy unit tests and other disasters. Those negligence lead to more confusing test codes, false positives which incurs more hidden bugs, hindering future developments, and raising the cost to maintain current applications.

The Lack of Knowledge on Unit Testing

There are a lot of frameworks dedicated to unit testing, which helps unit testing easier and maintainable. In the case of Java, you can use JUnit, Hamcrest, Mockito, Selenium, Spring MockMVC, FakeMongo, etc. Any layers of your service can be unit tested if you invest more time learning the advantages and features offered by those frameworks. Understanding the features provided by those frameworks can help your unit tests more robust and clean.

Let’s get a quick look at Mockito’s features, a mocking framework, as an example. You mock other classes methods to return the result you want and verify it was called by using. You can use ArgumentCaptors to verify parameters that are thrown to other methods. You can use Spy to inject dependencies that you don’t want to mock fully or even the class you are testing itself.

You can also learn Design Patterns for Unit Tests or the infamous Test-Driven Development (TDD) which I’ll explain in a minute.

The Myth of 100% Code Coverage

If you think that achieving 100% code coverage means you have good unit tests… well, you are wrong. Code coverage will never reveal the true quality of your unit tests. It doesn’t tell you if you have the correct scenarios or assertions, and I don’t believe it will ever be. Code coverage only reminds you which code has not been covered, not the flow that should have been tested by the unit tests. Writing unit tests that went through all the code is easy, writing unit tests that show what is wrong with your logic is all a different matter.

How to write better and clean Unit Tests

Okay, now you know what makes those unit tests look like a page from “War and Peace” (or maybe the opposite, less than this blog). Let’s see what we can do about it.

The F.I.R.S.T Principle

Have you ever heard of the F.I.R.S.T Principle? If you have read the book Clean Code by Robert C. Martin then you surely would have known. I have mixed feelings about this principle since the book didn’t explain in detail if you handled big data structures. Nonetheless, it’s a good principle to follow. Let’s take a look :

The F is Fast, well a unit test should be fast, you don’t want a unit test to be slow. I have experience with an application with all unit tests taking up to 15 minutes to run, and it is a painful experience. How to make it fast? Well, it depends. It could have been the wrong configuration from the start, or wrong implementation of the production code, or something else entirely.

The I is Independent. Each unit test should run independently. Meaning all each test isn’t dependent on the execution of the test before it. You should be able to easily spot this one.

The R is Repeatable, which means unit tests should be able to run in any environment. It shouldn’t know which server it was running, or which database it is connected to. If you found your unit tests depending on external connections, you should fix it before it became a bigger problem.

The S is Self Validating, and it means that the unit tests knows when the scenario is considered a pass or a fail. In Layman’s Term, assert correctly.

The T is Timely. Well, if you read the book, it says :

The tests need to be written in a timely fashion. Unit tests should be written just before the production code that makes them pass.

Which implies TDD.

The Fight of Priority: Production Code vs Unit Tests

Have you ever wondered why TDD was created? It’s because people created unit tests that follow/after the production code (disclaimer note: this is personal opinion). In my opinion, it should be the other way around.

When creating unit tests, each scenario should be independent of the production code. Unit tests should be the one that says, “I have these inputs, and these methods should be invoked. When I gave these input to the tested method, then I should expect this kind of results”. It shouldn’t be “Oh, the production code have these methods invoked, I should verify them. Oh, the production code will need these kinds of inputs, well here they are. Oh, so it changes the state of these objects into this, better assert it”.

If you wrote the unit tests first, you realize what the goals are and achieve it one by one through multiple scenarios. One by one, the scenarios will shape your production code. This is one of the advantages lies in using TDD.

TDD in a nutshell is :

Create a Scenario (Red Unit Test)-> Implement Production Code (Green Unit Test) -> Refactor -> Repeat

The Possible and Impossible Scenarios

So let’s talk about scenarios. So how much scenarios is good enough? all possible flows of the logic are good enough. If you found yourself making too much-needed scenarios, then you should start to realize that it is either too complex, too many conditions, or too many repeated scenarios. Start by refactoring your code into smaller processes or change the way the process starts.

Don’t write unnecessary scenarios (well, duh). You will only raise the cost of maintenance and confuse people into thinking that it is possible to have that kind of flow since you wrote a unit test for it.

The beginners’ mistakes usually wrote tests that cover impossible scenarios such as empty parameters, excessive null-checks, impossible data invariants, etc. Most of these scenarios are based on the feeling of unease, which is “what if” scenarios that “could” happen in the future.

Small and Big Assertions

Then let’s talk about assertions. Let’s say there are 10 scenarios for a feature, and the object output for this feature has 10 properties. This means you will be writing 100 assertions to make sure the object state is valid in each scenario… Well, that’s a problem.

The problem starts with this question, do you need to assert the whole data integrity in every scenario you have? And here’s my answer, no. Limit the number of assertions in each scenario to one if you can. If you want to verify the integrity of the whole data, then just do it on a single success scenario and assert only the changes that happen in other scenarios.

“But what if someone added changes to the property that is not asserted in the other scenario? We still must verify those properties on the related scenario too right?” Well, no. The reason is the same as why you don’t give null check validations on each method because someone might pass a null parameter to the code in the future. If it does happen, he/she should check the related scenario and added the assertion not relying on the existing scenario to validate his/her changes. If you apply TDD, then you check the related scenario and add the assertion first to make the test fail, then change the production code to make the test pass.

The Curse of Method Verifications

On to the last one, verifications. Should we verify all called methods in each scenario? The answer is, you guessed it, no. This has the same reason as assertions. If you have verified the methods in a single specified scenario, then it is unnecessary to verify it in another scenario if it went through the same process and resulting in the same result when calling that method.

Another mistake people tend to do when verifying methods is overusing verifyNoMoreInteraction to enforce method verification on all scenarios. Why is this a bad practice? This will cause overspecified and less maintainable unit tests. So when should you use this method? Only when you need to do so, which in my case, a single scenario that represents the assertion of the data integrity and method calls. Other scenarios can focus on each specified result.

Conclusion

Initially, I wrote this article with a section dedicated to explaining how to structure the unit tests. However, it was quite long that it could be a new single article. Please, look forward to new updates.

Hopefully, you guys got something from this writing. You may find yourself agree or disagree on the things I wrote, and that’s good. I hoped my writings pique your interest in learning more and explore other possible solutions.

Unit tests are double-edged swords. A well-written unit test will save you a lot of trouble while coding and raise your confidence in your code. Badly written unit tests will raise the cost of development, application maintenance, or even provides false positives.

Further Readings, which I find very interesting:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store