In this blog, we’ll deal with one of the most popular methods of testing a system or system component thoroughly – namely, white box testing. An effective guide to white box testing, supported by a step by step example. This white box testing example guide teaches you everything you need to know.
Test managers among you will have, by now, been involved with some level of white box testing throughout your career.
“In the name of Agile, I see project teams skimp on a lot of things – chief among them Testing.”
In an increasingly Agile-driven world, techniques like white box testing are fighting for survival. I’ve said this before, and will repeat it here again – pursuing Agile methodologies to improve your IT delivery does not equal to ignoring any activity that needs to be accomplished to deliver your project successfully.
In the name of Agile, I see projects and teams skimp on a lot of things – chief among them testing. white box testing is one of the most affected as a result.
Do not fear. With this post, we’ll set right all the wrongs cast upon white box testing.
We’ll refresh our understanding of white box testing. And how it can help you deliver 100% test coverage when necessary.
We’ll pick up an example and walk you through it step by step.
And at the end of it all, you’ll be able to appreciate the unsung role that white box testing often plays in important projects, and go back with a renewed enthusiasm towards incorporating this and other similarly less attractive but equally effective testing techniques back into your arsenal.
And with that, let’s begin…
This being a refresher of sorts, let us begin by defining the technique.
White box testing refers to a scenario where (as opposed to black box testing), the tester deeply understands the inner workings of the system or system component being tested.
Gaining a deep understanding of the system or component is possible when the tester understands these at program- or code-level. So almost all the time, the tester needs to either understand or have access to the source code that makes up the system – usually in the form of specification documents.
Armed with the level of technical detail that is normally visible only to a developer, a Tester will then be able to design and execute test cases that cover all possible scenarios and conditions that the system component is designed to handle.
We’ll see how this is done in our example later.
By performing testing at the most granular level of the system, you are able to build a robust system that works exactly as expected, and ensure it will not throw up any surprises whatsoever.
The key principles that assist you in executing white box tests successfully are:
And we know path coverage is favoured above branch coverage for the sheer comprehensiveness it provides.
You know by now, that such intensive testing is not for everyone or every situation.
The rigour that white box Testing employs is quite useful – yes, but not all the time.
white box Testing is usually reserved for mission critical systems and components, because, well, such systems simply deserve the attention to detail that this technique can bring.
“The rigour that white box Testing employs is quite useful – yes, but not all the time.”
By mission critical, we mean for instance the Core Banking system that provides the IT backbone to operate a Bank. The core banking system will house all transactions and corresponding customer data, and helps run the bank day-to-day.
Another example could be IT systems that help governments-run defence operations.
Any system that provides such critical utility to a company, organisation or government needs to be bug-free. Any level of bugs or downtime is unacceptable for such systems, as they perform extremely vital functions for the stakeholders involved.
It could quite literally mean a difference between life and death if such systems do not work as expected:
So where critical systems and components are involved, it is necessary to ensure they are bug-free (well, almost). And making a mission critical system bug-free in turn depends on employing extensive testing.
White box testing isn’t the be-all and end-all for critical systems quality assurance. It is, however, one of the central and indispensable techniques.
Simply because of what it is designed to do – i.e., test all possible scenarios that a system can encounter. By testing at the source code-level, a tester will be able to run through every single permutation and combination that the program can theoretically spew out.
When testing happens at such granular level, this then brings any possible defects out in the open. And your team will have an opportunity to evaluate whether some or all of them need to be fixed.
White box testing is a type of testing where the tester can see the code. The main purposes of this type of testing are to test the inner workings of the software, as well as strengthen its security, and improve its usability and design.
This is also known as structural testing as the tester chooses which inputs to test and follows their paths through the software to reach their expected outputs. White box testing is used in the unit, integration and systems phases of software testing.
While this testing method is useful for finding errors in various parts of the software, it can also miss many problems in areas that the tester did not test.
Here are some of the top white box testing tools to use:
You’re no doubt familiar with the now famous white box/black box dichotomy.
Which technique is better than the other?
The answer is – it depends. Depends on the specific project, system, situation.
And as a thumb rule, employing one technique does not preclude using the other.
The Internet will give you varied and colorful opinions from individuals and organisations that favour either the one or the other. And in almost all the cases, the proponents are right – for their given situation, project, system, experience.
I try not to employ favouritism when choosing between white box or Black Box testing. Usually, I choose both.
Black box testing allows me to test a system from the user’s point of view.
By ignoring the inner workings of the system, I can use Black box testing to focus on intended customer/user journeys and test for expected results. Especially during early stages of product development and even for the first few Sprints in a release, Black Box testing allows progress after eliminating ‘show stopper’ bugs.
On the other hand…
White box testing allows me to test a system from the developer’s point of view.
By examining the system using the code specifications, I can bring out bugs that would otherwise not be caught with Black Box or other testing methodologies. And I will be able to fix journeys and scenarios that are normally considered exceptions, but can surface in real life usage and cause a lot of damage – reputational, regulatory, monetary.
With that, let’s sink our teeth into a simple example of white box testing. For this purpose, let’s consider the following sample journey:
A customer needs to transfer money to a friend who lives abroad. They’re going to use the mobile banking service provided by their bank to do this.
Zero in on what you want to test.
When it comes to white box testing, the smaller your target system component, the better it is. Given what we’re trying to achieve – test all possible scenarios and cases for a given feature – individually testing individual features helps you focus on a small enough area of the code.
A narrower focus also implies the ability to be more thorough. Remember, in a lot of the cases, you’re doing white box testing mainly because that specific system, feature, component is critical and needs to be tested through and through to guarantee it works as expected.
“A narrower focus also implies the ability to be more thorough.”
So, you should try and identify the smallest logical module or component for the system being tested, and work on this first. When you’re finished testing this module, you can move on to the next one.
Taking on a larger scope
Yes – it is possible that you’d take on white box testing for entire systems. Where the system involved is critical, draping it in white box testing will give you a high degree of confidence about code quality and resilience.
However, you need to weigh the effort involved versus the benefits derived. white box Testing is often labour intensive and will consume considerable resource. So it is important to balance effort with need.
“White box Testing is often labour intensive and will consume considerable resource. So it is important to balance effort with need.”
This step covers a big portion of your preparation to plan and execute white box Testing successfully.
As with any effort – be it development or testing – understanding ‘Scope’ is paramount.
And, we already know that Path Coverage provides a comprehensive solution to Test coverage.
Here, we’re trying to understand all the possible paths that can be tested for a given feature, component, module… Identifying all possible paths helps in writing test cases to cover each one of them. And the best way to do this, is to draw a flowgraph that brings out these paths.
Consider the example below:
Obviously, transferring funds from your bank account isn’t this simple. Sure, for some of us it can feel like it, but that’s because we don’t normally face the exceptions that quite a few others encounter when attempting to move money.
Having said that, this simple flowgraph will do nicely to demonstrate white box Testing.
To complete the flowgraph, you may need to refer to the following:
Now, as you can see from Figure 1, there are two possible paths for this journey:
Being a simple example, only two paths exist to this journey. Nevertheless, you get the idea. Identify every permutation and combination for how the journey could flow from start to end. Identify any midway drop off points. And you’re all set.
When you have all available paths plotted on the flowgraph, then go ahead and write test cases to test each of these paths.
When you have a bunch of test cases that you are confident will cover every single path, you’re ready to go to the execution phase.
You’re now ready to execute white box Testing for the identified system, component or module. And you have the flow graph and Test Cases necessary to complete Testing.
That, in simple terms, is how you plan and execute white box Testing.
You can also refer to this blog, which does a good job of explaining how to approach white box testing too.
Well, we needed a simple example to demonstrate how white box Testing works. But real life isn’t so simple.
The funds transfer journey above stripped a number of steps and reduced the example to bare essentials in a happy-path scenario. But this isn’t always (probably never) going to be the case.
Now let us consider a more detailed example – although still much simpler than it would be in real life…
As you can see, suddenly the number of combinations possible with this journey increases manifold. As you add further layers of complexity:
You’ll suddenly notice the flowgraph growing dozens of tentacles in all directions that give you hundreds of possible paths. There will be many interactions with systems outside your testing scope (e.g. a customer information system that clarifies the customer’s standing with the bank), that you don’t need to necessarily test as part of your scope. Yet, those links are there and need to be accounted for – with dummy test data, an integrated test environment etc.
And remember, there are still other unexplored ‘exceptional’ or ‘edge case’ journeys that haven’t yet been accounted for. For example, what happens when a customer drops out of the journey – at step 1? at step 2? at step 10?
Hence my suggestion at the beginning to identify the smallest component or module that you can test independently.
You can go further deep, and get to a level where you can ensure statement coverage, i.e., where every single line of code is tested.
Nine times out of ten, I’d say this is going too far.
But then again, it depends on your circumstances. If you’re building software for a rocket ship or a critical government system or protecting your customer data, you may want to implement statement coverage to ensure 100% coverage.
Just make sure you consider the cost and effort involved before you jump into such an endeavour.
Now It’s Your Turn
White box testing represents a solid way for your team to prepare for and execute comprehensive testing when it is necessary.
The key here, is to know when to go for white box testing. Depending on the maturity of the system in question, the tester’s experience with the system, and whether you’re at early stages of a product life cycle or trying to improve code quality, your decision will vary.
Use white box testing wisely. Given the intensive resource engagement, you need to be very sure you need it.
Share your comments and feedback below. Found this blog useful? Why not hit ‘share’ or ‘like’ below to let your friends know?
