When we finished the design of our objects for setting up
the preconditions of an automated test case we showed it to one of our
developers and his first reaction was “it is the same with the app’s domain
model”. This comment made me think that knowing the application internals would
be extremely useful in developing correct test scripts. Knowing this had me thinking that the gray box
testing would be more appropriate in our automation tests than black box, thus
a switch to that direction is worth considering.
Why is gray box more appropriate in automation and not
manual testing? Because any automation tester will have the technical skills to
understand the internals of a software application and to prepare the
appropriate testing code to interface with available hooks or services,
something that a manual (non-programming-savvy) tester would not have.
As a case study I used the current project I was working for.
In this project there was a front end used as an emulator of an external system.
The front-end app is retrieving data from a database. Black box testing ignores
were the front-end app gets its data as long as the displayed data is correct.
While this is acceptable for any functional test case, the big picture is
revealing shady areas of operations. For example for large amount of data we
need pagination (more effort and man hours out of the developers), in parallel
testing there are significant delays due to concurrency issues (front-end was
not designed initially for heavy duty traffic) and in some cases there was data
loss misleading the testing team into opening false defects.
Switching in gray box testing to identify the internals (understanding
how data is written to database and how it is queried by the front-end) was
simple with the usage of some Spring connectors to read the database. The
advantages were the following:
- No more maintenance for the front end emulator
- Parallel testing speed up
- Single point of failure (database not emulator)
Switching to gray box testing made our scripts more robust
and lifted the maintenance effort from the development team therefore becoming
more suitable for our needs. Expanding
the gray box testing technique allowed us to start using services like JMX to
communicate with the app in more accurate and direct way, leading in testing
various paths inside the app’s code that were not exposed to us initially. This
assisted us in revealing not functional but bottleneck-related bugs of the
software.
Gray box testing is the way to go for any automation tester
because it can reveal potential bottlenecks inside the code before the
performance test does and because it simplifies the way test scripts are
created. Gray box testing pushes you to learn more tools to hook into the app
thus expanding your software development skills thus becoming a better
automation engineer.
