Test automation demands a wholehearted approach

If you are considering test automation for your business, you should start by asking why you want to do it. After that, set aside plenty of time, resources and elbow grease if you want to get something out of your investment. So says Mads Loose Winkler, one of Denmark's best test automation consultants.

"The reason why you need to clarify why you want to automate is a simple one. That's because you can't just assume that you will save time or money on manual regression testing by automating.

The company has to be of a certain size because this is a complex process, and it costs money almost whether you use it or not as the automated scripts must be maintained in order to continue to function. In addition, cuts to the manual regression test require that you have set aside time to perform it and that it hasn't already been cut from the budget."

It is incredibly difficult to calculate in terms of time. But you get something additional out of it. You get much wider coverage and faster testing, which in turn means a greater likelihood of running tests that you would not run in a standard release because they had been cut from the budget, or the developers couldn't manage it.

"During the release process there is a testing window. That window may shrink if the release is postponed, or if things don't work the way they are supposed to. By having faster testing, there's a greater chance that the test will take place at all – which is likely to result in a product with fewer errors."

When a company throws itself into test automation, management should be aware that the process begins with a fundamental change in business processes. And this kind of process change will always require focus and time.

There must also be management willingness and someone to drive the process – preferably someone dynamic and enthusiastic – because it also requires finding the right people to do the job.

Mads Loose Winkler explains: ”It's not enough just to go buy automation tools and get a manual tester to go in and automate the company's test. You need someone with specialist knowledge, or believe me, you'll be disappointed in the results – or the lack thereof."

In addition, it is important that testers, test managers, developers and project managers are aware that automation exists, and how best to utilise this function in the company.
It is a form of process change that requires getting used to in the same way as when introducing a new meeting culture.

Mads Loose Winkler explains: "It is not enough to send out an email containing information about the changes – it is a maturing process that demands ongoing
commitment."

TESTING is a maturing process

At ATP, the maturing process is still underway. A process that began before Mads Loose Winkler came to the organisation five years ago.

The organisation has benefited from test automation, and Mads Loose Winkler believes that it has paid off for ATP, but he also feels that there is always room for improvement.

"The work is being done, and a great deal has already been automated at ATP. In a large organisation with many different areas that must interact together, it is not always easy to implement process changes," says Mads Loose Winkler.

He believes that there can be many explanations as to why test automation is difficult to get 100 % anchored in the organisation. The causes can range from failure to inform the rest of the organisation and too little focus on operations to political considerations about the extent of automation and about the resources to be allocated for it.

Centralised or decentralised automation

Whether management of test automation should be centralised or decentralised depends to some extent on the resources you have in the company. In Mads Loose Winkler's view, the greatest benefits come with centralising automation.

It allows the organisation to assemble knowledge with a few employees who can keep track of scripts across the organisation. Furthermore, it is an advantage if the automation
team primarily focuses on development and that all test knowledge resides with the manual testers.

Mads Loose Winkler explains: ”It's hard to find good developers who are also skilled at manual testing, and the automation role quickly becomes very versatile: In addition to developing scripts, there is considerable focus on the running and operation of scripts, including follow-up of errors in the application, adjustment of test data, estimating tasks and general advice on test automation."

There are other benefits to centralising test automation. Solutions and documents become more standardised, and the company has a few experts instead of very diffuse knowledge. In a decentralised solution you quickly have many key people who cannot necessarily take over each other's work because they perform it in each their own way.

Quickly deprioritised

The word 'focus' is something of a mantra for Mads Loose Winkler, for time and again he experiences that at a busy company, test automation lands at the bottom of the priority list. It's something that gets put off until the next release, in the future, so often the resources that should have been devoted to creating ​​scripts are delegated to other, more urgent tasks.

"Generally speaking, one of the major challenges is to get tests that are described in such a way that they can be automated. Many of the test cases that I have seen are constructed more like keywords for someone with specialist knowledge of the application, and as an outsider you can't follow instructions that say: Find a customer and establish a customer agreement," says Mads Loose Winkler.You should know what needs to be done in detail so that you can go in and perform the test.

Operating costs are a surprise to many

It may come as a surprise to some companies that it takes time and resources to operate and maintain the automation suite after the development of automation scripts has taken place.

All scripts should be run on a regular basis to ensure that they still work and are ready for testing when it counts. Changes in environment, test machines, test data, automation software and the application itself can easily cause a script to fail. Preparedness is essential to maintaining a workable automation suite.

Mads Loose Winkler explains: ”For example, at ATP we at one point considered whether we should run all of our scripts every day. But that requires someone looking
at them every day, finding errors and correcting them or removing outdated scripts. It's a resource-heavy process. Some of the errors are not necessarily script or application errors. There could be a problem in the testing environment, and that means finding the right people in the organisation who can help to fix the problem. It's a huge undertaking."

Benefits of automation

• Save manual hours during regression testing

• Avoid manual errors during repetition testing

• Reduced need for resources with specific knowledge

• Potential for greater test coverage

• Faster response during regression testing

• Regression testing is maintained

• Errors are identified sooner