Just as money makes the world go round, automation makes the testing world go round.
Automation has gone beyond just a buzz word and has now become a critical part of most organisations backbone to software releases. But the question remains, is Test Automation being done the right way? or an even bigger question should be, is there a right way to do Test Automation? The answer is usually it depends…it depends on so many different factors that come in to play regarding the application under the test, an environment, data, architecture and tons of other factors.
With any major project or task one does need a plan or a sort of strategy to get going with Automation. Many people use Quadrants, pyramids and other models to use as references to get the wheels in motion when deciding on an Automation approach for example for some awesome detail around some great models (including an Automation triangle model by myself). check outthis link.
In my 16 years+ experience in the Test Automation world, my observations and experiences has led me down a path of putting down some of the key elements that goes into consideration when developing a Test Automation Strategy. I will discuss a few of these below.
1. The case of the test case – Test management tools or similar
Lets take it all the way back to the start. Whether you are using a specific Test Management tool or checklists/mind-maps etc in an Exploratory Testing world, there is an underlying aim. The aim to bring some sort of planning aspect or structure to how you tackle a testing task. At this stage believe it or not, is the first place where Test Automation ideas and thoughts should come to mind. You should identify typical automation candidates during the defining of your test cases/scenarios.
The other aspect around a Test management tool or similar is the key element ofvisibility and transparency that it provides to all. Test Automation should not happen in isolation and the better a tools ability to provide full on transparency, traceability and visibility to the team, the more relevant it will be to your test automation effort. Never under-estimate the power this element brings, especially to a developing Agile universe- where team play is critical to the success of any organisation. Everyone should be on the same page regarding your teams automation effort. Visibility seen in the terms of test coverage, test results, test trends and all sorts of fancy automation stats comes to mind here.
2. Where to automate- Show me where!
Another key element regarding an automation strategy is ‘Where’ and ‘What’ to automate. Well in a way the ‘What’ to automate is tackled above, the ‘Where’ to automate in this context is not referring to a specific test environment but rather speaking to the layer of the application to automate. Are we talking automation of Front-end or Back-end, are we talking API or UI or possibly even DB. And then the question of E2E testing comes in to play and even further to that Multiple cross- system E2E tests. As you can see automation involves a lot of thinking before doing. So typically inter-Team and tester discussions are vital to get right before deciding on ‘Where’ to focus your automation efforts
It doesn’t get any easier you know
Now that we have an idea of ‘What and Where’ to automate the even tougher task of choosing the right automation tool for the job becomes key. In some instances a tool is already purchased at a company and automation has to now abide by this pre-selected tool, that scenario provides its own challenges which I wont get into in this post. If destiny was in the testers own hands a then the choice of a tool should span multiple areas. Some quick points that come to mind regarding tool selection are as follows:-
a.) Tool to technology —> can the tool automate our web application? can the tool automate our desktop application? Does it support API automation? How about mobile platforms?
b.) Tool support —> is there support to aid with assistance regarding problems, crashes or tool maintenance. How often are newer versions of the tool released?
4. Environment Stability- keep it tidy
This is one area which is often over-looked yet negligence or instability in this area could almost nullify all you test automation efforts. Think about this, imagine having the flashiest, beautifully crafted new age Yacht! But you take it to the most rockiest waters to sail on. What an epic fail that would be!
This is exactly how it feels or would feel if there is environment in-stability. Questions around environment up-time, dependency availability (whether local or external) needs to be address here. An example that comes to mind here is something like a google API dependency to test log-in feature of an application. If there is interrupted connection to external services and your tests depends on it, then how reliable or trustworthy is your automation.
Finding ways to weed out problems in your environment and coming up with ways to make your environment more stable as you grow your automation efforts becomes critical.
Data is another concept which is so key to an automation approach. Data is complicated and can bring many challenges, this is mostly down to the numerous permutations that could exist per any given scenario. Hard coding automation test inputs and variations are almost frowned upon and rightly so in many instances as the power of your scripts reach to find defects is greatly enhanced by feeding it random data per execution. You chances of finding defects per unique data scenario greatly increases with this approach. In a way this can be seen as a form of automation exploratory testing which is a win-win situation.
So focus on finding ways to randomly generate or obtain different combination of data inputs to strengthen your automation efforts, whether this be in the form of sourcing data from tables, files or api’s, it is definitely worth investigating. Some thing to keep at the back of your mind here is the speed aspect. Would sourcing this data slow down your scripts? By how much will it slow them down? These are calls you would need to make as a team when considering how to approach the data vs coverage topic.
To summarise, It all starts with with how you start. For a well thought out automation approach one has to keep in mind that automation does not just happen. Early automation candidate identification, visibility, team interaction and general planning is imperative to have a solid foundation. Once this is established the other key factors like tool selection, level of automation, data and environment issues need to be addressed to make your automation a success in your world.
Toyer Mamoojee is a 'die-hard' passionate Agile Automation Test Analyst with almost 15 years testing experience across many industries.
From Waterfall to Agile approaches, Commercial to Open Source Automation tools and Manual to Automation Testing he has been through the wars of Software Delivery.
He is the co-founder of Cape Town Automation User group and co-organizer of Cape Town Testing Meetup back in his home country of South Africa.
In his spare time he loves exploring the great outdoors, taking on the world playing online video games , major football fan, adopting the title of technological husband and playing super dad to 2 toddlers!
Presented at Agile Testing Days 2017, Automation Guild 2018 and European Testing Conference 2018
View all posts by Toyer