Script based test automation tools have been around since early 1990's. In fact Visual Test 1.0 from Microsoft was released in 1992. And many of the tools we have now – have their genesis of that era. Right from QTP, Rational Functional Tester, Winrunner, and many others leverage types of frameworks to automate GUI testing. These frameworks in the last two decades have undergone drastic change. They have evolved in the pursuit to make test automation better. They have matured to minimize the maintenance overhead, provide highly tested and reliable test scripts, and offer simpler ways to develop test automation. In this article we will limit our scope to these frameworks and revisit why they faded into oblivion.
Record and Playback:
Even though this involves either no or minimal code writing, manageability of automation code makes Record/Playback only viable on small scale. The fundamental problem of this approach is literally the problem of scale. If we want another automated test, we'll record another script which eventually results in two scripts to be maintained as the AUT's interface has changed with time. The more tests we record, the more automation code needs to be maintained – which will become too much of a burden on testing budget. In record and playback each automated test case is a sequence of actions with test data hard coded in to it. This, firstly is not a good approach as per the software engineering principles. Secondly, payback in automation starts when we playback the same automation to test the subsequent versions of the application. But, in this approach each test is run once per release and is not used repeatedly to increase the test coverage. Therefore, to manage such automation code makes this approach impractical for large scale automation. Therefore, it is only due to the cost of maintenance, test automation has moved past Record and Playback.
Data Driven:
In comparison to Record and Playback, Data Driven framework addresses two major pain points: maintainability and test coverage. Test data is stored in a separate file which is read by the script to be used as input to AUT. Each script is programmed and maintained by test specialists but can be used repeatedly with different data sets to increase the test coverage. This also imparts confidence into the reliability of the script. But, testing is not about just inputting data. It is about simulating real life business scenarios to test the AUT thoroughly. Therefore, testers need the facility to specify actual tests. They need an approach to specify which data to use when from which data file.
Keyword Driven:
This takes test automation to the next level. It is not the script now that directs testing but it is the test data itself. The test data with use of keywords sequences out the actions to be followed. When the automated test case runs, it will read through the test data and call for the relevant script specified by the keyword, passing AUT the data for that line. Hence, keywords are scripts written by test specialists to perform all necessary actions to test the business/functional task for which this script was written. With this approach, the testers have the full control over what to do and in which order. But, development of automation code is still AUT specific. The rate and the amount at which the UI of AUT undergoes change so will the maintenance of the scripts. Therefore, for large scale test automation, we still need to find ways to decrease the need to regularly maintain scripts.
UI Object Map Based:
In pursuit to make test automation better, UI Object Map framework solves all the three challenges of test automation. It resolves the maintainability, reliability, and the ease of development of test scripts. This framework takes instruction from test data, recognizes the class of object to be acted upon, and then performs the specified action on that object by calling a script for that particular object class, passing actions and data to it. This means the scripts are no more AUT specific but are UI object class specific. No script for specific instance of an object but for class of the object. Once the script for a class of UI object is written, it can be reused in any automation project where this class of UI object is being used. Through this framework scripts automate the class of UI object. And whenever the AUT undergoes change, you need to only change object map and the data, not the scripts for the UI object class. The only time this needs to be changed is when new UI objects are introduced or when the behavior of existing UI-object-class changes.
