Why SaaS Automation Testing
Oracle SaaS applications are normally being upgraded quarterly. This frequent upgrade cycle enables clients with rapid response capability to swiftly evolving business environment. On the flip side, it brings additional challenge to QA process.
Most on-premise applications don’t have heavy requirement for regression testing because functionalities are relatively stable after initial build. However, for SaaS application, new features are involved every quarter. Typical business flow may consists of multiple applications, and to ensure the correct behavior of cooperation among multiple applications, regression testing is becoming more important and indispensable.
Manual testing is both time-consuming and cost-consuming. It often involves multiple users to complete a complex test case. It is also tedious and easy to make mistake during the lengthy operation steps.
Automation testing is an alternative option to enhance the QA process at a acceptable cost. It is designed to repeatly execute testing steps to ensure no existing function broken after upgrade or enhancement. It also increases the testing speed and coverage.
Cases Suitable and Not Suitable for Automation Testing
Suitable cases
- Testing cases to be executed repeatedly
- Testing cases which are tedious
- Testing cases which are time consuming
- Testing cases which are high risk and business critical
NOT Suitable cases
- Testing cases which are newly designed and not manually tested for enough rounds
- Testing cases which are executed less-frequently or even on adhoc basis
Why Selenium
Selenium is the most popular free web testing tool. It has following benifits:
- Open source
- Support multiple program languages including Java, Perl, Python, C#, Ruby, Groovy, JavaScript
- Support multiple operations including Windows, Mac, Linux, and UNIX
- Support all mainstream browsers including Internet Explorer, Chrome, Firefox, Opera, and Safari
- Support testing framework in different language. For example, in Java its framework can be integrated with JUnit, TestNG etc.
- Support testing across devices. Selenium Test Automation can be implemented for mobile web application automation on Android and iPhone
- Easy to integrate into Continuous Integration tool chain
- Less hardware requirement, each to deploy in development machine
Data Driven Testing and Keyword Driven Testing
Data Driven Testing
A typical regression testing is to repeatedly testing same business process for multiple times. However, in many cases, same data can not be tested for multiple times. It is necessary to change the testing data before every time the automation testing is executed. It is costly and unnecessary to modify the testing script by only changing the testing data.
An effecient way is to build Data Driven framework to seperate testing process and testing data. Testing data can be stored in spreadsheet or even database. During runtime, testing data are fetched dynamically and fed to testing framework. It is easy to maintain testing data in a dedicated phase to reduce the overall effort required.
Following is a sample of testing data. It is easy to understand and modify before execution.
Keyword Dirven Testing
Selenium provides native support for developing test cases in program languages. However, developing testing cases fully in program language need program skills and not flexible enough.
To increase the maintenance efficiency, it is better to separate the testing process and testing script. An option is to store testing scripts in spreadsheet or even database. During runtime, testing scripts are fetched dynamically and fed to testing framework. It increases both testing scripts composition and maintenance efficiency. It provides better readability to the testing steps. And more important, It enables modulization of testing steps.
Following is a sample of testing script. It is easy to read for what each step has done. It is also easy to adjust incase testing sequence has any change.
Integration with Git, Maven and Jenkins for End to End Automation
Integration Selenium with Git, Maven and Jenkins provides automation build and execution capability. A typical use case is, once testing data or testing framework has been modified and pushed to SCM (source code management system) like Git, the jenkins job is triggerred and testing framework is built immediately using Maven. After that testing is executed and report is sent to stakeholder via mail or other approach. It can also be scheduled at regular frequency to build and test at hardware free time.
For detail implementation of the whole architecture, please refer to the other article: Integrate Git, Maven and Jenkins to Automate Selenium Testing
