SOA Testing Framework
The framework is the foundation of the automation testing; it’s synonymous to the design document of development. It consists of the approach, assumptions, rules and guidelines to manage the whole SOA testing process. It’s the reference point throughout the testing life cycle, it consists of (but not limited to):
· Coding standard
- Structure and flow of scripts
- Script naming convention
- Test Data Management
- Test log Management
- Test reporting Management
- Error and exception handling
- Back-up and recovery Plan
There are different approaches followed to design the framework depending on the nature of application, like:
- Keyword driven Framework
- Modular Framework
- Data Driven Framework
- Hybrid Framework
All the type of framework will have the above mentioned components but their sequence and integration will be different for the different frameworks. All the types have their own pros and cons, so choosing any framework are purely based on need of the application under test.
Let us briefly discuss about all the frameworks, before selecting the suitable one from the SOA prospective.
Key word driven frame work is to create the application flow using the key words, the keywords can be application components or actions. The execution flow is driven by the sequence of key words arranged in a tabular form which covers the whole application functionality. This is suitable for web application with a graphical interface, not very suitable for SOA application to test services.
In modular framework, small scripts are built based on application modules. The script structure replicates the application structure with respect to the different modules and executed as the application flow. This can be used for SOA but more suitable for modular application with number of modules.
Data driven framework emphasises on testing of scenarios based on different set of data. The script uses in built data table or external sources to cover different flows. This is more suitable for SOA application. The main component of testing the SOA application is the services, which accepts the input XML, process them and give a response in the form of output XML. So the input XML tags can be parameterized and as per the business scenarios, the services can be invoked with different set of data. All the SOA testing tools provide the option to parameterize input data using XLS or any other external data source like SQL DB or ODBC connection.
Depending on the application complexity and business flows, any one particular framework may not accommodate all the automation need, so best fit would be the hybrid framework. As the name suggests it’s a hybrid of all of the above approaches. As per the need of the application customize the framework to cover all aspects of it.
Let us see diagrammatically how the framework looks like:
The driver script is the starting point of the execution flow. As per the business flow or application flow, the driver script is designed integrating the scripts. The scripts can be specific to an application module, which in turn calls the validation scripts or the reusable scripts to complete the main validation of the requirement. To execute the scripts driver script need the other parameters like test data, environment details, configuration files and details of specific actions to be taken when any error is encountered during execution.
The driver script runs on the specific tool as adopted for the platform with the add-ins as required. During execution, the execution log is captured which helps to debug the execution and provide more details for logging defects. For SOA, capturing the exception is also very important to identify issues later. Then after the execution of the scripts it should provide the execution report, all the tools provide intuitive graphical execution reports depicting the summary of the execution. Then the report can be attached with the defect report.
The framework should be an evolving one, as you mature with the automation process refine and update the framework. We can work even without a framework, as it’s only a guideline which you may or may not follow, but using a good framework enhances the code modularity, re-usability, portability, maintainability of the automation infrastructure. It helps you to improve the process with time, tracking and analyzing the whole process. This is only a generic guideline; the best framework may not fall into any of the above categorization but should accommodate all your automation need.