Tuning and Testing Enterprise Web Services with SoapUI and JMeter
In this article, I will talk about two test tools for the enterprise web services tuning and testing. These tools can be used to test performance, simulate heavy loads, test functional behavior, monitor servers, as well as find problems and troubleshoot issues with the servers by simulating client requests. Both tools are written in Java. They are free and open source.
The first tool Apache JMeter, is very powerful and has a very generic pluggable architecture which can be used to test a lot of technologies besides web services, such as Java Servlets, CGI scripts, Java objects, FTP servers , JMS, Databases, etc. But for the purposes of this article I'll concentrate on the Web (SOAP) Web services testing aspect of the tool. The second tool, SoapUI is now fully mature and stable tool designed primarily for testing web services. It is also open source, but has a pro (paid) version as well. Both tools are very popular and I'll cover some of the best features on both.
I should also mention that both tools are designed for tuning and testing SOAP based web services, which means they document literal requests that can be sent to the server in the form of XML, and schema or canonicals which needs to be already defined.
Install and Setup
Installation of both tools is very straight forward, but JMeter requires an additional step due the fact that it uses Apache SOAP for service invocation, and that package needs
activation.jar which are not supplied with the JMeter installation.
Since both tools are Java based you will also need JRE to be installed on the system as well. SoapUI comes with the installation package for various operating systems and is self contained.
JMeter comes as an archive for different operating systems. Download and extract the latest JMeter, separately download and place mail.jar and activation.jar into the JMeter lib folder.
You can also set up environment variables for JMeter, which the program will use if its started from the command line without GUI mode.
- JMETER_HOME/lib - used for utility jars
- JMETER_HOME/lib/ext - used for JMeter components and add-ons
To run SoapUI launch the executable from the windows shortcut (ex
To run JMeter, run the
jmeter.bat (for Windows) or jmeter (for Unix) file also in the bin directory.
Click here for larger image
JMeter was originally developed by Stefano Mazzocchi of the Apache Software Foundation. He wrote it primarily to test the performance of Apache JServ (a project that has since been replaced by the Apache Tomcat project). Since then the tool was enhanced with a more robust GUI and redesigned to add functional-testing capabilities.
Out of the box JMeter can currently test: Web - HTTP, HTTPS, SOAP based Web services, Databases via JDBC, LDAP, JMS and Mail - POP3(S) and IMAP(S) interfaces, and can be extended to test other technologies as well. It is sort of a Swiss army knife of application testing, and it is indeed a very robust and flexible tool. Note, if you want to do JDBC testing, you will need to add your database vendor's JDBC driver to the JMeter's classpath.
Setting up test case in JMeter is more involving then in the SoapUI, and I can say that the learning curve with the application is overall higher than with other tools, but thankfully there is a very good documentation online and once you learn the tool, you will appreciate its flexibility.
The first step in making Web services test case, is to set up a test plan, which is a series of steps JMeter will execute when run. A complete test plan will consist of one or more Thread Groups, logic controllers, sample generating controllers, listeners, timers, assertions, and configuration elements. Out of the box, JMeter comes with the sample comptrollers for FTP, HTTP, JDBC, Java object, LDAP, SOAP/XML-RPC Requests and WebService (SOAP) Requests. This means that there is GUI which you can configure to run the test case.
Click here for larger image
Figure 2: Example of the test case for different operations on one SOAP service end point.
A web server HTTP test case plan can be something like this:
- Test Plan
- Thread Group
- Interleave Controller
- Some Operation (HTTP Sampler)
- HTTP default request (Configuration Element)
You should add listeners in your Test Plan in JMeter. Listeners provide access to the information JMeter gathers about the test cases while it runs. For instance, the Graph Results listener plots the response times on a graph, or the "View Results Tree" Listener shows details of sampler requests and responses, and can display basic HTML and XML representations of the response. Other listeners provide summary or aggregation information. All Listeners work on the same data; the only difference is in the way the data is presented on the screen. For more information on listeners in JMeter, please see online reference.
To validate something in the response from the Sampler, you can use Assertions. For example, to check for a presence of a particular XML tag in the SOAP response indicating valid response, or in stress testing a web application. You can add assertions to check for certain HTML/XML tags, common error strings, and so on. You can use regular expressions in JMeter to create these assertions.