Running user interface, unit and integration tests suites
By Chris Mcmahon
The problem of long running tests
User Interface, (UI) tests and certain kinds of integration tests like SOAP or REST API, are different than unit tests in many ways. For one thing, they take a long time to run. For another thing, they test not only the code, but also the whole environment in which the tests run. Also, they are not coupled to the code itself. We can be working in the test suite without regard for what is happening in the code base; and we can work in the code base without regard for the state of the test suite; and we can run such tests any time, as long as the underlying application has passed all its unit tests and is known to be in good shape.
The best approach I know to running a test suite that takes a long time to run is to do so inside an infinite loop: check out the latest tests; run them; get the test results and publish them; and then do the cycle over again, 24 hours a day, 7 days a week.
Unit testing the Eclipse way – Part 2
In Part 2 of this series, Michael Nyika continues to discusses Unit Testing using Eclipse providing practical Scenarios where jMock can can be used to simulate interface calls.
Scenario 1: Use jMock to mock interfaces
Testing the service method in the ServiceClass class is simple. Suppose the test requirement is to assert that the runService() method did not run — in other words, that the Boolean result returned is false. In such a case, the ICollaborator object passed to the runService() method is mocked to expect a call on its method, executeJob(), and return a string other than “success.” In this way, you ensure that the Boolean string false is returned to the test.
The code for the ServiceClassTest class, which is shown below, contains the test logic.
Continue reading »
Unit testing the Eclipse way – Part 1
In Part 1 of this series, Michael Nyika discusses Unit Testing using Eclipse.
Does your legacy code demand a matching test suite of classes to run against its source code base? For such purposes, jMock qualifies as a great testing framework. However, not all cases may suit the occasion, especially when you must construct objects in a manner that jMock doesn’t expect. To avoid the hassle of producing suites of custom mock objects to support the unit tests in an application, you can tailor RMock to work with jMock seamlessly to achieve a positive result.
Mock objects mimic the behavior of classes written with the sole purpose of guiding code execution so that it falls within those areas under test. In time, the number of mock objects can grow with the number of application classes. Frameworks such as jMock, RMock, and even EasyMock help eliminate the need for a physical and separately existing set of mock objects.
One major disadvantage of the EasyMock framework is that you can’t mock concrete classes — only interfaces. In this article, I show how you can use the jMock framework to mock concrete classes and interfaces, as well as how to test certain obscure cases with RMock.
How to automate testing of graphical user interfaces?
Abstract
This lecture discusses strengths and weaknesses of commercially available Capture-and-Replay GUI testing tools (CR-Tools) and presents a pragmatic and economic approach for testing Graphical User Interfaces using such tools. The results presented were developed within the ESSI Process Improvement Experiment (PIE) 24306 [EU1], [EU2] in 1997/98 at imbus GmbH, Germany [im1].
Today’s software systems usually feature Graphical User Interfaces (GUI’s). Because of the varied possibilities for user interaction and the number of control elements (buttons, pull-down menus, toolbars, etc.) available with GUI’s, their testing is extremely time-consuming and costly. Manual testing of GUI’s is labor-intensive, frequently monotonous, and not well liked by software engineers or software testers. A promising remedy is offered by automation , and several tools for computer-based testing of GUI’s are already commercially available.
Calculating mean time to failure in performance testing
A challenge web application performance testers face is developing metrics for a successful performance test pass. In this tip, I focus on developing the metrics for a mean time to failure (MTTF) test pass.
Mean time to failure is the duration (in time or transactions) after which the system under test is likely to fail. Obviously, the higher the MTTF, the better the application. When devising MTTF metrics or requirements, I calculate my measurements to a lowest-common-denominator. In most cases, this is the transaction.
I define a web application transaction as one request to or one response from the server, regardless of whether the request/response is a user-initiated URL request or an Ajax/RIA request initiated behind the scenes. That’s the beauty of the transaction as a measurement: it’s source-agnostic! Not only is the transaction independent of the underlying implementation, it’s also easily monitored, logged, and measured in sequential order in the web server log.
The first step in developing MTTF metrics is to identify a suite of user scenarios. For instance, let’s assume we’re testing a simple web site which allows users to log in, view their graffiti wall, upload comments to their wall and read comments on their wall from other users. A common user scenario, therefore, might include:
How to apply modeling techniques to support software testing
Modeling is how we build cohesive ideas about the software we’re testing. It’s how we take the random bits of information we gain about the software we’re testing and put them together to build something useful. When we work on projects, our models grow and contract over time. As we learn new information, we test the model with it. If we need to, we incorporate that new information into the model.
As testers, we work with models all the time. Sometimes these models are formal, defined using UML or drawn up in a modeling tool like Visio. Sometimes they are informal, created in your head, drawn on the back of a napkin. Regardless of the format, we use models when we’re learning about the product and when we’re testing the product. When we’re learning, we’re model building. When we’re testing, we’re model checking.
Using models to help generate test cases
When I’m testing, I do a lot of modeling. I actually draw pictures. I draw them in my notebook, on whiteboards, and using tools. They aren’t just in my head. I have a couple of heuristics I use to know when I’m done modeling a testing problem:
- If I can’t derive explicit test cases from my model, perhaps it’s not detailed enough and I need to keep working on it.
- If I can’t explain it quickly to another tester, perhaps it’s too detailed and I need to abstract it a bit more to simplify it.
