TestCase4j Core

General

TestCase4j Core contains what you need to manage and execute test cases in a application of your own. This is the home of the test case container and its API. If you are looking for a replacement of NRPE, make sure you read the tutorial: TestCase4j as a NRPE replacement after grasping this page.

Test case logic is implemented as Java classes and consists of one or more steps. Steps are implemented as Java methods in the test case class. Each step defines a task that could be carried out as part of the test case.

Test cases steps (i.e. the Java methods) must begin with the prefix execute, i.e. executeLogin and must take one parameter of type org.testcase4j.core.TestCaseStepConfiguration. By default, the name of the method after the prefix, i.e. Login in the example, can then be referenced in the test case definition file.

Test case implementation logic and test case configuration is separated. Test case logic is, as described above, implemented as regular Java classes. Configuration is described in XML, in the test case configuration file. The XML file is used to configure how the final test case should execute, e.g. order of step execution, steps attributes, etc. Whenever the test case is executed, the first configured step is run. If that step succeeds, the next step in order is executed. This procedure is repeated until all steps are successfully executed or a step fails. A retry policy may be defined for each step, meaning that the step should be re-run upon failure.

Two basic types of test cases exist:

  • stateless
  • stateful

Their characteristics are described in more detail below. The test case container will automatically identify the type of test case.

Stateful test case

  • Shall implement org.testcase4j.core.StatefulTestCase
    • Test cases are instantiated and started once as they are added to the container
  • Provides two methods that can be overridden and that runs whenever the test case is added to the container
    • public void startTestCase() - executed when added to container
    • public void stopTestCase() - executed when removed from container

Stateless test case

  • Shall implement org.testcase4j.core.StatelessTestCase
  • Test case is instantiated for every new client request

Similaries between stateful and stateless test cases

  • Provides two methods that can be overridden:
    • protected void onBefore() - executed before the first step is executed
    • protected void onAfter() - always executed as the last step, regardless of the test case result (equivalent of a finally statement in Java)

Example

Below is a example of a stateless test case implementation.

public class LoginTestCase implements TestCase
{
 
   /** API instance reference used throughout this test case */
   protected API api;
 
   /** */
   protected Operation service;
 
   public Result executeLogin(TestCaseStepConfiguration config) throws Exception
   {
      Result result = Result.createOK();
 
      // Get configuration for this step
      String loginURL = config.getProperty("host");
      String loginDomain = config.getProperty("domain");
      String loginUser = config.getProperty("user");
      String loginPassword = config.getProperty("password");
 
      // Maximum time we will try to connect
      long loginTimeout = config.getTimeout();
 
      // Login to API
      api = API.login(loginURL, loginDomain, loginUser, loginPassword, loginTimeout);
 
      return result;
   }
}

The corresponding configuration for this example may look like:

<testcase name="LoginTestCase" class-name="com.mydomain.LoginTestCase">
   <steps>
      <step name="LOGIN" timeout="1000" retrycount="3">
         <attribute name="host" value="http://www.mydomain.com/serviceA"/>
         <attribute name="domain" value="test-domain"/>
         <attribute name="user" value="foo"/>
         <attribute name="password" value="bar"/>
      </step>
   </steps>
</testcase>

Test case configuration

Test case definitions are read at container start-up and upon re-configuration (modification to the test case definitions file). Each test case is defined by its implementation class, attributes and its steps with their relative order.

A DTD and examples for the test case definition can be found here.

Test case and step result

A step always returns a instance of com.testcase4j.core.Result indicating its result. This class contains two important fields:

  • the actual step result
  • an informative description of the result with a detailed description of the error, etc.

The following step status values are defined:

Status Code Description
OK 0 Should be used if step executes without errors. Execution will continue to next step
Warning 1 Should be used if step is executed with warnings. The step will be re-run until its result is OK or the maximum number of retries is reached (which will stop test case execution and return a warning result)
Failed 2 Should be used if step executes with errors. The step will be re-run until its result is OK or the maximum number of retires is reached (which will stop test case execution and return a error)
Unknown 3 Reserved for future use
Timeout 10 Should be used if the step does not execute within the defined timeout. This will be treated as a step error.

The easiest way of creating a Result instance is to use some of the following static methods in the org.testcase4j.core.Result class:

  • createOK()
  • createFailed(String description)
  • createWarn(String description)

If the result would change during step execution, the following helper methods can be used:

  • setFailed()
  • setFailed(String description)
  • setWarn()
  • setWarn(String description)
  • setTimeout()
  • setTimeout(String description)
  • setOK()
  • setOK(String description)

The test case container

The container holds a list of available test cases and exposes a simple API to manage them. The implementation class is com.testcase4j.core.TestCaseContainer.

Once you get hold of a container you can use it to:

  • Add new test cases
  • Get a list of available test cases
  • Execute a test case, given its name or by providing an instance of TestCaseConfiguration
  • Get meta-data for a test case
  • etc.

Handling exceptions

If an exception occurs in a test case step you got two alternatives:

  1. Declare the step to throw the Exception. The step will automatically fail with a description of the message found in the exception, once executed by the container
  2. Handle the exception and return a modified instance of type Result. This might be usable if you want to allow certain exceptions to be thrown (the step might expect an exception) or when you would like a more fine-grained control over how the exception should be handled with respect to the result.

Below is an example of step that catches an exception and changes the result instance:

...
   public Result executeLogin(TestCaseStepConfiguration config)
   {
      Result result = Result.createOK();
 
      // Get configuration for this action
      String loginURL = config.getProperty("host");
      String loginDomain = config.getProperty("domain");
      String loginUser = config.getProperty("user");
      String loginPassword = config.getProperty("password");
 
     // Login to API
     try
     {
        mgApi = API.login(loginURL, loginDomain, loginUser, loginPassword);
     } catch (APILoginException e)
     {
        result.setFailed();
        result.setDescription("Error logging in to API");
     }
     return result;
   }
   ...

Test cases and logging

By default, using a Log4j logger instance in your test case will make your log output go into the same log file as TestCase4j is using. To avoid cluttering this file with log statements specific to your test case you may configure your test case to write to a separate file.

  • The container automatically defines a Log4j logger for your test case upon startup. The name of the logger is the same as the name of the test case
  • This Log4j logger is created upon server startup, test case modifications (when the server re-reads the configuration) or if requested by the user with a call to method reinitializeTestCaseLoggers of the container.
  • The Log4j logger is hard-coded to be a DailyRollingFileAppender and produces one log file per day for every test case
    • By configuring your test case you may change the name of the log file and how log statements are presented in it. You may also change the log level per test case basis (debug, info, error, fatal).
    • In your test case you can get a hold of your logger by invoking Logger.getLogger(<name of test case>). Since you must know the actual configured name of your test case (as defined in the configuration) you must get a hold of your test case's configuration before you may get to it. For stateless test cases this can be accomplished as in the result below:
...
 
    /** */
    private static Logger logger;
 
    ...
 
    public void init(TestCaseConfiguration configuration)
    {
        logger = Logger.getLogger(configuration.getName());
    }
...
  • In the test case configuration file there is a element for defining a global log configuration (see element testcase-logger-config). These settings will apply to all test case if not overridden.
  • Each test case may override the global log configuration by defining a testcase-logger-config element in its configuration.
    • Attributes left out in a overridden configuration, automatically inherits attributes from the global configuration. One exception exist and applies to the logLevel attribute. If this value is defined for a test case, it is always used. For example, if global configuration defines the logLevel as ERROR and a test case overrides it with DEBUG, DEBUG is used for the test case. ERROR is used for all other test cases without a overridden log configuration in this example.
Unless otherwise stated, the content of this page is licensed under GNU Free Documentation License.