XML Test Suite Configuration¶

intaQt Client supports the execution of more complex test suites by means of an XML configuration file describing the test suites.

XML Test Suite Configuration
Example Configuration
Test Suite Execution
Test Suite and Test Cases Tags
Variables
Pass Configurations to Test Case
Delay Between Test Executions
Retry Failing Test Cases
Aggregate Test Results

Example Configuration ¶

The following example shows a file that may be named intaqt-config.xml, which holds a test suite's configuration.

<?xml version="1.0" encoding="UTF-8"?>
<TestSuiteCollection name="Test suites">
    <Description>
        A description of the test suites comes here
    </Description>

    <TestSuite name="testsuite 1">
        <TestCase name="testcase 1">
            <Description>description of testcase 1</Description>
            <TestScript name="intaqt" file="ts1-tc1.feature" workdir="ts1-tc1-report"/>
        </TestCase>
        <TestCase name="testcase 2">
            <Description>testcase 2</Description>
            <TestScript name="intaqt" file="ts1-tc2.feature" workdir="ts1-tc2-report"/>
        </TestCase>
        <Variables>
            <Variable name="Author" value="author"/>
            <Variable name="Jira Story" value="TCBI-11879"/>
        </Variables>
    </TestSuite>

    <TestSuite name="testsuite 2">
        <TestCase name="testcase 1">
            <Description>testcase 1</Description>
            <TestScript name="intaqt" file="ts2-tc1.feature" workdir="ts2-tc1-report"/>
        </TestCase>
        <TestCase name="testcase 2">
            <Description>testcase 2</Description>
            <TestScript name="intaqt" file="ts2-tc2.feature" workdir="ts2-tc2-report" retries="3"/>
        </TestCase>
        <Variables>
            <Variable name="Author" value="author"/>
            <Variable name="Jira Story" value="TCBI-11879"/>
        </Variables>
    </TestSuite>
</TestSuiteCollection>

Test Suite Execution ¶

Given a project called my-project, the following command can be run by intaQt Client:

1	intaqt test my-project intaqt-config.xml

This will make intaQt Client execute the configured test cases inside my-project. In this case it will execute ts1-tc1.feature, ts1-tc2.feature, ts2-tc1.feature and ts2-tc2.feature, which are configured in intaqt-config.xml.

For each of those test cases, intaQt Client will store the corresponding report artifacts inside the directory specified by the workdir attribute.

Test Suite and Test Cases Tags ¶

The XML configuration file allows for specifying an arbitrary number of TestSuite tags and an arbitrary number of TestCase tags within them.

Variables ¶

intaQt Client will also process the Variables inside test suites and will place them in form of key-value pairs inside the junit report file's <system-out> tag.

Example

<?xml version="1.0" encoding="UTF-8"?>
<testsuite>
  <system-out>
    Author: author
    Jira Story: TCBI-11879
  </system-out>
</testsuite>

Pass Configurations to intaQt Client ¶

Configurations may be passed to test cases inside of the TestScript using the Config within TestScript. These will be interpreted as if they are .conf files. The node's text content is used as an additional configuration for the execution. If a --config or -c command line option was specified, it will be appended to the configuration provided in the Config node.

Note
Only a single `Config` node bay be used within each `TestScript`.

Example Executing the following XML control file without specifying a --config command line option will send ContextObjects.test=test as an extra configuration to intaQt.

<?xml version="1.0" encoding="UTF-8"?>
<TestSuiteCollection name="Testsuite with config">
    <Description>
        Testsuite with config
    </Description>
    <TestSuite name="testsuite with config">
        <TestCase name="testcase">
            <Description>testcase</Description>
            <TestScript name="intaqt" file="passing.test" workdir="suite-with-config-report">
                    <Config>ContextObjects.test=test</Config>
            </TestScript>
        </TestCase>
    </TestSuite>
</TestSuiteCollection>

If -c ContextObjects.test=test2 -c abc=def were specified as command line options, the extra configuration below will be sent to intaQt:

1
2
3

ContextObjects.test=test
ContextObjects.test=test2
abc=def

Delay Between Test Executions ¶

Specifying the Delay between test cases is available as an additional attribute of the TestScript called delay This attribute specifies the number of seconds intaQt Client will wait before executing the next test case. By default, it is set to 0, meaning unless delay is specified, intaQt Client will not pause between test cases. When combined with retries, the delay will also be applied for each retry.

Example The following shows an XML configuration file containing the delay attribute:

<?xml version="1.0" encoding="UTF-8"?>
<TestSuiteCollection name="Testsuite with delay">
    <Description>
        Testsuite with delay
    </Description>

    <TestSuite name="testsuite with delay">
        <TestCase>...</TestCase>
        <TestCase name="testcase">
            <Description>testcase</Description>
            <TestScript name="intaqt" file="passing.feature" workdir="suite-with-retries-report" delay="5"/>
        </TestCase>
    </TestSuite>
</TestSuiteCollection>

Calling intaQt Client with the example above, intaqt test my-project test-config.xml will execute the passing.feature test case in the my-project project with a delay of 5 seconds.

Note
If specified, the `delay` command line option will override any individually-configured delays in the XML configuration file. For example, calling the command line option `intaqt test --delay 10 my-project test-config.xml` will run the same Feature File in the example above, but with a delay of `10` seconds instead.
The delay for the first test case will always be ignored. This means executing `intaqt test --delay 10 my-project` will ignore that option and just execute all tests inside the `my-project` project.

Retry Failing Test Cases ¶

Specifying the Retry Counter for test cases is available as an additional attribute of the TestScript called retries. By default, the retry counter is 0, meaning unless retries are specified, intaQt Client will not attempt to rerun failing test cases.

Note
If specified, the `retries` command line option will override any individually-configured retries counter in the XML configuration file.

Example The following example shows a file that may be named config.xml, which holds a test suite's configuration:

 <?xml version="1.0" encoding="UTF-8"?>
<TestSuiteCollection name="Testsuite with retries">
    <Description>
        Single testsuite single testcase
    </Description>

    <TestSuite name="testsuite with retries">
        <TestCase name="failing testcase">
            <Description>failing testcase</Description>
            <TestScript name="intaqt" file="failing.feature" workdir="failing-test-report" retries="3"/>
        </TestCase>
    </TestSuite>
</TestSuiteCollection>

Calling intaQt Client with the example above, intaqt test my-project config.xml will execute failing.feature inside my-project. If failing.feature always fails, intaQt Client will execute it 4 times (3 retries) before giving up.

The report artifacts from the first execution will be stored under the failing-test-report directory, while for each retry, the report directory will be appended with a --retry-<retry number> suffix to its name.

Using the example above, intaQt Client will generate the following report directories:

failing-test-report
failing-test-report–retry-1
failing-test-report–retry-2
failing-test-report–retry-3

Aggregate Test Results ¶

In addition to the single report files, intaQt Client provides the ability to aggregate all results into a single summary file (junit format). This is done by using the -s <summary-file-name>.xml option when executing intaQt Client.

Example 1

1	intaqt test -s summary.xml my-project intaqt-config.xml

Example 2

1	intaqt test --summary summary.xml my-project intaqt-config.xml