Configuration file - Attester documentation

Configuration file

The configuration file for a campaign can be either a JSON (.json) or a YAML (.yml or .yaml) file and it's divided into parts describing the source files to be tested, the list of tests and the reporting and running information.

Options that are not specific to a test campaign must be passed to the command line or the attester module

Resources

This section specifies which resource files should be tested.

The simplest example is

resources:
  /:
    - ../src

on in JSON

{
    "resources": {
        "/": ["../src"]
    }
}

This means that the server will be able to serve any file in the ../src folder from its root. The folder path is relative to the current working directory.

The resources object only describes what files can be served by the test server, not the files that are inserted in the test page.

The closes equivalent is Karma which allows to serve files without including them in the page. The difference is that with attester it's possible to map these files to a different path.

{pattern: '../src', included: false, served: true}

The resources object is particular important for project with dependencies.

Consider a project depending on npm packages or bower components, once installed, the dependencies are fetched in different folders. Configuring resources allows you to write more readable tests

Given this folder structure

/
  src/               # Source files
  node_modules/      # NPM packages
  bower_components/  # Bower dependencies
  tests/             # My test files
  campaign.yaml
resources:
  # Map the root folder to my source files
  /:
    - src

  # Map lib to both node_modules and bower_components
  /lib:
    - node_modules
    - bower_components

With the above configuration the test server is able to serve both node_modules and bower_components under the same lib path.

Tests

This section describes the list of tests that make a campaign and should be run by attester.

attester clearly separates the list of tests from the list of resources in order to better distribute tests among slaves. By doing this it's possible to split the testing also between multiple instances of the same browser.

Tests are grouped by test type (or unit testing framework). Not only you can use any testing framework, but you can also use more than once to run your campaign.

A typical configuration is

tests:
  mocha:
    files:
      includes:
        - tests/**/*.js

    extraScripts:
      - /source.js

This means that all .js files in the folder tests should be included by the mocha test-type.

extraScripts means that in the test page we also expect the listed scripts to be included. This is generally the source files under test.

The full configuration for files is

files:
  includes:
    - a list of included files
    - multiple paths are allowed

  excludes:
    - a list of files that shouldn't be tested

  # All paths inside includes and excludes are relative to this path
  rootDirectory: /base/path

Any other configuration specific to the test-type is specified next to files. Take for instance the configuration of a mocha campaign below:

tests:
  mocha:
    files:
      includes:
        - tests/**/*.js

    # User-interface (bdd|tdd|exports). Defaults to bdd
    ui: 'tdd'
    # Assertion library to be included.
    assertion: 'expect'

For a list of all available options and their meaning, refer to:

Browsers

This section is optional and describes the list of browsers that are expected to connect as slave.

If the browsers section is not present, attester will not make any difference between browsers and run each test only once, in whatever browser is connected.

However, if the browsers section is present, each test will be run once in each browser listed here. Browsers not listed here won't receive any task to run.

browsers:
  - browserName: 'Chrome'

  - browserName: 'IE'
    majorVersion: 10

  - browserName: 'Firefox'
    os: 'Desktop Linux'

Attester uses ua-parser for detecting browser version and operating system.

Most common names (browserName) are (not limited to):

  • IE
  • Firefox
  • Chrome
  • Safari
  • Opera
  • PhantomJS

Most common operating systems (os) are

  • Windows XP
  • Windows Vista
  • Windows 7
  • Windows 8
  • Mac OS X
  • Ubuntu
  • Debian
  • Fedora
  • Chrome OS
  • iOS
  • Android
  • Windows Phone
  • Firefox OS
  • BlackBerry OS

For convenience, two additional values are accepted by attester:

  • Desktop Linux (Linux, but not Android or webOS)
  • Desktop Windows (Windows, but not Windows Phone)

The version can be expressed with

  • majorVersion
  • minorVersion
  • revision

Reports

Some sections in the configuration file control where and what types of reports are written for each campaign.

Coverage

To have code coverage it's necessary to specify for which files you want to measure the coverage and where to save the report.

# Specifies which files will be instrumented for code coverage
coverage:
  files:
    rootDirectory: 'lib'
    includes:
      - '**/*.js'

# Path for each coverage test report type
coverage-reports:
  json-file: 'coverage.json'
  lcov-file: 'coverage.lcov'

Code coverage is obtained through node-coverage and the report can be written in

  • JSON, the internal format of node-coverage
  • LCOV, the format accepted by Sonar

Test Reports

This is an optional section to describe where the campaign reports should be written.

If missing, attester will simply write in the command line the result of each test task.

The configuration section for writing test reports on the disk is the following:

test-reports:
  json-file: 'report.json'
  xml-file: 'report.xml'
  xml-directory: 'reports'

The properties are

  • json-file JSON file with detailed information on each test run
  • xml-file JUnit-style XML file with the result of the test campaign
  • xml-folder Path of the folder in which attester generates a file for each test, this is the format accepted by Sonar.

Templating

The configuration files allows to use a simple templating engine to refer to other options specified in the configuration file or in the environment variables.

This syntax is <%= prop.subprop %>, for more information refer to the section on environment variables.