| AAX SDK 2.8.0 Avid Audio Extensions Development Kit

- Welcome
- Manual
- Classes
- Files
- Lists
- Support

Loading...
Searching...
No Matches
DTT Guide
AAX SDK Manual » AAX Host Guides

How to automate different test scenarios for DSH.

# Contents

- What is DTT
- How to run tests and suites in DTT
- Writing DTT scripts
- Logging in DTT and debugging DTT scripts
- Working with DTT test suites

# What is DTT

DishTestTool (DTT) is a Ruby-based command line tool, which provides the ability to script and thus automate DSH test scenarios. It is included in the DigiShell Tools package in the /DTT directory.

Note
  Ruby is installed by default on macOS. On Windows, you will need to install Ruby and add it to your PATH variable manually. For information regarding Ruby version compatibility with a specific build of DTT, see the ReadMe.txt file in /DTT/sources.

The DTT folder consists of:

- Sources
  - DTT core

  - scripts folder - place all DTT script files here

    By default, this folder includes a few example scripts demonstrating basic DTT operations and plug-in testing steps:

    - DSH_SigCancellation.rb - script for the cancellation test
    - DSH_TI_CycleCounts - script for performing cycle count test
    - SuiteGenerator.rb - generates suites for the DSH_SigCancellation and DSH_TI_CycleCounts tests
  - suites folder - place your DTT suite files here
- run_test.command (on Mac) or run_test.bat (on Windows) - command file for runnung tests
- run_irb.command (on Mac) or run_irb.bat (on Windows) - command-line interpreter

# How to run tests and suites in DTT

`run_test` is the main DTT execution program. `run_test` is able to execute Ruby scripts which have been placed in the scripts folder within the DTT directory.

- `run_test.command -l` - lists all the available scripts and suites
- `run_test.command 1` - runs test by number
- `run_test.command DSH_SigCancellation` - runs a test by name. Pay attensions that the name of test should be without (!) extension.
- `run_test.command –script DSH_SigCancellation -a sample_rate=48000 -a threshold=-80` - runs a test with test script arguments, which are specified using the `-a` option

Figure 1: running DTT tests.
For more information about script arguments, see Describing and using input arguments of the script

# Writing DTT scripts

Most of the DTT scripts require `DigiShell`, which allows them to run dsh and execute different dsh commands. Each script should be represented in the form of class, which inherits Script class, and also each script must have at least two elements: self.inputs section, where all the input arguments of the test should be described, and run method, which is the main body of your script.
require 'DigiShell'
class ScriptSample < Script
def self.inputs
return {}
end
def run
return pass("Well, it didn't explode. So that's something.");
end
end
Listing 1: Skeleton of the script

## Describing and using input arguments of the script

The available parameters and their values for a script are listed in the static `self.inputs` routine. Input arguments must be organized in the form of a hash map which is returned from this routine.
def self.inputs
return {
:sample_rate => [44100,[44100,48000,88200]],
:path_to_tfx => ['none'],
:threshold => [-96],
}
end
@dsh.init_dae(sample_rate)
Listing 2: Desribing input arguments for the script and using them
Hash entries should be in the following format:
:arg_name => [default_value, [range of allowed values]]
These arguments can be used then by just calling them by name, like in the example above with `sample_rate` argument.

## Writing body of the script

The body of the script must be enclosed in the body of the `run` method of the script class. As far as most DTT tests need DSH, in the example below it's shows how to create a DSH instance in the script and how to use it then. DSH instance can be creted with `DigiDhell.new` method, which requites DigiShell module, as has already been said. Then all the DSH commands become available as methods of the DSH instance, and input arguments can be passed to these command as input arguments for the methods, i.e. in parentheses `dsh.load_dish("DAE")`. Also it's recommended to handle possible exceptions that may occure during the execution of the code, and to make sure that DSH has been closed, if it was instantiated on the moment of the failure.
def run
begin
dsh = DigiShell.new(target)
dsh.load_dish("DAE")
dsh.init_dae(sample_rate)
dsh.close
return pass("Well, it didn't explode. So that's something.")
rescue Exception => e
# make sure to close down dsh before returning
if (dsh)
dsh.close
end
return fail(e)
end
end
Listing 3: Example of the body of the script.

# Logging in DTT and debugging DTT scripts

DTT tool has logging, and all the logs collected in the Logs folder, which is located in root directory of the DTT. DTT creates a separate folder for each test and names these folders with the corresponding names + the time when the particular test has been executed. For example:
`DSH_SigCancellation_20131225_185146_0001`
Inside each folder there are several log files:

- xxx.html – contains info about input & output of the test in a fancy form (tables)
- xxx_c.txt – contains the list of the DSH commands that have been executed
- xxx_d.txt – DSH output
- xxx_i.txt – info about your system
- xxx_l.txt – standard output
- xxx_v.txt – verbose output

## Interactive mode

There is an option to run DTT in interactive mode using interactive ruby shell (irb). When running in this mode, DTT creates a shell which is an extended version of the standard Ruby interpreter. Besides the standard functionality, it knows how to work with DTT classes and can give hints on their methods. In particular, the DTT interactive mode shell knows how to work with DigiShell.
To run DTT in interactive mode, go to the DTT folder and launch the `run_irb` program. At this point you will send ruby commands to dsh through the pipe in YAML format:
t = Target.new # creates an instance of Target. In this case target is a local machine, though we have a possibility to run test on remote machine.
dsh = DigiShell.new(t) # creates an instance of DigiShell(aka launching dsh binary)
dsh.load_dish('DAE') #Loads 'DAE' dish
dsh.help('init_dae') # Requests help from dsh for 'init_dae' command
dsh.init_dae
plugins = dsh.run #returns an array of plug-ins and writes them to plugins var.
plug-ins[0] #reaching first plug-in from the list
#... whatever you want to do
Listing 4: Running DTT in interactive mode

# Working with DTT test suites

Suites are files which contain the list of DTT scripts that should be run, and parameteres for these tests. These files should be created in YAML format. The list of the tests should be preceeded by `tests:` line. Then tests to be run should be described as a map with the following members:

- `name:` - name of the test
- `enabled:` - determines whether test will be run or skipped
- `args:` - input parameters of the test

The input parameters of the test should be orginized as a hash map. That means that all keys should start with ":" and look like ":plugin_spec: ".
Also suite may contain a section with the general parameters like:

- `verbose: false` which determines whether the output of the test in the console will be verbose or not.
- `timeoutFactor: "16.0"` which defines the time period, after which test will exit in case it stuck on the execution of the certain peice of code.

Example of the suite:
suitesettings:
verbose: false
tests:
#
# Cycle counting test
#
- name: DSH_TI_CycleCounts
enabled: true
args:
:plugin_spec: 'Digi,Pich,Psmm'
:sample_rate: 48000
Listing 5: Example of the DTT test suite

Note
  All the suites files should have an extension .gss

## Autogeneration of the suites

Sometimes it is necessary to generate the suites for the particular script for all the plug-ins from the bundle and/or for different sample rates. In this case instead of the copy-paste, which may lead to some mistakes and erratums, suitegenerator can be used. This is a special script, which takes as arguments the name of the script(s), for which the suites should be generated, and the list of their input parameters. Strange as it may sound, this data should be formed as a suite. Script itself is available as SuiteGenerator.rb along with other scripts in the DTT.

Note
  SuiteGenerator.rb can generate suites only for the two tests: DSH_SigCancellation test and DSH_TI_CycleCounts test

Here is the example of how to use this script to generate the suites for all the plug-ins from the 'D-Verb' bundle for all the sample rates the cancellation test AAX Native vs AAX DSP, and for the cycle counts test:
tests:
# Generate suite for Cancellation test: AAX Native vs AAX DSP
- name: SuiteGenerator
args:
:plugin_name: 'D-Verb'
:path_to_audio_files: /Volumes/G_Audio/GS_Test_Resources/audio/
:path_to_presets: /Volumes/G_Audio/GS_Test_Resources/PL_Settings/
:test_script: DSH_SigCancellation
enabled: true
# Generate suite for Instance Count test
- name: SuiteGenerator
args:
:plugin_name: 'D-Verb'
:path_to_presets: /Volumes/G_Audio/GS_Test_Resources/PL_Settings/
:test_script: DSH_TI_CycleCounts_se
enabled: true
suitesettings:
verbose: false
timeoutFactor: "16.0"
Listing 6: Example of how to use SuiteGenerator script for generating suites for all the plug-ins from the 'D-Verb' bundle for the all sample rates for the cancellation and for the cycle counts test.
To generate the suites, one should run this suite for the SuiteGenerator as an ordinary suite by executing the `run_test.command <the name of the suite for the SuiteGenerator>` command. All the suites will be generated into the single file, which will be located inside the suites folder, and will be called like:
`dspVSnative(optional)_<the name of the plug-in>_<the name of the test>.gss`
Examples:

- TRIM_DSH_TI_CycleCounts.gss
- dspVSnative_TRIM_DSH_SigCancellation.gss

Collaboration diagram for DTT Guide: