Custom Tests

In order to use the custom test functionality, you first need to define the custom test. This should be exposed in a Python file. This is slightly different for each modality. Please see below for detailed descriptions of what the Python file should look like for each modality.

• Tabular Custom Tests
• NLP Custom Tests

Once this python file exists, we need to include it in the test configuration for a given run. This is done with the custom_tests key within either the tests_config_path or tests_config, eg:

"tests_config": {"custom_tests": [{"test_path": "path/to/file.py"}]}