Tutorial Notebooks

These notebooks illustrate how to use Robust Intelligence to validate, protect, and monitor your ML pipeline across a wide range of tasks, from classification to ranking, and across tabular, natural language processing (NLP), and computer vision (CV) data modalities.

Each notebook walks through the steps of setting up Robust Intelligence with an example dataset/model then running a stress test on the model. The datasets and models used in these examples come from well-known public sources such as UCI ML or arXiv

Tabular Notebooks

Tabular tests are performed on table-formatted data. Generally speaking, data that consists of a list of records with various attributes for a given data can be considered tabular.

• RI Fraud Classification Walkthrough
• RI Lending Classification Walkthrough
• RI NYC Taxi and Limousine Data Walkthrough 🚖
• RI Movie Ratings Ranking Data Walkthrough 🎥

NLP Notebooks

NLP models attempt to correctly parse human speech, recognize entities being referred to by that speech, and analyze implied qualities of that speech, such as emotional subtext.

• RI Text Multiclass Classification Walkthrough
• RI Adversarial File Scanning Walkthrough

CV Notebooks

CV models attempt to interpret images to discern specific objects within the image or generally classify the contents of an image.

• RI Image Classification Walkthrough

REST API Notebooks

Although we provide a Python SDK for interacting with Robust Intelligence, you can also use the REST API directly. This notebook demonstrates how to use the API to run a stress test and query the results.

• RI NLP Walkthrough using REST API

Set Up Your Own Test Run

For further guides on setting up your own test run, begin with the following guide:

• Preparing Your Models and Datasets

Once you have your model set up, you can run a stress test or set up a schedule of stress tests using the SDK, Web UI or REST API:

• Creating a New Stress Test
• Scheduled Stress Testing
• Updating your Continuous Test