Run RIME on Managed Docker Images

There may be circumstances when you need to run RIME with a model which requires additional dependencies that are not specified in the provided base testing engine image.

This guide will cover how to run RIME with a custom Docker image when you need a particular version of a library or any additional dependencies to run your model.

This guide assumes that your administrator has enabled the Managed Images feature in your RIME deployment. This feature allows you to create Docker images directly from the SDK without needing to manually write Dockerfiles. You can also use this feature to query for existing Managed Images so that you can easily find images with the requirements you need and use the same images across team. Finally, when you upgrade your version of RIME, your old managed images will still be usable because the RIME backend will upgrade them to be compatible with the latest version.

Note: The following assumes you have instantiated Client, and that you are familiar with running RIME with the RIME SDK. To learn more about the RIME SDK, please refer to the SDK Reference for more information.

Building a Managed Docker Image

The Managed Images feature allows you to build custom Docker images directly from the SDK. You only need to specify the name of the image to use as an alias and the list of additional pip requirements to be installed.

Say you have a model that requires xgboost at version 1.0.2 and tensorflow at a flexible version. You can call this image "xgboost102_tensorflow" as an alias to remember it later.

To begin, initialize the Client and point it to the location of your RIME backend.

from rime_sdk import Client

rime_client = Client("rime.<YOUR_ORG_NAME>.rime.dev", "<YOUR_API_TOKEN>")

Next, you may use the helper method Client.pip_requirement(library_name: str, version_spec: Optional[str]) to construct a list of pip requirements for your image. The first argument is the name of the library (e.g. "tensorflow" or "xgboost") and the second argument is a valid pip version specifier (e.g. ">=0.1.2" or "==1.0.2"). Omitting the version specifier is equivalent to not fixing the version.

requirements = [
  # Fix the version of `xgboost` to `1.0.2`.
  rime_client.pip_requirement("xgboost", "==1.0.2"),
  # We do not care about the installed version of `tensorflow`.
  rime_client.pip_requirement("tensorflow")
]

To start the job that will build your custom image, use rime_client.create_managed_image(name: str, requirements: List[ManagedImage.PipRequirement]). The return value will be a Python class that you can use to track the status of the image building job.

# Start a new image building job 
builder_job = rime_client.create_managed_image("xgboost102_tensorflow", requirements)

# Wait until the job has finished and print out status information.
# Once this prints out the `READY` status, your image is available for use in stress tests.
builder_job.get_status(verbose=True, wait_until_finish=True)

Start a Stress Test on a Managed Docker Image

Once the image has been built successfully (this will take a few minutes), you may start stress tests using your managed image. The following code snippet shows how you can use your "xgboost102_tensorflow" image.

# Insert your own stress testing configuration here.
config = ...

# Kick off a stress test on the "xgboost102_tensorflow" image.
stress_test_job = rime_client.start_stress_test(test_run_config=config, rime_managed_image="xgboost102_tensorflow")

# Wait until the job has finished and print out status information.
stress_test_job.get_status(verbose=True, wait_until_finish=True)

Querying for Managed Docker Images

The true power of the Managed Images feature is that it allows you to query for existing images with desired requirements. Instead of building a new custom Docker image for each stress test, you can query the backend for existing images that have the desired requirements. That makes managed images much easier to use across a team.

Say you want an image with catboost at version 1.0.3. Perhaps one of your teammates has worked on a model with the same requirement or you previously built a catboost image. To recover those images, you can first construct filters using the helper method Client.pip_library_filter(library_name: str, fixed_version: Optional[str]).

filters = [rime_client.pip_library_filter("catboost", "1.0.3")]

Then, you can provide the filters as an argument to Client.list_managed_images(). This is a paginated call so you can limit the number of images returned.

Note: list_managed_images returns a tuple of (images, next_page_token). See the reference for more detail on the function’s behavior.

images, next_page_token = rime_client.list_managed_images(pip_library_filters=filters)

Each element of images is a dictionary that contains information about all the installed pip libraries, the alias of the image, etc. Here is a code snippet to get all the names of the images in the first page of the query result.

names = [x["name"] for x in rime_client.list_managed_images(pip_library_filters=filters)[0]]