Uploading to Cloud Storage (Optional)
You can use the SDK to upload data files and model directories to a S3 bucket that Robust Intelligence has permission
to use (See Data Access Permissions for instructions).
All files uploaded will be stored in a subdirectory within the files directory.
File uploading is not currently supported in a Cloud deployment. Please use an external data source instead.
Uploading Data Files
To begin, initialize the Client and point it to the location of your RIME backend.
from rime_sdk import Client
rime_client = Client("rime.<YOUR_ORG_NAME>.rbst.io", "<YOUR_API_TOKEN>")
Afterwards, use the rime_client.upload_file method to upload data.
When the upload_path is not specified, the path contains a random string as directory name.
ref_path = rime_client.upload_file(file_path="./titanic/datasets/train.csv")
print("ref_path:", ref_path)
"ref_path: s3://rime-blob-c78976e9367daf-environment/files/84f82c37-0d8f-43af-8d05-76jknwefnjkm/data/train.csv"
When the upload_path is specified, the path contains the upload_path string in place of the random string.
ref_path = rime_client.upload_file(file_path="./titanic/datasets/train.csv", upload_path="rime_tests/titanic_tests")
print("ref_path:", ref_path)
"ref_path: s3://rime-blob-c78976e9367daf-environment/files/rime_tests/titanic_tests/data/train.csv"
Uploading Images
Robust Intelligence expects image dataset files as a list of JSON dictionaries with keys specified in the image_features parameter that each reference an image by an absolute or relative path.
Each image in the dataset file is uploaded to the Robust Intelligence cluster’s storage.
Use rime_client.upload_local_image_dataset_file to upload an image dataset file containing paths to images stored locally. When your image paths already reference external blob storage, use upload_file.
When the upload_path is not specified, the path contains a random string as directory name.
_, img_dataset_path = rime_client.upload_local_image_dataset_file(file_path="./titanic/images.tgz", image_features=["Image Path"])
print("img_dataset_path:", img_dataset_path)
"img_dataset_path: s3://rime-blob-c78976e9367daf-environment/files/84f82c37-0d8f-43af-8d05-76jknwefnjkm/data/images.tgz"
When the upload_path is specified, the path contains the upload_path string in place of the random string.
_, img_dataset_path = rime_client.upload_local_image_dataset_file(file_path="./titanic/images.tgz", image_features=["Image Path"], upload_path="rime_tests/titanic_tests")
print("img_dataset_path:", img_dataset_path)
"img_dataset_path: s3://rime-blob-c78976e9367daf-environment/files/rime_tests/titanic_tests/data/images.tgz"
Uploading Model Directories
The model directory is a directory containing the model.py file and any
extras, such as pre-processing code or the pickle file, that the model.py file depends on.
Model directories can easily be uploaded using the rime_client.upload_directory method.
When the upload_path is not specified, the path contains a random string as directory name.
model_dir = rime_client.upload_directory(dir_path="./titanic/model")
print("model_dir:", model_dir)
"model_dir: s3://rime-blob-c78976e9367daf-environment/files/c83aa17d-709b-48b5-89ed-a9e4bad42f0b/model"
When the upload_path is specified, the path contains the upload_path string in place of the random string.
model_dir = rime_client.upload_directory(dir_path="./titanic/model", upload_path="rime_tests/titanic_tests")
print("model_dir:", model_dir)
"model_dir: s3://rime-blob-c78976e9367daf-environment/files/rime_tests/titanic_tests/model"