#Model Streamer

[!IMPORTANT] Repository move and rename notice: On September 13, 2026, Run:ai Model Streamer was renamed to Model Streamer and the repository moved from the run-ai GitHub organization to dsx-ai-factory/model-streamer. The runai-model-streamer PyPI packages and Python import paths are unchanged. Existing repository URLs and standard Git operations continue to work through GitHub redirects. If you maintain automation or integrations that reference run-ai/runai-model-streamer, such as GitHub Actions, webhooks, or pinned repository URLs, update them to dsx-ai-factory/model-streamer.

#Overview

Model Streamer is a Python SDK designed to facilitate the streaming of tensors from tensors files to GPU memory with concurrency and streaming. It provides an API for loading SafeTensors files and building AI models, allowing loading models seamlessly.

For documentation click here

For benchmarks click here

#Usage

Install the package

pip install runai-model-streamer

And stream tensors

from runai_model_streamer import SafetensorsStreamer

file_path = "model.safetensors"

with SafetensorsStreamer() as streamer:
    streamer.stream_file(file_path)
    for name, tensor in streamer.get_tensors():
        gpu_tensor = tensor.to('CUDA:0')

#Development

Our repository is built using devcontainer (Further reading)

The following commands should run inside the dev container

[!NOTE] You can use devcontainer-cli tool (Further reading) by installing it, and run every command with the following prefix devcontainer exec --workspace-folder . [COMMAND]

Build

make build

[!NOTE] We build libstreamers3.so and statically link it to libssl, libcrypto, and libcurl. if you would like to use your system libraries by dynamic link to them, run USE_SYSTEM_LIBS=1 make build

[!NOTE] On successful build, the .whl file would be at py/runai_model_streamer/dist/<PACKAGE_FILE> and py/runai_model_streamer_s3/dist/<PACKAGE_FILE>

Run tests

make test

Install locally

pip3 install py/runai_model_streamer py/runai_model_streamer_s3

[!IMPORTANT] In order to the CPP to run, you need to install libcurl4 and libssl1.1_1