Helix RPC — LLM Streaming Tutorial¶

This tutorial demonstrates how to build a production-ready LLM token streaming service using Helix RPC and Python.

By running this example, you will see a simulated zero-serialization streaming architecture where tokens generated by a model are flushed directly to the client via Server-Sent Events (SSE).

Features¶

HuggingFace Integration: Automatically loads gpt2 if transformers is installed.
Demo Mode: Gracefully falls back to a simulated streaming response if ML libraries aren't present.
SSE Streaming: OpenAI-compatible data: {...} streaming chunks.
Modern UI: A polished React-style interface written in vanilla JS/CSS.
Performance Tracking: Real-time Time-to-First-Token (TTFT) and Tokens-per-Second (t/s) metrics.

Quick Start¶

Install dependencies:
```
pip install helix-rt aiohttp
```
(Optional) To use a real HuggingFace model instead of demo mode:
```
pip install transformers torch
```
Start the server:
```
python server.py
```
Open the UI: Navigate to http://localhost:8081 in your browser.

Architecture Overview¶

In a full Helix Rust/PyO3 deployment, the architecture looks like this:

[Web Client] 
     │ (HTTP GET /v1/chat/completions)
     ▼
┌────────────────────────────────────────────────────────┐
│ Helix Rust Gateway (Hyper + PyO3)                      │
│                                                        │
│  1. Receives SSE Connection Request                    │
│  2. Calls Python generator in-memory via PyO3 FFI      │
│  3. Receives yielded tokens directly in Rust memory    │
│  4. Flushes SSE chunks immediately to connection       │
└──────────────────┬─────────────────────────────────────┘
                   │
                   ▼ (Zero-Serialization / In-Memory)
┌────────────────────────────────────────────────────────┐
│ CPython Runtime (PyTorch / Transformers / HuggingFace) │
└────────────────────────────────────────────────────────┘