Setup guide

Requirements

Linux x86-64 with H100 NVL or H100 SXM GPUs. The first-run example uses two GPUs: one for Worker and one for Judge.

Install Docker Engine 24+, an NVIDIA driver, and the NVIDIA Container Toolkit.

On Ubuntu, install the host tools:

sudo apt-get update
sudo apt-get install -y git git-lfs jq util-linux rsync patch curl ca-certificates tmux python3 coreutils tar xz-utils
curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
uv tool install 'huggingface_hub[cli]==1.23.0'
git lfs install
sudo usermod -aG docker "$USER"

Open a new login session for Docker group access, then configure and check the GPU runtime:

export PATH="$HOME/.local/bin:$PATH"
umask 022
sudo nvidia-ctk runtime configure --runtime=docker --set-as-default
sudo systemctl restart docker
docker info --format '{{.DefaultRuntime}}'
docker run --rm -e NVIDIA_VISIBLE_DEVICES=all \
  nvidia/cuda:12.8.1-base-ubuntu24.04 nvidia-smi -L

The runtime should print nvidia and the container should list your GPUs. Allow about 25 GB for one task’s images and additional space for its models and run outputs; Miles requires a large scratch disk.

Installation

Clone the repository and install its Python dependencies. Run the following setup commands in Bash from this checkout.

git clone https://github.com/InfrathonX/Infrathon.git
cd Infrathon
uv sync --frozen --dev --python 3.12
uv pip install --python .venv/bin/python numpy==2.5.2
uv pip install --python .venv/bin/python \
  --index-url https://download.pytorch.org/whl/cpu torch==2.14.1
.venv/bin/sforge --help

uv venv ~/.venvs/host-torch --python 3.12
uv pip install --python ~/.venvs/host-torch/bin/python \
  --index-url https://download.pytorch.org/whl/cu128 torch==2.8.0
export PATH=~/.venvs/host-torch/bin:$PATH

Agent harness setup

The first-run example uses Codex. The builder downloads Node.js and the CLI into ../agent_runtimes/. The versions below are pinned in the repository.

Codex

export CODEX_CLI_VERSION=0.159.3
export SFORGE_AGENT_RUNTIME_DIR="$(realpath -m "../agent_runtimes/codex-$CODEX_CLI_VERSION")"
if [ ! -d "$SFORGE_AGENT_RUNTIME_DIR" ]; then
  launch/build-agent-runtime.sh --agent codex --version "$CODEX_CLI_VERSION"
fi
PATH="$SFORGE_AGENT_RUNTIME_DIR/bin:$PATH" "$SFORGE_AGENT_RUNTIME_DIR/bin/codex" --version

Enable device-code login in your ChatGPT account or workspace, then sign in and save file-based credentials:

PATH="$SFORGE_AGENT_RUNTIME_DIR/bin:$PATH" \
  "$SFORGE_AGENT_RUNTIME_DIR/bin/codex" \
  -c 'cli_auth_credentials_store="file"' login --device-auth
chmod 600 "$HOME/.codex/auth.json"

For API-key login or signing in from a local computer, see Codex authentication.

CLAUDE_CODE_VERSION=2.1.287
CLAUDE_RUNTIME_DIR="$(realpath -m "../agent_runtimes/claude-code-$CLAUDE_CODE_VERSION")"
if [ ! -d "$CLAUDE_RUNTIME_DIR" ]; then
  launch/build-agent-runtime.sh --agent claude-code --version "$CLAUDE_CODE_VERSION"
fi
PATH="$CLAUDE_RUNTIME_DIR/bin:$PATH" "$CLAUDE_RUNTIME_DIR/bin/claude" --version
PATH="$CLAUDE_RUNTIME_DIR/bin:$PATH" "$CLAUDE_RUNTIME_DIR/bin/claude" setup-token

mkdir -p "$HOME/.config/infrathon"
read -rsp 'Claude OAuth token: ' CLAUDE_TOKEN; printf '\n'
(umask 077; printf '%s\n' "$CLAUDE_TOKEN" > "$HOME/.config/infrathon/claude.token")
unset CLAUDE_TOKEN
chmod 600 "$HOME/.config/infrathon/claude.token"

Change the version

Change the version variable in the chosen example and build that runtime. Save its SFORGE_AGENT_RUNTIME_DIR in launch/mlsysforge.env; for Codex, also save CODEX_CLI_VERSION. Task images can be reused.

Miles stores its runtime and version settings in the task’s run.env.

Configuration

After Codex login, enter a model ID available to your account. Worker and reviewer use the same model and credentials.

read -rp 'Model ID: ' MODEL
CODEX_CLI_VERSION="${CODEX_CLI_VERSION:-0.159.3}"
CODEX_RUNTIME_DIR="$(realpath -m "../agent_runtimes/codex-$CODEX_CLI_VERSION")"
cat > launch/mlsysforge.env <<EOF
MODEL=$MODEL
REASONING_EFFORT=max
CODEX_CLI_VERSION=$CODEX_CLI_VERSION
CODEX_AUTH_JSON_PATH="$HOME/.codex/auth.json"
SFORGE_AGENT_RUNTIME_DIR="$CODEX_RUNTIME_DIR"
INFRATHON_ASSETS_ROOT="$HOME/.cache/infrathon/assets"
SFORGE_VERSION_STORE_OPTIONAL=1
SFORGE_LLM_JUDGE_MODE=enforce
SFORGE_LLM_JUDGE_ENGINE=agent
SFORGE_LLM_JUDGE_AGENT=codex
SFORGE_LLM_JUDGE_MODEL=$MODEL
SFORGE_LLM_JUDGE_AGENT_RUNTIME_DIR="$CODEX_RUNTIME_DIR"
SFORGE_LLM_JUDGE_CODEX_AUTH_JSON="$HOME/.codex/auth.json"
EOF
set -a
. launch/mlsysforge.env
set +a

The FP8 example saves submission archives without a candidate-history store. Miles, Concurrent serving, KVPress, KVzip and Quant-VideoGen also need Candidate history. Use each task’s Run the task section for its GPU, model, data, and launcher settings.

MODEL=MODEL_ID
MODEL_API_BASE_URL=http://ENDPOINT_HOST:PORT/v1
MODEL_API_KEY_FILE=/absolute/path/to/api.key

SFORGE_LLM_JUDGE_MODEL=MODEL_ID
SFORGE_LLM_JUDGE_API_BASE_URL=http://ENDPOINT_HOST:PORT/v1
SFORGE_LLM_JUDGE_API_KEY_FILE=/absolute/path/to/api.key

Models and data

Select Dynamic grouped FP8 GEMM and download its task assets. The download scripts read INFRATHON_ASSETS_ROOT from the configuration loaded into your shell.

TASK=standalone-dynamic-grouped-fp8-h100-v3-scratch
TASK_DIR="tasks/latest/$TASK"
"$TASK_DIR/assets/fetch.sh"
.venv/bin/python scripts/infrathon/assets.py check "$TASK"

hf auth login

Docker images

With TASK and TASK_DIR selected in Models and data, choose the host hardware: h100-94gb-400w for H100 NVL or h100-80gb-700w for H100 SXM.

export INFRATHON_HARDWARE=h100-94gb-400w
INFRATHON_PULL=1 bash "$TASK_DIR/scripts/build_images.sh"

The helper pulls the recorded Work and public Judge images, adds the Judge’s private data, and records the local images for launch. For Miles and MInference, use this helper instead:

scripts/infrathon/pull_images.sh "$TASK"

IMAGE_REPOSITORY="${INFRATHON_REGISTRY_REPOSITORY:-$(jq -r '.repository' scripts/infrathon/registry.json)}"
PUBLISHED="$TASK_DIR/images/published.json"
WORK_TAG="$(jq -er --arg hw "$INFRATHON_HARDWARE" '.hardware[$hw].work_tag' "$PUBLISHED")"
JUDGE_TAG="$(jq -er --arg hw "$INFRATHON_HARDWARE" '.hardware[$hw].judge_public_tag' "$PUBLISHED")"
docker pull "$IMAGE_REPOSITORY:$WORK_TAG"
docker pull "$IMAGE_REPOSITORY:$JUDGE_TAG"

bash "$TASK_DIR/scripts/build_images.sh"

Candidate history

Miles, Concurrent serving, KVPress, KVzip and Quant-VideoGen use a store for candidate versions. With TASK_DIR set to your task, initialize and register it:

export SFORGE_VERSION_STORE_ROOT="$PWD/../infrathon-version-store"
.venv/bin/python - "$SFORGE_VERSION_STORE_ROOT" "$TASK_DIR" <<'PY'
import json, sys, uuid
from pathlib import Path
from sforge.harness.version_store import VersionStore
root, task = (Path(p).expanduser().resolve() for p in sys.argv[1:])
specs = list((task / "tasks").glob("*.json"))
assert len(specs) == 1, "Expected one task definition"
task_id = json.loads(specs[0].read_text())["task_id"]
root.mkdir(parents=True, exist_ok=True)
marker = root / ".sforge-version-store.json"
if not marker.exists():
    marker.write_text(json.dumps({"schema_version": 1,
        "kind": "sforge-version-store", "store_id": uuid.uuid4().hex}) + "\n")
VersionStore.open(root)
folder = root / "tasks" / task.name
(folder / "runs").mkdir(parents=True, exist_ok=True)
path = folder / "task.json"
record = {"schema_version": 1, "task_directory": task.name, "task_id": task_id}
if path.exists():
    assert json.loads(path.read_text()) == record, "Existing registration differs"
else:
    path.write_text(json.dumps(record) + "\n")
assert VersionStore.open(root).supports_task(task_id)
print("Registered", task_id, "in", root)
PY
.venv/bin/sforge versions --version-store-root "$SFORGE_VERSION_STORE_ROOT" --verify

For Miles, copy the printed absolute path into SFORGE_VERSION_STORE_ROOT in the task’s run.env. For Concurrent serving, keep this variable exported in the launch shell. Register each task once in the same store.

Run a task

Start the FP8 task with Codex after completing Configuration, Models and data, and Docker images. Select two available GPU indices and an unused Judge port.

set -a
. launch/mlsysforge.env
set +a
nvidia-smi --query-gpu=index,uuid --format=csv,noheader
RUN_ID="fp8-$(date -u +%Y%m%d-%H%M%S)"
launch/standalone-fp8/run.sh --work-gpu 0 --judge-gpu 1 \
  --model "$MODEL" --effort max --hours 12 \
  --judge-port 18096 --run-id "$RUN_ID" \
  --auth "$CODEX_AUTH_JSON_PATH"

This starts the agent in the foreground. Add --tmux to run it in a detached session.

sforge-submit --details
sforge-submit --list

Other tasks have their setup and launch commands under Task List.

Results and logs

Runs are saved under outputs/<task-directory>/runs/<run-id>/. Flux, ORPO, GKD and StreamDiffusionV2 use tasks/latest/<task>/logs/runs/<run-id>/; the Miles tasks use $MILES_SCRATCH_ROOT/<task>/.

For setup questions, join Discord.