Flexie SAG

Inference node for Linux

SAG is an AI orchestrator. It runs conversations and agents against the models you choose, gives them tools that reach your own systems, and keeps the whole thing under a permission model you administer.

Models can come from a vendor's API, or they can run on hardware you own. This page is for the second kind: the inference node, a service you install on a machine with an NVIDIA graphics card. Once it has joined, the models on it appear in SAG like any other, and nothing you send them leaves your own network.

Installing

Run this on the GPU machine. The token comes from the console, under Inference, and it stands for an hour.

curl -fsSL https://sag-repo.flexie.io/install.sh | sudo sh -s -- \
    --url https://your-sag-server.example.com \
    --token <token>

It reads the graphics card, downloads the matching build and starts a service. Nothing is compiled on your machine. If your card is one we do not publish for, it says so and writes nothing.

Which graphics cards work

Any NVIDIA card from the Ampere generation onwards, which means 2020 and later. The engine needs bfloat16 in hardware, and cards older than that do not have it.

Not supported: Tesla T4, V100, and the RTX 20 series or older. These are common on the cheapest cloud instances, so it is worth checking before you rent one. The least expensive cards that do work well are the L4 and the A10.

Downloads

The installer picks the right one by itself. They are listed here so you can fetch one by hand for a machine with no route to the internet, and pass it with --from.

Build	For	CUDA	Size
cpu	No graphics card. Works anywhere, and is a great deal slower.	—	21 MB
cuda80	A100, A30	12.8	51 MB
cuda86	A10, A10G, A40, RTX 3090, RTX A6000	12.8	50 MB
cuda89	L4, L40, L40S, RTX 4090, RTX 6000 Ada	12.8	49 MB
cuda90	H100, H200, GH200	12.8	50 MB
cuda100	B200, B100, GB200	12.8	54 MB
cuda103	B300, GB300 (Blackwell Ultra)	13.0	52 MB
cuda120	RTX 5090, RTX PRO 6000 Blackwell	12.8	53 MB

Every file has a .sha256 beside it. builds.txt lists what is published.

Checking a machine can be reached

A node listens on 19443, and SAG connects to it. The machine cannot tell you whether that works, because a firewall, a security group or a missing port forward all look like a healthy install from the inside. So ask from out here instead:

# the address the internet sees you at
curl https://sag-repo.flexie.io/v1/ip

# and whether your node answers on it
curl https://sag-repo.flexie.io/v1/reachable?port=19443

The installer does this for you at the end and prints the result. It only ever connects back to the address the request came from, so it cannot be aimed at anything else.

Adding it to SAG

The machine registers itself when the installer runs, using the token.
It appears in the console under Inference.
Download a model onto it there, then give that model to a workspace.

Machines and models are separate on purpose: one machine is racked once, and any workspace can be given a model on it. A second workspace using the same model downloads nothing.