MIT Licensed · Local PyTorch Inference · fal-Compatible API

Local image & video generation with a fal-compatible API and one-CLI setup.

Run Qwen-Image-2.1, Wan 2.2, and MiniMax H3 on your own Windows, Linux, or macOS host—without wiring ComfyUI node graphs or stitching together separate downloaders and servers.

Models: FP8 / INT8 single-file checkpoints UI: Built-in Studio + /docs Queue: Polling + SSE streams
openfal cli — v0.1.0
# 1. Inspect GPUs, RAM, and free disk
$ openfal detect
✓ Detected compute GPU, host RAM & recommended profile

# 2. Install smallest supported image setup (~16.1 GB)
$ openfal autosetup --tier minimal -y
✓ Checkpoints, encoders, VAE & tokenizers verified

# 3. Serve fal-compatible API, queue & Studio (127.0.0.1 default)
$ openfal serve --port 8008
✓ Studio:  http://127.0.0.1:8008/
✓ OpenAPI: http://127.0.0.1:8008/docs
Why openfal

Built for developers who want local endpoints, not workflow plumbing.

openfal sits between hosted APIs and node-graph workstations: familiar client ergonomics, pre-wired local checkpoints, and a single CLI lifecycle.

Drop-in fal-style API & SDK

Exposes fal-compatible route names, /queue/* submission, SSE status streaming, and a fal_client-style Python SDK (openfal.subscribe, run, submit, upload_file).

Scope: Covers supported local models only—does not claim full hosted fal.ai catalog or moderation parity.

No ComfyUI GUI or graph UX

Consumes supported Comfy-Org single-file FP8/INT8 checkpoints under the hood and verifies supporting assets (encoders, VAEs, and Wan Animate's 403 CLIP/pose/SAM2 files) behind fixed HTTP routes and Studio.

Trade-off: Fixed, verified endpoints instead of arbitrary custom node-graph composability.

One CLI beyond just serving

detect, autosetup, download (including --runtime-assets-only repair), models, cleanup, and serve live in one tool—no manual weight hunting.

Unlike LLM engines such as vLLM (continuous batching, multi-GPU TP), openfal is a serialized PyTorch block-offloading runner.
Quickstart

From clone to local Studio in three steps.

Requires Python 3.10+ and a PyTorch build matching your compute device. Model files stay on the server host; any machine on your network can call the API.

STEP 1 · INSTALL & DETECT

Install CUDA PyTorch & openfal

git clone https://github.com/esokullu/openfal.git
cd openfal
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
pip install -e ".[inference]"
openfal detect

Install CUDA PyTorch first on Windows/Linux NVIDIA GPUs (macOS/CPU can skip the index-url line).

STEP 2 · DOWNLOAD WEIGHTS

Pick a tier or component

openfal autosetup --dry-run
openfal autosetup --tier minimal -y

Or target a model directly with openfal download --component qwen_image_21 --quant fp8_e4m3.

STEP 3 · SERVE

Start API, queue & Studio

openfal serve --port 8008

Binds 127.0.0.1 by default. For LAN access, pass --host 0.0.0.0 --api-key <secret>. Open http://127.0.0.1:8008/ for Studio or /docs for OpenAPI.

qwen_image_21~16.1 GB

Qwen Turbo DiT, Qwen3-VL-8B encoder & image VAE.

wan22_animate~21.7 GB

Animate DiT, CLIP, pose/SAM2 assets & shared Wan deps.

minimax-omni-compact~48.0 GB

Ref2VA and shared Qwen3-VL-32B / video / audio VAEs.

minimax-only~88.1 GB

FL2VA, Ref2VA, H3-Image & shared MiniMax components.

alibaba-only~98.6 GB

Wan T2V, I2V, Animate assets & Qwen-Image-2.1.

all~186.7 GB

Both model families (approx. FP8/INT8 download size).

API & SDK

Call local endpoints like fal.ai.

Use the Python SDK, direct synchronous POSTs, or queued requests with status polling and Server-Sent Events.

import openfal

server = "http://127.0.0.1:8008"
reference = openfal.upload_file("portrait.png", base_url=server)
result = openfal.subscribe(
    "qwen/image-2.1/edit",
    arguments={
        "prompt": "Keep the person recognizable and change the lighting to warm evening light",
        "reference_image_urls": [reference],
        "resolution": "768P",
        "aspect_ratio": "1:1",
        "seed": 20261117,
    },
    base_url=server,
)
print(result["images"][0]["url"])
import openfal

server = "http://127.0.0.1:8008"
identity = openfal.upload_file("character.png", base_url=server)
driver = openfal.upload_file("performance.mp4", base_url=server)
result = openfal.subscribe(
    "wan/v2.2/animate",
    arguments={
        "prompt": "The reference character follows the driving performance naturally",
        "face_image_url": identity,
        "driving_video_url": driver,
        "resolution": "480P",
        "duration": 3,
        "fps": 16,
        "seed": 20261117,
    },
    base_url=server,
)
print(result["video"]["url"])
curl -X POST http://127.0.0.1:8008/qwen/image-2.1/text-to-image \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A golden poodle running on an Istanbul cobblestone street",
    "resolution": "768P",
    "aspect_ratio": "16:9",
    "seed": 20261117
  }'
# Submit to queue (returns request_id, status_url, response_url, cancel_url)
curl -X POST http://127.0.0.1:8008/queue/wan/v2.2/text-to-video \
  -H "Content-Type: application/json" \
  -d '{"prompt":"A dog walking down a quiet street","resolution":"480P","duration":2,"seed":20261117}'

# Optional: stream live status updates via SSE
curl -N http://127.0.0.1:8008/wan/v2.2/text-to-video/requests/<request_id>/status/stream
Supported Workflows

Implemented routes and local checkpoints.

Inspect live installed routes anytime via GET /v1/endpoints and readiness via GET /v1/models/status.

Workflow Endpoint Local Model
Text-to-image qwen/image-2.1/text-to-image Qwen-Image-2.1-Turbo
Reference-guided image editing qwen/image-2.1/edit Qwen-Image-2.1-Turbo
Text-to-video wan/v2.2/text-to-video Wan2.2-T2V-A14B
Image-to-video wan/v2.2/image-to-video Wan2.2-I2V-A14B
Character animation wan/v2.2/animate Wan2.2-Animate-14B
Character replacement wan/v2.2/recast Wan2.2-Animate-14B
Text/image-to-video with audio minimax/h3/text-to-video, minimax/h3/image-to-video MiniMax FL2VA
Reference-conditioned video & editing minimax/h3/reference-to-video, minimax/h3-max/recast, minimax/h3-max/relight, minimax/h3-max/insert-video MiniMax Ref2VA (H3-Max routes map to local H3 partitions)
Text-to-image & editing minimax/h3/text-to-image, minimax/h3/image-to-image MiniMax H3-Image
Positioning

How openfal compares.

Different tools optimize for different constraints. Here is where openfal fits—and where another tool is the better choice.

Dimension openfal fal.ai (Hosted) ComfyUI vLLM
Primary focus Local image & video diffusion API + Studio Managed cloud catalog across many models Visual node-graph experimentation High-throughput LLM / multimodal serving
Setup UX One CLI (detect, autosetup, serve) API key only (zero local install) Node graphs, custom nodes, JSON workflows Serving flags & separate HF weights
API ergonomics fal-style routes, queue, SSE & Python SDK Native fal.ai SDK & cloud queue Graph prompt API OpenAI-compatible REST API
Execution model Serialized single-device PyTorch + host block offload Autoscaled cloud fleet Local graph execution Continuous batching, PagedAttention, multi-GPU TP
Best fit Self-hosted app integration & simple local runs Zero-ops scale & broad model variety Custom pipelines & node-level control Production LLM serving

Verified engineering limits to know before deploying

  • Single compute device: Runners execute on logical cuda:0 (or MPS/CPU fallback). Profiles named dual_* are memory-planning configs and do not perform multi-GPU tensor parallelism.
  • Profile telemetry is estimated: Studio's block map and /v1/engine/status report profile estimates (telemetry_kind: "profile_estimate"), not live hardware measurements.
  • macOS status: MPS and CPU fallback branches have host regression coverage, but actual Apple Silicon inference and unified-memory headroom have not been benchmarked yet.
  • Local trust boundary: Binds 127.0.0.1 by default with an in-memory queue (lost on restart). Pass --api-key (or set OPENFAL_API_KEY) whenever exposing --host 0.0.0.0 on a LAN.
Benchmarks · RTX 5090 (32 GiB)

Measured local performance: openfal vs ComfyUI vs vLLM-Omni.

Sequential VRAM-isolated runs on a single NVIDIA GeForce RTX 5090 (--warmup 1 --repeats 2 --concurrency 1 --sequential-local, 2026-11-18 / 2026-11-19). openfal and ComfyUI ran natively on Windows; vLLM-Omni ran against the same GPU under WSL2.

Scenario Engine Startup Cold Warm Mean / p50 Req/min Peak VRAM GPU Util
qwen21_t2i
1344×768 · 8 steps
openfal 1.85 s 16.23 s 3.98 s 15.06 24.7 GB (25333 MiB) 35.9%
Qwen-Image-2.1-Turbo ComfyUI 18.55 s 12.88 s 5.32 s 11.27 16.6 GB (17009 MiB) 65.6%
Fastest warm latency vLLM-Omni 53.40 s 5.05 s 2.45 s 24.49 24.7 GB (25257 MiB) 74.7%
minimax_h3_t2i
1344×768 · 6 PDD steps · Fastest warm
openfal 1.81 s 32.44 s 3.47 s 17.28 26.1 GB (26711 MiB) 19.5%
MiniMax-H3-Image ComfyUI 18.55 s 10.18 s 3.83 s 15.68 30.8 GB (31513 MiB) 65.3%
BF16 + DLO streaming vLLM-Omni 128.59 s 65.25 s 65.00 s 0.92 16.3 GB (16697 MiB) 23.2%
wan22_i2v_14b
848×480 · 33f @ 16 fps · 20 steps
openfal 1.86 s 127.27 s 109.14 s 0.55 19.3 GB (19803 MiB) 92.2%
Wan 2.2 A14B I2V ComfyUI 18.55 s 121.74 s 85.87 s 0.70 31.3 GB (32021 MiB) 85.7%
Fastest warm latency vLLM-Omni 107.76 s 90.18 s 75.58 s 0.79 21.2 GB (21729 MiB) 93.6%
minimax_h3_i2v
832×480 · 20 steps · 24 fps + audio
openfal 1.81 s 84.45 s 52.78 s 1.14 26.4 GB (27061 MiB) 76.2%
MiniMax FL2VA · Fastest warm ComfyUI 18.55 s 54.70 s 48.34 s 1.24 30.9 GB (31635 MiB) 90.6%
97f req → 107f output (4.46 s) vLLM-Omni 94.26 s 133.22 s 116.41 s 0.52 14.2 GB (14503 MiB) 69.2%

Precision & configuration notes across engines

  • Alibaba models: vLLM-Omni runs dequantized BF16 checkpoints with online --quantization fp8 (and --enable-cpu-offload on Wan 2.2), winning qwen21_t2i (2.45 s) and wan22_i2v_14b (75.58 s). openfal uses native INT8 ConvRot / FP8 scaled Tensor Core GEMMs + cuDNN SDPA priority, beating ComfyUI on qwen21_t2i (3.98 s vs 5.32 s) and cutting wan22_i2v_14b to 109.14 s at 19.3 GB VRAM (12 GB less than ComfyUI).
  • MiniMax-H3 models: openfal wins minimax_h3_t2i (3.47 s vs ComfyUI 3.83 s, 18.7× faster than vLLM-Omni) and runs within 9% of ComfyUI on minimax_h3_i2v (52.78 s vs 48.34 s, 2.2× faster than vLLM-Omni) using 4.5–4.7 GB less VRAM than ComfyUI. vLLM-Omni streams dequantized BF16 weights via DistributedLayerwiseOffloadBackend (--dlo-resident-layers 12, 14.2–16.3 GB VRAM), where PCIe H2D transfer dominates latency.
  • FL2VA duration floor: openfal and ComfyUI generate 73 frames (3.04 s at 24 fps); vLLM-Omni requires at least 4 s, receives num_frames=97, and outputs 107 frames (4.46 s) with 32 kHz stereo AAC audio.
  • Full report & raw JSON telemetry: View RTX 5090 Benchmark Record (Markdown) or consolidated JSON results on GitHub.