Comfy Lab
ComfyUI driven as an API from Python: parameterised workflows, verified outputs, and a planner that keeps video generation inside one GPU's memory.
- Status
- Experiment
- Role
- Developer
- Year
- 2026
- Type
- Generative workflow automation
Where it stands
Local tooling and measurements. Nothing is hosted.
- Workflows
- 24, in API format
- Models
- Text-to-image, text-to-video, image-to-video
- Interface
- Command line, JSON in and out
There is no public link for this project.
The problem
ComfyUI is a good place to design a generation workflow and an awkward thing to call from a program. Node ids are opaque, failures are quiet, and a video model can exhaust memory several minutes into a job.
Goals and constraints
It had to
- Call any workflow by name with readable parameters.
- Know whether a job really produced a playable file.
- Fit generation into the memory that is actually free.
Within these limits
- One laptop GPU.
- Models with different frame-count and resolution grids.
My part
A personal lab. The workflows wrap third-party models; the bridge, planner and measurements are mine.
- Wrote the bridge: submit, poll, fetch, verify.
- Wrote the parameter-map convention and the workflows that use it.
- Wrote the resource planner, the out-of-memory ladder and the telemetry capture.
What it is
Each workflow is exported from ComfyUI in API format and paired with a small file that maps friendly names such as prompt or seed onto node inputs. A command returns one JSON object and a non-zero exit code on failure.
- Parameter maps
- A workflow's inputs are addressed by name. A mapped seed is randomised when omitted and reported back.
- Verified output
- A missing, empty or undecodable file fails the job at a named verify stage, with codec, size, frame rate and frame count read from the file.
- Out-of-memory ladder
- Tiled decode, then a lower resolution tier, then fewer frames, each tried once with the same seed and prompt.
- Backends own their grids
- Each video backend knows its own legal frame counts and resolutions, so a plan is valid before it is submitted.
Decisions and trade-offs
01The planner picks settings; ComfyUI manages memory
The planner only chooses workflows and parameters. Loading and unloading models stays with the tool that owns the GPU.
The cost. Estimates are provisional until history replaces them, and plans say which they used.
02Named profiles record what was tried
A profile is a bundle of settings with the date it was tested and what happened.
The cost. Profiles are specific to this hardware.
How it is built
- Standard-library Python talking to ComfyUI's HTTP API.
- Telemetry sampled during runs and stored beside the outputs.
Python · ComfyUI API · Wan 2.2 · Z-Image Turbo · LTX-Video · GGUF · FFmpeg
Hard problems
Posting a workflow in the editor's format, or with the wrong text encoding, is rejected before queueing with little explanation.
All submissions go through the bridge, which enforces API format and UTF-8.
A prompt that describes an action without an ending produces a pose that keeps drifting.
Every action phrase now names the final pose. The test clips recovered to a stable stance once it did.
Limits and open work
- Measurements come from a small number of clips on one machine.
- Generated footage is not published here because of model licence terms.
Need something like this built?
Tell me what you are making. I will tell you how I would approach it, and whether I am the right person.
Start a conversation