From a ComfyUI graph to an API a brand's team can call

Our try-on pipeline lived as a ComfyUI graph with implicit state, hand-picked seeds, and custom nodes tracking whatever was on main that week. Turning it into something another company's engineers could call meant writing down which code ran, which seed, when the work happens, and what we refuse.

HimanshuSeptember 09, 2025Updated September 08, 2026

During one integration week, our endpoint received three distinct inputs: a 4K studio flat-lay of a lehenga, a smartphone photo of a kurta hanging on a door, and a PNG with an alpha channel. All three came from engineers who reasonably assumed a virtual try-on API accepts any image of clothing.

The alpha channel crashed our PIL preprocessor, resulting in a terminal traceback on my machine and a 504 gateway timeout for the caller.

Running the workflow interactively in ComfyUI lets you intervene. You watch the browser tab, drop the denoise when fabric goes plastic, and re-seed until segmentation stops clipping fingers. Once beta API keys went out to design partners like GettoIndia and EcomBuddha, nobody was watching the tab.

A ComfyUI graph is not an API with the wrapper missing. What makes it callable by someone else is the part the graph never had to write down: which code ran, which seed, when the work happens, and what you refuse.

Operational gaps between UI graphs and backend services

Our initial API wrapper took a single day to build. ComfyUI natively exposes an HTTP endpoint accepting workflow graphs in JSON format. I built a lightweight Python service that loaded our saved workflow, injected incoming model and garment URLs into the LoadImage nodes, dispatched the execution, and polled for output images.

While sufficient for internal demos, this setup failed across four areas when exposed to external engineering teams:

Failure Vector	Underlying System State	Impact on API Client
Floating dependencies	Custom nodes (SAM2, RMBG, LoRA merging) tracked `main` git branches	Unversioned, non-reproducible output across deployments
Resident GPU state	ComfyUI process cached in-memory LoRA merges from earlier calls	Changing weights on disk without restarting workers served stale cached layers
Unstructured exceptions	VRAM allocations, aspect-ratio mismatches, and empty masks threw Python tracebacks	HTTP timeouts without actionable error messages
Missing input validation	No formal constraints on color spaces, dimensions, or apparel categories	Silent generation failures on unsupported garments

To transition from an internal research tool to an enterprise service, we defined four operational guarantees: explicit code manifests, deterministic seeding, asynchronous job processing, and rigid input validation.

Step 1: Versioning execution manifests rather than UI graphs

We replaced loose workflow JSON files with comprehensive manifests that track model weight hashes, exact git commits for all node packages, and fixed merge hyperparameters.

json

// manifests/vton-2025-09-a.json
{
  "manifest_id": "vton-2025-09-a",
  "created": "2025-09-02",
  "graph": {
    "file": "graphs/flux_vton_plus.api.json",
    "sha256": "9f2c1e0b7a4d3f88c5b6e2a1d0c9f4b3e8a7d6c5b4a39281706f5e4d3c2b1a09"
  },
  "nodes": [
    { "repo": "comfyui-sam2", "commit": "4b1e9c2" },
    { "repo": "comfyui-rmbg", "commit": "a07d331" },
    { "repo": "comfyui-essentials", "commit": "c19f4ab" }
  ],
  "models": [
    { "role": "base", "file": "flux_fill.safetensors", "sha256": "1a2b3c4d" },
    { "role": "redux", "file": "flux_redux.safetensors", "sha256": "5e6f7a8b" },
    { "role": "vae", "file": "ae.safetensors", "sha256": "9c0d1e2f", "dtype": "fp16" },
    { "role": "lora_drape", "file": "drape_physics_r32.safetensors", "sha256": "3a4b5c6d" },
    { "role": "lora_occlusion", "file": "occlusion_depth_r32.safetensors", "sha256": "7e8f9a0b" }
  ],
  "merge": {
    "node": "model_merge_lora",
    "lambda_drape": 0.6,
    "lambda_occlusion": 0.4
  },
  "sampling": {
    "sampler": "euler_ancestral",
    "scheduler": "beta",
    "steps_primary": 30,
    "steps_refine": 10
  },
  "context_window": {
    "roi_padding": 0.2,
    "mask_dilation_px": 9,
    "inference_resolution": [1344, 1344]
  }
}

Manifests eliminate ambiguity around adapter ratios (such as 0.6 drape and 0.4 occlusion). Every API request pins a specific manifest ID, and responses echo the exact manifest executed. Old manifests remain pinned in production as long as their corresponding weights reside on disk, preventing silent breaking changes for live client catalogs.

Step 2: Deriving deterministic seeds from request payloads

Interactive diffusion relies on randomized seeds to generate candidate varieties. In a production API, randomized execution prevents caching, invalidates regression test suites, and makes customer bugs unreproducible.

python

# api/seeding.py
import hashlib
import json
from dataclasses import dataclass
 
 
@dataclass
class TryOnRequest:
    """A single try-on job received from an API caller."""
    request_id: str
    model_image_key: str
    garment_image_key: str
    category: str
    manifest_id: str
    seed: int | None = None
 
 
def derive_seed(req: TryOnRequest) -> int:
    """Return a deterministic seed for the request.
 
    If the caller provides an explicit seed, we use it directly to support
    exact replays. Otherwise, we hash the request payload into a uint32.
    Worker retries following transient hardware failures land on the identical
    seed rather than altering fabric drape.
    """
    if req.seed is not None:
        return req.seed % (2 ** 32)
 
    payload = json.dumps(
        {
            "model": req.model_image_key,
            "garment": req.garment_image_key,
            "category": req.category,
            "manifest": req.manifest_id,
        },
        sort_keys=True,
    ).encode()
    return int.from_bytes(hashlib.sha256(payload).digest()[:4], "big")
 
 
def apply_seed(graph: dict, seed: int) -> dict:
    """Inject the computed seed into all sampler nodes and lock randomisation."""
    for node in graph.values():
        if node.get("class_type") in {"KSampler", "KSamplerAdvanced"}:
            node["inputs"]["seed"] = seed
            node["inputs"]["noise_seed"] = seed
            node["inputs"]["control_after_generate"] = "fixed"
    return graph

Identical seeds run on matching GPU architectures with identical drivers yield visually consistent renders. Responses always return the derived seed so brand creative directors can store approved seeds across SKU batches.

Step 3: Asynchronous job queues and worker leasing

A full-body try-on pass takes approximately 12 seconds on an RTX 4090. Holding synchronous HTTP connections open for that duration creates connection pool exhaustion and exposes clients to gateway timeouts.

We designed the service around asynchronous job polling and webhooks using Redis for queue management.

python

# api/queue.py
import json
import time
import uuid
 
import redis
 
r = redis.Redis(decode_responses=True)
 
QUEUE_KEY = "vton:pending"
LEASE_TTL_SECONDS = 180
MAX_INFLIGHT_PER_KEY = 4
 
 
def submit(api_key: str, payload: dict) -> dict:
    """Validate concurrency and register a queued job."""
    inflight = r.scard(f"vton:inflight:{api_key}")
    if inflight >= MAX_INFLIGHT_PER_KEY:
        return {"status": "rejected", "reason": "concurrency_limit"}
 
    job_id = f"job_{uuid.uuid4().hex[:12]}"
    record = {
        "job_id": job_id,
        "api_key": api_key,
        "status": "queued",
        "manifest_id": payload["manifest_id"],
        "submitted_at": time.time(),
        "payload": payload,
    }
    r.hset(f"vton:job:{job_id}", mapping={"data": json.dumps(record)})
    r.sadd(f"vton:inflight:{api_key}", job_id)
    r.lpush(QUEUE_KEY, job_id)
    return {"job_id": job_id, "status": "queued", "queue_depth": r.llen(QUEUE_KEY)}
 
 
def claim(worker_id: str) -> dict | None:
    """Lease a job from the pending queue.
 
    If a worker crashes mid-inference, the lease expires and an automated
    sweeper requeues the job. Deterministic seeding ensures that the retry
    generates the expected image.
    """
    job_id = r.brpoplpush(QUEUE_KEY, f"vton:leased:{worker_id}", timeout=30)
    if job_id is None:
        return None
    r.setex(f"vton:lease:{job_id}", LEASE_TTL_SECONDS, worker_id)
    raw = r.hget(f"vton:job:{job_id}", "data")
    record = json.loads(raw)
    record["status"] = "running"
    r.hset(f"vton:job:{job_id}", mapping={"data": json.dumps(record)})
    return record

Our architecture uses Next.js for client interfaces, NestJS for REST APIs and PostgreSQL metadata management, Redis for leasing queues, S3 for object storage, and headless Python workers dedicated strictly to consuming jobs from the queue.

Step 4: Explicit API input contracts and rejection policies

Rather than attempting to render unsupported edge cases and producing broken outputs, the API validates inputs up front and returns descriptive HTTP 422 errors.

yaml

# api/surface.yaml
categories:
  supported:
    - dresses_one_pieces
    - tops
    - bottoms
  planned:
    - intimates_swimwear
    - coords_sets
 
input_requirements:
  model_image:
    color_space: rgb        # alpha channels rejected during pre-validation
    min_short_side: 1024
    max_faces: 1
  garment_image:
    background: any         # RMBG segments background prior to conditioning
    on_body_reference: preferred
 
rejections:
  category_unsupported:
    http: 422
    hint: "This category has no specialised model yet. See categories.planned."
  no_garment_found:
    http: 422
    hint: "Segmentation returned an empty mask for the requested category."
  sheer_fabric_unsupported:
    http: 422
    hint: "Lace and chiffon currently render opaque. Not accepted."
  pose_out_of_distribution:
    http: 422
    hint: "Non-standing, non-seated poses are outside the trained range."

Returning a clear 422 status code with diagnostic hints immediately informs client developers of input issues, preventing billing disputes over corrupted renders.

Failure modes and edge cases

Missing idempotency keys: Network timeouts during job submission caused clients to retry requests, occasionally generating duplicate GPU tasks. Adding request idempotency keys resolved duplicate submissions.
Aggressive webhook retry loops: Webhook delivery engines must use exponential backoff with circuit breakers to avoid overwhelming client staging servers during outages.
Unbounded manifest retention: Keeping every experimental manifest deployable indefinitely leads to disk bloat on inference workers. Deprecation schedules must accompany version releases.

When this is the wrong choice

The graph is still changing shape. While you are testing node architectures, rewiring a canvas beats updating a manifest schema, a seed derivation, and a rejection table for a topology you will throw away.
One operator reviews every output. An internal studio where an artist looks at each render can tune parameters by hand. Determinism buys you reproducibility for bugs nobody is going to file.
The pass finishes in under half a second. A synchronous HTTP endpoint is fine at that latency. Leasing queues, webhook retries, and idempotency keys exist because a full-body try-on pass takes about 12 seconds on a 4090.
You control both sides of the call. Manifests, 422 hints, and input contracts are for engineers who cannot ask you what the endpoint accepts.