---
title: "From a ComfyUI graph to an API a brand's team can call"
description: "Our try-on pipeline lived as a ComfyUI graph with implicit state, hand-picked seeds, and custom nodes tracking whatever was on main that week. Turning it into something another company's engineers could call meant writing down which code ran, which seed, when the work happens, and what we refuse."
date: "September 09, 2025"
url: "https://himanshuat.com/blogs/from-comfyui-graph-to-an-api"
---
# From a ComfyUI graph to an API a brand's team can call

During one integration week, our endpoint received three distinct inputs: a 4K studio flat-lay of a lehenga, a smartphone photo of a kurta hanging on a door, and a PNG with an alpha channel. All three came from engineers who reasonably assumed a virtual try-on API accepts any image of clothing.

The alpha channel crashed our PIL preprocessor, resulting in a terminal traceback on my machine and a 504 gateway timeout for the caller.

Running the workflow interactively in ComfyUI lets you intervene. You watch the browser tab, drop the denoise when fabric goes plastic, and re-seed until segmentation stops clipping fingers. Once beta API keys went out to design partners like GettoIndia and EcomBuddha, nobody was watching the tab.

> A ComfyUI graph is not an API with the wrapper missing. What makes it callable by someone else is the part the graph never had to write down: which code ran, which seed, when the work happens, and what you refuse.

## Operational gaps between UI graphs and backend services

Our initial API wrapper took a single day to build. ComfyUI natively exposes an HTTP endpoint accepting workflow graphs in JSON format. I built a lightweight Python service that loaded our saved workflow, injected incoming model and garment URLs into the `LoadImage` nodes, dispatched the execution, and polled for output images.

While sufficient for internal demos, this setup failed across four areas when exposed to external engineering teams:

| Failure Vector | Underlying System State | Impact on API Client |
|---|---|---|
| Floating dependencies | Custom nodes (SAM2, RMBG, LoRA merging) tracked `main` git branches | Unversioned, non-reproducible output across deployments |
| Resident GPU state | ComfyUI process cached in-memory LoRA merges from earlier calls | Changing weights on disk without restarting workers served stale cached layers |
| Unstructured exceptions | VRAM allocations, aspect-ratio mismatches, and empty masks threw Python tracebacks | HTTP timeouts without actionable error messages |
| Missing input validation | No formal constraints on color spaces, dimensions, or apparel categories | Silent generation failures on unsupported garments |

To transition from an internal research tool to an enterprise service, we defined four operational guarantees: explicit code manifests, deterministic seeding, asynchronous job processing, and rigid input validation.

---

## Step 1: Versioning execution manifests rather than UI graphs

We replaced loose workflow JSON files with comprehensive manifests that track model weight hashes, exact git commits for all node packages, and fixed merge hyperparameters.

```json
// manifests/vton-2025-09-a.json
{
  "manifest_id": "vton-2025-09-a",
  "created": "2025-09-02",
  "graph": {
    "file": "graphs/flux_vton_plus.api.json",
    "sha256": "9f2c1e0b7a4d3f88c5b6e2a1d0c9f4b3e8a7d6c5b4a39281706f5e4d3c2b1a09"
  },
  "nodes": [
    { "repo": "comfyui-sam2", "commit": "4b1e9c2" },
    { "repo": "comfyui-rmbg", "commit": "a07d331" },
    { "repo": "comfyui-essentials", "commit": "c19f4ab" }
  ],
  "models": [
    { "role": "base", "file": "flux_fill.safetensors", "sha256": "1a2b3c4d" },
    { "role": "redux", "file": "flux_redux.safetensors", "sha256": "5e6f7a8b" },
    { "role": "vae", "file": "ae.safetensors", "sha256": "9c0d1e2f", "dtype": "fp16" },
    { "role": "lora_drape", "file": "drape_physics_r32.safetensors", "sha256": "3a4b5c6d" },
    { "role": "lora_occlusion", "file": "occlusion_depth_r32.safetensors", "sha256": "7e8f9a0b" }
  ],
  "merge": {
    "node": "model_merge_lora",
    "lambda_drape": 0.6,
    "lambda_occlusion": 0.4
  },
  "sampling": {
    "sampler": "euler_ancestral",
    "scheduler": "beta",
    "steps_primary": 30,
    "steps_refine": 10
  },
  "context_window": {
    "roi_padding": 0.2,
    "mask_dilation_px": 9,
    "inference_resolution": [1344, 1344]
  }
}
```

Manifests eliminate ambiguity around adapter ratios (such as 0.6 drape and 0.4 occlusion). Every API request pins a specific manifest ID, and responses echo the exact manifest executed. Old manifests remain pinned in production as long as their corresponding weights reside on disk, preventing silent breaking changes for live client catalogs.

---

## Step 2: Deriving deterministic seeds from request payloads

Interactive diffusion relies on randomized seeds to generate candidate varieties. In a production API, randomized execution prevents caching, invalidates regression test suites, and makes customer bugs unreproducible.

```python
# api/seeding.py
import hashlib
import json
from dataclasses import dataclass


@dataclass
class TryOnRequest:
    """A single try-on job received from an API caller."""
    request_id: str
    model_image_key: str
    garment_image_key: str
    category: str
    manifest_id: str
    seed: int | None = None


def derive_seed(req: TryOnRequest) -> int:
    """Return a deterministic seed for the request.

    If the caller provides an explicit seed, we use it directly to support
    exact replays. Otherwise, we hash the request payload into a uint32.
    Worker retries following transient hardware failures land on the identical
    seed rather than altering fabric drape.
    """
    if req.seed is not None:
        return req.seed % (2 ** 32)

    payload = json.dumps(
        {
            "model": req.model_image_key,
            "garment": req.garment_image_key,
            "category": req.category,
            "manifest": req.manifest_id,
        },
        sort_keys=True,
    ).encode()
    return int.from_bytes(hashlib.sha256(payload).digest()[:4], "big")


def apply_seed(graph: dict, seed: int) -> dict:
    """Inject the computed seed into all sampler nodes and lock randomisation."""
    for node in graph.values():
        if node.get("class_type") in {"KSampler", "KSamplerAdvanced"}:
            node["inputs"]["seed"] = seed
            node["inputs"]["noise_seed"] = seed
            node["inputs"]["control_after_generate"] = "fixed"
    return graph
```

Identical seeds run on matching GPU architectures with identical drivers yield visually consistent renders. Responses always return the derived seed so brand creative directors can store approved seeds across SKU batches.

---

## Step 3: Asynchronous job queues and worker leasing

A full-body try-on pass takes approximately 12 seconds on an RTX 4090. Holding synchronous HTTP connections open for that duration creates connection pool exhaustion and exposes clients to gateway timeouts.

We designed the service around asynchronous job polling and webhooks using Redis for queue management.

```mermaid
flowchart LR
  A[submit] --> B{under limit}
  B -->|no| R[rejected]
  B -->|yes| Q[202 queued]
  Q --> W[worker lease]
  W -->|crash| Q
  W --> D[result]
  D --> P[poll or webhook]
```

```python
# api/queue.py
import json
import time
import uuid

import redis

r = redis.Redis(decode_responses=True)

QUEUE_KEY = "vton:pending"
LEASE_TTL_SECONDS = 180
MAX_INFLIGHT_PER_KEY = 4


def submit(api_key: str, payload: dict) -> dict:
    """Validate concurrency and register a queued job."""
    inflight = r.scard(f"vton:inflight:{api_key}")
    if inflight >= MAX_INFLIGHT_PER_KEY:
        return {"status": "rejected", "reason": "concurrency_limit"}

    job_id = f"job_{uuid.uuid4().hex[:12]}"
    record = {
        "job_id": job_id,
        "api_key": api_key,
        "status": "queued",
        "manifest_id": payload["manifest_id"],
        "submitted_at": time.time(),
        "payload": payload,
    }
    r.hset(f"vton:job:{job_id}", mapping={"data": json.dumps(record)})
    r.sadd(f"vton:inflight:{api_key}", job_id)
    r.lpush(QUEUE_KEY, job_id)
    return {"job_id": job_id, "status": "queued", "queue_depth": r.llen(QUEUE_KEY)}


def claim(worker_id: str) -> dict | None:
    """Lease a job from the pending queue.

    If a worker crashes mid-inference, the lease expires and an automated
    sweeper requeues the job. Deterministic seeding ensures that the retry
    generates the expected image.
    """
    job_id = r.brpoplpush(QUEUE_KEY, f"vton:leased:{worker_id}", timeout=30)
    if job_id is None:
        return None
    r.setex(f"vton:lease:{job_id}", LEASE_TTL_SECONDS, worker_id)
    raw = r.hget(f"vton:job:{job_id}", "data")
    record = json.loads(raw)
    record["status"] = "running"
    r.hset(f"vton:job:{job_id}", mapping={"data": json.dumps(record)})
    return record
```

Our architecture uses Next.js for client interfaces, NestJS for REST APIs and PostgreSQL metadata management, Redis for leasing queues, S3 for object storage, and headless Python workers dedicated strictly to consuming jobs from the queue.

---

## Step 4: Explicit API input contracts and rejection policies

Rather than attempting to render unsupported edge cases and producing broken outputs, the API validates inputs up front and returns descriptive HTTP 422 errors.

```yaml
# api/surface.yaml
categories:
  supported:
    - dresses_one_pieces
    - tops
    - bottoms
  planned:
    - intimates_swimwear
    - coords_sets

input_requirements:
  model_image:
    color_space: rgb        # alpha channels rejected during pre-validation
    min_short_side: 1024
    max_faces: 1
  garment_image:
    background: any         # RMBG segments background prior to conditioning
    on_body_reference: preferred

rejections:
  category_unsupported:
    http: 422
    hint: "This category has no specialised model yet. See categories.planned."
  no_garment_found:
    http: 422
    hint: "Segmentation returned an empty mask for the requested category."
  sheer_fabric_unsupported:
    http: 422
    hint: "Lace and chiffon currently render opaque. Not accepted."
  pose_out_of_distribution:
    http: 422
    hint: "Non-standing, non-seated poses are outside the trained range."
```

Returning a clear 422 status code with diagnostic hints immediately informs client developers of input issues, preventing billing disputes over corrupted renders.

---

## Failure modes and edge cases

1. **Missing idempotency keys:** Network timeouts during job submission caused clients to retry requests, occasionally generating duplicate GPU tasks. Adding request idempotency keys resolved duplicate submissions.
2. **Aggressive webhook retry loops:** Webhook delivery engines must use exponential backoff with circuit breakers to avoid overwhelming client staging servers during outages.
3. **Unbounded manifest retention:** Keeping every experimental manifest deployable indefinitely leads to disk bloat on inference workers. Deprecation schedules must accompany version releases.

---

## When this is the wrong choice

- **The graph is still changing shape.** While you are testing node architectures, rewiring a canvas beats updating a manifest schema, a seed derivation, and a rejection table for a topology you will throw away.
- **One operator reviews every output.** An internal studio where an artist looks at each render can tune parameters by hand. Determinism buys you reproducibility for bugs nobody is going to file.
- **The pass finishes in under half a second.** A synchronous HTTP endpoint is fine at that latency. Leasing queues, webhook retries, and idempotency keys exist because a full-body try-on pass takes about 12 seconds on a 4090.
- **You control both sides of the call.** Manifests, 422 hints, and input contracts are for engineers who cannot ask you what the endpoint accepts.

---

Source: https://himanshuat.com/blogs/from-comfyui-graph-to-an-api