Fitting a saree into 24GB: the context window trick

A saree fails at low resolution in a specific way. The pallu stops reading as fabric folded over a shoulder and becomes a printed pattern on skin, and the pleats at the waist collapse into a smear. You need pixel density on the garment for the model to have anything to work with. What you can't do is buy that density by running the whole frame at high resolution, because a full body shot at the resolution the pleats need will not fit on a 24GB card.

The whole frame is the wrong unit

Start with what a full body e-commerce shot actually contains. A person occupies the middle of a tall frame. The garment occupies maybe a third of that. The rest is backdrop, floor, headroom, and a lot of pixels that the diffusion model is going to encode, denoise for every step, and decode, in order to reproduce something we already had and didn't want changed.

At native inference resolution the model is doing useful work everywhere. Scale the frame up to the point where the pleats survive and most of that work is now being spent on a grey sweep. Memory in a diffusion transformer scales with the number of latent tokens, and the token count grows with area, so doubling the linear resolution of the frame roughly quadruples what you need to hold. That's how you get to a place where a 48GB L40S is comfortable and a consumer 24GB card is not, on an image that is mostly backdrop.

We had to care about 24GB because that's what a lot of the hardware our work would eventually run on looks like. Designing the pipeline so it only ever ran on datacentre cards would have been a decision with consequences we couldn't take back later.

Why naive tiling breaks pleats

The standard answer to a big image is to cut it into tiles, run each, and stitch. That's the right answer for upscaling a photograph of a hillside and the wrong answer here.

A pleat is a continuous structure. It starts at the waist, runs down, and its shadow, direction and spacing are consistent along its whole length because gravity is consistent. Cut that structure across two tiles and each tile is denoised by a model that cannot see the other half. The model has a prior about fabric and it will happily produce plausible pleats in both tiles. It has no mechanism at all for making the pleat in tile B continue the pleat in tile A at the same angle, spacing and phase. You get two convincing halves of two different garments meeting at a line.

Overlap and blending between tiles reduces the visible discontinuity without fixing it. Blending averages two incompatible structures, and the average of two out of phase pleat fields is a mush that reads as a manufacturing defect. This is the same reason tiled upscaling is bad at text and faces: it fails on anything where the correct value of a pixel depends on content further away than the tile.

The structures that matter most for the garments we cared about are exactly the long range ones. So tiling was out, and the question became how to give the model high pixel density over a continuous garment without asking it to hold the whole frame.

The context window

The answer is to change what the model sees rather than how it processes what it sees. Crop to the garment plus context, work at native resolution on that crop, and put the result back. Four steps.

1. Extract the ROI

Take the bounding box of the final mask and pad it. The padding factor matters more than it looks: too tight and the model has no surrounding body to anchor the garment against, and it will render a garment that doesn't know where the shoulders are. We use rho = 0.2.

# pipeline/context_window.py
"""Crop, upscale, inpaint and composite: the context window mechanism.
 
The unit of inference is the garment region, not the frame. Everything
here exists to make the model see a high pixel density crop while the
delivered asset stays full resolution.
"""
 
from __future__ import annotations
 
from dataclasses import dataclass
 
import numpy as np
 
 
NATIVE_RESOLUTIONS = ((1024, 1024), (1344, 1344))
ROI_PADDING = 0.2
 
 
@dataclass(frozen=True)
class Window:
    """A crop rectangle in the coordinate space of the source image."""
 
    x0: int
    y0: int
    x1: int
    y1: int
 
    @property
    def size(self) -> tuple[int, int]:
        return self.x1 - self.x0, self.y1 - self.y0
 
 
def roi_from_mask(mask: np.ndarray, rho: float = ROI_PADDING) -> Window:
    """Bounding box of the mask, padded by rho and clamped to the frame.
 
    rho is a fraction of the box dimension, applied on every side, so a
    tall narrow mask gets proportionally more horizontal context than a
    fixed pixel pad would give it.
    """
    ys, xs = np.nonzero(mask)
    if ys.size == 0:
        raise ValueError("empty mask: nothing to inpaint")
 
    y0, y1 = int(ys.min()), int(ys.max())
    x0, x1 = int(xs.min()), int(xs.max())
 
    pad_y = int((y1 - y0) * rho)
    pad_x = int((x1 - x0) * rho)
 
    h, w = mask.shape[:2]
    window = Window(
        x0=max(0, x0 - pad_x),
        y0=max(0, y0 - pad_y),
        x1=min(w, x1 + pad_x),
        y1=min(h, y1 + pad_y),
    )
 
    print(f"mask box: {(x0, y0, x1, y1)}")
    print(f"window:   {(window.x0, window.y0, window.x1, window.y1)} size={window.size}")
    return window
 
 
def pick_native(window: Window) -> tuple[int, int]:
    """Choose the inference resolution whose aspect is closest to the crop."""
    crop_w, crop_h = window.size
    crop_aspect = crop_w / crop_h
    return min(
        NATIVE_RESOLUTIONS,
        key=lambda r: abs((r[0] / r[1]) - crop_aspect),
    )

The padding is a fraction of the box rather than a fixed pixel count on purpose. A saree mask is tall and narrow; a fixed pad would give it a sliver of context on the sides where it needs the most, because the model has to place the drape relative to a torso it can see.

2. Upscale to native resolution

The crop gets resized up to whatever the model actually wants, which for us is 1024x1024 or 1344x1344. This is the step that buys the pixel density. The garment now occupies most of a native resolution canvas instead of a third of a downscaled frame, and the pleats have enough pixels to survive encoding into latents.

3. Inpaint the crop

The inpainting model runs on the high fidelity crop and nothing else. Same mask, same conditioning, same expert weights, applied to a much better input.

# pipeline/context_window.py (continued)
 
INFERENCE = {
    "primary_steps": 30,
    "refinement_steps": 10,
    "sampler": "euler_ancestral",
    "scheduler": "beta",
    "vae_dtype": "fp16",
    "mask_dilation_px": 12,   # bleeding zone, tuned per garment in 5..15
}
 
 
def run_window(
    image: np.ndarray,
    mask: np.ndarray,
    pipe,
    conditioning,
) -> np.ndarray:
    """Inpaint the garment region at native resolution and return the crop."""
    window = roi_from_mask(mask)
    target = pick_native(window)
 
    crop = resize(image[window.y0:window.y1, window.x0:window.x1], target)
    crop_mask = resize(mask[window.y0:window.y1, window.x0:window.x1], target)
    crop_mask = dilate(crop_mask, radius=INFERENCE["mask_dilation_px"])
 
    generated = pipe(
        image=crop,
        mask_image=crop_mask,
        num_inference_steps=INFERENCE["primary_steps"],
        **conditioning,
    ).images[0]
 
    generated = pipe(
        image=generated,
        mask_image=crop_mask,
        num_inference_steps=INFERENCE["refinement_steps"],
        strength=0.25,
        **conditioning,
    ).images[0]
 
    print(f"inference at {target}, window {window.size}")
    return np.asarray(generated)

The dilation on the mask is the same bleeding zone we use everywhere, a radius somewhere between 5 and 15 pixels depending on the garment. It lets the model see skin adjacent to the garment so the boundary between fabric and body is something it generates rather than something we impose afterwards.

4. Composite back

Downscale the generated crop to the window's original size and paste it into the full resolution frame. The delivered asset keeps the resolution of the source photograph; only the region inside the window has been touched.

The seam is the whole difficulty

That last step is where the mechanism stops being straightforward, because pasting a rectangle of generated pixels into a photograph produces a visible rectangle.

There are two distinct causes and they need different treatment. The first is a hard edge at the window boundary, which you fix by feathering the alpha over the dilated mask rather than over the window rectangle, so the transition happens across the bleeding zone where the model already blended fabric into skin. Compositing on the window edge is the naive version and it's the one that produces a faint rectangular outline that a brand's art director will find in about four seconds.

The second cause is harder. The crop was generated at native resolution and then downscaled; the surrounding frame was never resampled. Those two regions have different noise characteristics and different high frequency content, and even with a perfect alpha the eye picks up the change in grain across the boundary. Downscaling is a low pass filter, so the generated region comes back slightly smoother than the photograph it's landing in. You can match grain, or you can arrange for the boundary to fall in a region that has little high frequency detail anyway, which the padding factor helps with by pushing the boundary out into flatter skin and backdrop.

We got this to a point where output scales to 4K for e-commerce print use. The quality was best with 2K input where the source garment photography was already high quality, which is worth saying plainly: this mechanism raises the ceiling that resolution imposes, and it does not manufacture detail that was never in the reference. A soft input garment produces a soft output garment at any resolution you like.

For reference on cost, a full body try-on takes about 12 seconds on an RTX 4090. That's a lab measurement on a single image, not a production figure, and I don't have a production figure to give you.

What this post doesn't cover

The four steps above are the mechanism. What I'm not documenting here is how the window strategy generalises when one window isn't enough: how many windows get used for a given garment, how they're placed, how much they overlap, and how the blend across multiple windows is handled. Those choices exist, they matter for the garments that don't fit in one crop, and they're not something I'm writing up. Take the four steps as the description of the single window case, which is the case that covers most e-commerce shots.

What I learned

Cropping is a modelling decision, not a preprocessing detail. The padding factor changes what the model knows about the body it's dressing, and it belongs in the same review as the sampler and the step count.

The failures that kill you are the ones at boundaries. The generation was never the hard part after we had the adapters. Getting generated pixels to sit inside a photograph without announcing themselves took longer than getting the pleats right.

Resolution constraints propagate into product decisions. Because we designed against 24GB, the pipeline runs in more places than it would have otherwise, and that constraint improved the architecture rather than compromising it.

A mechanism that raises a ceiling still has a floor. We spent a while confused by inconsistent results before recognising that the variable was input photography quality, which no amount of inference resolution can fix.

Next step is measuring the seam properly. Right now a boundary artefact is caught by a person looking at the image, which means it's caught late and inconsistently, and it should be a check that runs on every generated asset before anyone sees it.