What 5,000 images taught me about curation

The instinct when a generative model isn't good enough is to get more data. I had that instinct, acted on it for a few weeks, and the model got worse in a specific and embarrassing way: it started producing images that scored well on everything we could measure automatically and that our clients would not put on a product page. The fix was going in the other direction. The seed adapters behind AuraX-V1 were trained on 5,000 curated images, and the curation is the part I'd defend in a room full of people who disagree.

Why scale was the wrong lever

Fashion imagery on the open internet is abundant and mostly unusable for this.

Not because it's low quality in an obvious sense. Plenty of it is beautiful. It's unusable because the distribution is wrong for the job. Editorial fashion photography is dramatic: strong side lighting, deep shadow, heavy grade, sometimes motion blur as a stylistic choice. E-commerce photography is the opposite. Even lighting, the garment fully legible, a background that doesn't compete, a pose that shows the product rather than the photographer. If you train on a large pile of the former and ask for the latter, you get the former with a lighter background.

The second problem is that quantity buys you more of what's already common. Every additional ten thousand images scraped from public sources contains proportionally more t-shirts, more denim, more of exactly the garments the base model already handles. The categories where we needed help were the categories that were rare in any pile you could assemble quickly. Scaling the dataset scaled the bias.

So the question stopped being how many images we could get and became which images were worth a gradient step.

The technical gate

The first stage is mechanical and throws out a lot. We filtered public images on resolution, on sharpness, and on aspect ratio, in that order, because each one is cheaper than the next.

# filter_public_images.py
"""First-stage technical filter over candidate training images.
 
Nothing here is about taste. These are the checks that decide whether an
image is physically capable of teaching the model anything about fabric.
Aesthetic and brand judgement happen downstream, on what survives.
"""
from collections import Counter
from dataclasses import dataclass
from pathlib import Path
 
import cv2
import numpy as np
 
MIN_SHORT_SIDE = 768        # below this, weave detail is already gone
SHARPNESS_FLOOR = 120.0     # variance of Laplacian on the luma channel
ASPECT_RANGE = (0.55, 1.45) # portrait-ish through square; rejects banners
MAX_CLIP_FRAC = 0.02        # fraction of pixels blown to pure white
 
 
@dataclass(frozen=True)
class Verdict:
    """Outcome of the technical gate for one candidate image."""
 
    path: Path
    keep: bool
    reason: str
 
 
def inspect(path: Path) -> Verdict:
    """Apply the technical gate to a single image.
 
    Sharpness uses the variance of the Laplacian, which is a blur proxy, not
    a quality score. It reliably catches upscaled thumbnails and camera shake
    and says nothing at all about whether the photograph is any good.
    """
    img = cv2.imread(str(path))
    if img is None:
        return Verdict(path, False, "unreadable")
 
    h, w = img.shape[:2]
    if min(h, w) < MIN_SHORT_SIDE:
        return Verdict(path, False, "resolution")
 
    aspect = w / h
    if not ASPECT_RANGE[0] <= aspect <= ASPECT_RANGE[1]:
        return Verdict(path, False, "aspect")
 
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    if cv2.Laplacian(gray, cv2.CV_64F).var() < SHARPNESS_FLOOR:
        return Verdict(path, False, "sharpness")
 
    if float((gray >= 250).sum()) / gray.size > MAX_CLIP_FRAC:
        return Verdict(path, False, "blown_highlights")
 
    return Verdict(path, True, "ok")
 
 
def run(root: Path) -> Counter:
    """Walk a candidate directory and tally why images were dropped."""
    tally: Counter = Counter()
    for path in sorted(root.rglob("*.jpg")):
        verdict = inspect(path)
        tally[verdict.reason] += 1
        if verdict.keep:
            print(path)
    for reason, n in tally.most_common():
        print(f"{reason:>18}: {n}", file=__import__("sys").stderr)
    return tally

Two notes on that. The aspect ratio check looks like the least interesting gate and turned out to be one of the most useful, because a wide crop of a fashion image is almost always a banner, a lookbook spread with three models in it, or a detail shot with no garment context. It's a cheap proxy for "this is one person wearing one outfit".

The blown highlights check went in after we noticed the model producing garments with flat white regions where a fold should be. Overexposed training images teach the model that white is a legitimate value for a shadow-facing surface, and once it learns that, silk and satin come out looking like paper.

Sharpness is the one people over-trust. Variance of Laplacian catches blur. It cannot tell you an image is well composed or well lit, and if you treat it as a quality score you will keep a lot of sharp, ugly photographs.

What curated actually meant

Past the technical gate, the corpus was organised by material rather than by garment type, which was the decision I'd most want to explain to someone starting this.

The categories the seed adapters needed to cover were denim, silk and leather. Those are not product categories, they're behaviours. Denim holds a crease and has a visible twill that has to survive at the scale a viewer zooms to. Silk has specular highlights that move with the drape and a falloff that gives away whether the model understands the surface or is painting a gradient. Leather sits between them, with a sheen that reads as fake the moment it's uniform.

Organising by material meant the balance question became answerable. A jacket in leather and a jacket in denim are two different training examples for our purposes even though a product catalogue would file them together.

# seed_corpus.yaml
# Manifest for the 5,000-image curated corpus behind the seed adapters.
# Every entry below is a gate or a policy. Per-material counts live in the
# generated index, not here, because they moved every time we re-reviewed.
 
technical_gate:
  min_short_side: 768
  sharpness_floor: 120.0
  aspect_range: [0.55, 1.45]
  max_clip_frac: 0.02
 
materials:
  - denim
  - silk
  - leather
 
policy:
  balance_by: material
  max_share_single_source: 0.15   # no single origin dominates a material
  require_full_garment_visible: true
  reject_if_face_occluded: false  # faces are not what these adapters learn
  duplicate_check: perceptual_hash
 
review:
  stage_1: technical_gate
  stage_2: brand_aesthetic_score
  stage_3: human_pass          # every surviving image seen by a person

The max_share_single_source policy exists because our first balanced corpus was balanced by material and completely unbalanced by photographer. One source can supply a thousand technically excellent silk images that all share a lighting setup, and the adapter learns the lighting setup.

The line I'd underline is stage_3. Every image that made it into the 5,000 was looked at by a human being. That is feasible at five thousand and it is not feasible at fifty thousand, and I think the constraint was doing more work than the number.

One thing worth keeping straight, because I've conflated it myself in conversation: this 5,000-image corpus is not the same corpus as the draping physics expert. That one is a separate 5,000 images of sarees, hanfus, kimonos and complex haute couture, assembled for a different problem. The occlusion and depth expert is a third set, 3,000 images of hands on hips, arms crossed and jewellery worn over clothing. Same order of magnitude, unrelated contents.

The scorer we didn't want to build

We tried to automate stage 2 with an off-the-shelf aesthetic scorer, and this is where the interesting failure lives.

Generic aesthetic models, including some very good recent ones, have a learned bias toward darker and moodier imagery. Ask one to rank a batch of fashion photographs and it will consistently prefer the one with crushed blacks, a warm grade and a strong key light. That preference is real and it reflects what people upvote on the internet, which is what those models were trained on.

It is also precisely backwards for our customer. No e-commerce brand is going to ship a moody product shot where the garment is half in shadow. The whole job of the image is to show the product clearly. Using a generic scorer as a filter meant systematically selecting training data that pushed the model away from what our clients would accept, while the score went up.

So we trained our own, on our own labels. A small head over frozen image embeddings, fit on pairwise preferences rather than absolute ratings, because people are bad at giving a photo a 7 out of 10 and quite good at saying which of two photos they'd rather put on a product page.

# train_brand_aesthetic.py
"""Fit the Brand-Centric Aesthetic Model on in-house pairwise preferences.
 
Absolute ratings drift between sessions and between people. Pairwise choices
are stable enough to learn from, so every label is "A over B for a product
page", and the head learns a scalar whose ordering matches those choices.
"""
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, Dataset
 
EMBED_DIM = 1024
HIDDEN = 256
LR = 1e-4
EPOCHS = 20
MARGIN = 0.2
 
 
class PreferencePairs(Dataset):
    """Precomputed embeddings for (preferred, rejected) image pairs."""
 
    def __init__(self, pairs: list[tuple[torch.Tensor, torch.Tensor]]):
        self.pairs = pairs
 
    def __len__(self) -> int:
        return len(self.pairs)
 
    def __getitem__(self, i: int):
        return self.pairs[i]
 
 
class AestheticHead(nn.Module):
    """Maps a frozen image embedding to a single commercial-fitness scalar."""
 
    def __init__(self, dim: int = EMBED_DIM, hidden: int = HIDDEN):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(dim, hidden),
            nn.GELU(),
            nn.Dropout(0.1),
            nn.Linear(hidden, hidden // 2),
            nn.GELU(),
            nn.Linear(hidden // 2, 1),
        )
 
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.net(x).squeeze(-1)
 
 
def train(pairs: PreferencePairs) -> AestheticHead:
    """Train with a margin ranking loss on preferred-over-rejected pairs."""
    head = AestheticHead().cuda()
    opt = torch.optim.AdamW(head.parameters(), lr=LR)
    loader = DataLoader(pairs, batch_size=64, shuffle=True)
    target = torch.ones(1).cuda()
    criterion = nn.MarginRankingLoss(margin=MARGIN)
 
    for epoch in range(EPOCHS):
        running = 0.0
        for better, worse in loader:
            s_better = head(better.cuda())
            s_worse = head(worse.cuda())
            loss = criterion(s_better, s_worse, target.expand_as(s_better))
            opt.zero_grad()
            loss.backward()
            opt.step()
            running += loss.item()
        print(f"epoch {epoch:02d} mean_loss={running / max(len(loader), 1):.4f}")
 
    return head

The honest description of where those pairs came from is that we scored images ourselves and argued about it. There was no panel, no protocol document, no measured agreement statistic. It was a handful of people who had spent the previous month on calls with brand teams, sitting with a folder of images and disagreeing out loud until a rule emerged that we could apply consistently. Some of those arguments took an hour and produced one line of guidance, like whether a visible garment wrinkle is realism or a defect. (It depends on the material, which is why the corpus is organised by material.)

I'm not going to dress that up as a study. It's a small group of people encoding a commercial taste they had absorbed from customers, and its main virtue is that it was our customers' taste rather than the internet's.

What broke

The scorer overfit to our own reviewing habits before it overfit to anything else. Early versions learned that images we had cropped in a particular way were preferred, because the ones we'd bothered to crop were the ones we liked.

Balance by material did not give us balance by body type or by skin tone, and I did not catch that until later than I should have. A corpus can be carefully curated along the axis you're thinking about and unexamined along every other axis.

And there's a ceiling. Curation gets you a model that reliably produces the kind of image you selected for. It cannot produce a kind of image that wasn't in the corpus at all. When a client asked for a look outside what we'd curated, the answer was more curation, not more sampling.

What I learned, and where this goes next

Filter for physical capability first and taste second, and keep the two stages separate in code. Mixing them produces a scorer that quietly rejects good photographs for being slightly soft and accepts sharp ones that nobody would ship.

A generic quality metric encodes somebody's preferences, and if you don't know whose, you're optimising toward a stranger. The darker-and-moodier bias was the clearest example I've hit of a metric that was working correctly and pointed the wrong way.

Five thousand was not a target we chose. It's the number of images that survived a process where a person looked at every one, and I now think the real constraint was human attention rather than dataset size. Any process that scales past what you can personally inspect had better have something else keeping it honest.

Organise the corpus by the thing the model has to learn. Material behaviour was the right axis for these adapters. It would be the wrong axis for the occlusion expert, where the axis is the pose.

The next piece of this is closing the loop with the product. Users favouriting and downloading generated images is a preference signal collected at a scale no folder review can reach, and it comes from the people whose taste actually matters. Turning that into training pairs, without letting it drag the scorer back toward whatever is merely popular, is the problem I'm sitting with now.