Generic aesthetic scorers hate e-commerce photography

When we ran an off-the-shelf aesthetic scoring model over a batch of generated product images, it consistently ranked dramatic, low-key shots highest: heavy rim lighting, deep cast shadows, half-occluded faces, and dark textured backdrops.

They are good photographs. As catalog assets they are unusable.

Generic aesthetic models learn preferences from online photography communities where mood and dramatic contrast are rewarded. E-commerce catalogs require uniform lighting, accurate textile colors, and clear garment geometry.

Dual roles of automated scorers in generative pipelines

Automated scoring does two jobs in our pipeline:

1. Training set curation: Filtering out technically flawed or visually unsuitable photos prior to fine-tuning.
2. Inference candidate selection: Ranking multiple diffusion samples to select the best output for client presentation.

When a scorer filters training data and subsequently ranks generation candidates from that trained model, its specific biases compound. If the scorer over-rewards dark, moody aesthetics, the model learns to generate darker imagery, and the inference filter selects the darkest outputs from that distribution.

Step 1: Aligning evaluation criteria with catalog requirements

Commercial e-commerce imagery demands properties that contradict general photography awards:

Color fidelity directly impacts return rates: apparel that arrives in a shade different from the product page gets returned. Using an uncalibrated general aesthetic scorer systematically selects outputs that commercial brands reject.

Step 2: Collecting pairwise preferences rather than absolute ratings

Human raters struggle with 1-to-10 scales. The same rater will assign different scores to identical images across sessions, and two raters rarely agree on the boundary between a 6 and a 7.

Pairwise comparisons ("Which of these two images better fits this brand's catalog?") yield far more consistent training data.

# scorer/collect_preferences.py
"""Pairwise preference collection for brand-specific aesthetic evaluation.
 
Raters compare two images and select which better fits a catalog.
Pairwise decisions provide stable gradient signals compared to noisy
absolute numerical ratings.
"""
 
import json
import random
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from pathlib import Path
 
 
@dataclass
class Comparison:
    brand_id: str
    left: str
    right: str
    winner: str          # "left", "right", or "skip"
    rater_id: str
    flipped: bool        # tracks whether display order was reversed
    recorded_at: str
 
 
def sample_pair(pool: list[str], seen: set[frozenset]) -> tuple[str, str] | None:
    """Draw an unseen pair. Returns None once pool is exhausted."""
    for _ in range(200):
        left, right = random.sample(pool, 2)
        if frozenset((left, right)) not in seen:
            return left, right
    return None
 
 
def present(left: str, right: str) -> tuple[str, str, bool]:
    """Randomize display order to cancel out screen position bias."""
    flipped = random.random() < 0.5
    return (right, left, True) if flipped else (left, right, False)
 
 
def record(out_path: Path, comparison: Comparison) -> None:
    """Append one comparison record. Skipped pairs mark ambiguous boundaries."""
    with out_path.open("a") as handle:
        handle.write(json.dumps(asdict(comparison)) + "\n")
 
 
def session(brand_id: str, rater_id: str, pool: list[str], out_path: Path,
            target: int = 100) -> None:
    seen: set[frozenset] = set()
    collected = 0
 
    while collected < target:
        pair = sample_pair(pool, seen)
        if pair is None:
            break
 
        left, right = pair
        shown_a, shown_b, flipped = present(left, right)
        choice = prompt_rater(shown_a, shown_b)   # UI presentation hook
 
        seen.add(frozenset(pair))
        record(out_path, Comparison(
            brand_id=brand_id,
            left=left,
            right=right,
            winner=choice,
            rater_id=rater_id,
            flipped=flipped,
            recorded_at=datetime.now(timezone.utc).isoformat(),
        ))
        collected += 1
 
    print(f"brand={brand_id} rater={rater_id} collected={collected} "
          f"pairs_exhausted={pair is None}")

Randomizing presentation position mitigates left/right bias, and preserving skipped choices helps identify ambiguous feature boundaries.

Step 3: Tri-bucket routing for human-in-the-loop review

Rather than using a single pass/fail threshold, our inference pipeline sorts generated candidates into three tiers: auto-accepted, human-review required, and rejected.

# scorer/rank_candidates.py
"""Rank generation candidates using brand policy thresholds.
 
Sorts outputs into surfaced, review, and rejected buckets based on
per-brand score thresholds.
"""
 
from dataclasses import dataclass
from typing import Protocol
 
import yaml
 
 
class BrandScorer(Protocol):
    def score(self, image_path: str, brand_id: str) -> float:
        """Higher indicates better commercial alignment for target brand."""
        ...
 
 
@dataclass(frozen=True)
class BrandPolicy:
    brand_id: str
    accept_threshold: float     # candidates scoring above this are auto-surfaced
    review_threshold: float     # candidates in middle band route to human review
    max_surfaced: int
 
 
def load_policy(path: str, brand_id: str) -> BrandPolicy:
    with open(path) as handle:
        config = yaml.safe_load(handle)[brand_id]
    return BrandPolicy(brand_id=brand_id, **config)
 
 
def rank(scorer: BrandScorer, candidates: list[str],
         policy: BrandPolicy) -> dict[str, list[str]]:
    """Partition candidates across three operational tiers."""
    scored = sorted(
        ((path, scorer.score(path, policy.brand_id)) for path in candidates),
        key=lambda item: item[1],
        reverse=True,
    )
 
    surfaced, review, rejected = [], [], []
    for path, value in scored:
        if value >= policy.accept_threshold and len(surfaced) < policy.max_surfaced:
            surfaced.append(path)
        elif value >= policy.review_threshold:
            review.append(path)
        else:
            rejected.append(path)
 
    print(f"brand={policy.brand_id} candidates={len(candidates)} "
          f"surfaced={len(surfaced)} review={len(review)} rejected={len(rejected)}")
    return {"surfaced": surfaced, "review": review, "rejected": rejected}

Three buckets stop the model from having to make a binary call right at its decision boundary. The review tier absorbs the borderline candidates, so automation only handles the clear passes and the clear rejections.

Step 4: Calibrating preferences per brand profile

Commercial aesthetics vary across fashion brands:

• High-contrast commercial: hard direct lighting, pure white cyc backdrops, smoothed skin tones.
• Editorial natural: diffuse window light, warm neutral backdrops, natural skin texture with visible pores.

Both brands describe their requirement as "clean commercial imagery," yet an asset approved by one will be rejected by the other. Scoring heads must be calibrated against specific brand profile datasets.

Step 5: Documenting evaluation boundaries

We developed this scoring setup as an operational filtering pipeline rather than a formal academic study.

Automated aesthetic scoring evaluates commercial style alignment, not garment correctness. Separate checks must validate garment fidelity, occlusion boundaries, and pleat geometry. An aesthetically pleasing image of the wrong garment remains a defect.

Failure modes and edge cases

1. Aesthetic collapse toward flat monotony: Over-penalizing shadows can cause models to generate lifeless, uniformly washed-out imagery.
2. Seasonal style drift: Catalog guidelines evolve across seasonal collections. Scorers trained on autumn lookbooks reject spring visual directions unless recalibrated.
3. Confusing aesthetic score with try-on accuracy: Aesthetic heads can assign high scores to beautifully lit images where logos or print motifs are distorted.

When this is the wrong choice

• You are still prototyping. Looking at the first batch of outputs yourself is faster than standing up a pairwise comparison UI, recruiting raters, and collecting enough decisions to train a head. Build the scorer when eyeballing candidates becomes the bottleneck.
• The goal is art, not catalog. Everything here is a correction for the fact that open scorers reward mood and contrast. If mood and contrast are what you want, the off-the-shelf scorer is already aligned with you and a custom head is work spent arriving where you started.
• The brand's creative direction is still moving. Preference labels encode a specific look. Collect them against guidelines that change next quarter and you have a training set that argues for the old direction.
• You need garment correctness, not style alignment. An aesthetic head scores lighting, framing, and color. It will happily rank a beautifully lit image of a distorted logo at the top. That check is a different model, and building this one does not get you it.