We beat the benchmarks and ran out of runway

We started AuraX in December 2024, and we're winding it down. The decision is made even though the last of the paperwork isn't, so I'm writing this from the middle rather than from the other side. We were incubated at IIIT-Hyderabad's CIE, we built a virtual try-on pipeline that outperforms the baselines we tested it against, and we signed commercial deals with Aditya Birla Fashion Group and several Mensa Brands labels. None of that was enough, and I want to write down what the work got right while the details are still in my head, and be equally specific about the part where being right didn't matter.

The numbers were real

The evaluation set was 500 images, split evenly between what we called Western Casual and Global Traditional. That split was the whole point. Anyone can post a good t-shirt result.

Against SDXL with ControlNet, Flux-VTON+ moved FID from 28.4 to 18.5. On Western garments the SSIM improvement was respectable and unremarkable, 0.82 to 0.94, because Western garments were never the hard part. On Global Traditional the number went from 0.45 to 0.85. Occlusion accuracy, measured by human raters judging whether a hand resting on a hip survived the generation intact, went from 62% to 92%. Base Flux Fill without our LoRA experts sat between the two on everything, at 22.1 FID and 0.58 SSIM on global garments, which told us how much of the gain came from the base model choice and how much came from the work we'd added on top.

The abstract claim we were proudest of was the success rate on complex global garments: 85% for our pipeline against 15% for the baselines. That gap is the entire company thesis in one line.

# eval/report.py
"""Tabulate the Flux-VTON+ evaluation run.
 
The eval set is 500 images, split evenly between Western Casual and
Global Traditional. Every number here comes out of a scored run; nothing
is entered by hand except the method labels.
"""
from dataclasses import dataclass
 
 
@dataclass
class MethodResult:
    name: str
    fid: float            # lower is better
    ssim_western: float   # higher is better
    ssim_global: float    # higher is better
    occlusion_acc: float  # human raters, higher is better
 
 
RESULTS = [
    MethodResult("SDXL + ControlNet", 28.4, 0.82, 0.45, 0.62),
    MethodResult("Base Flux Fill", 22.1, 0.91, 0.58, 0.70),
    MethodResult("Flux-VTON+ (ours)", 18.5, 0.94, 0.85, 0.92),
]
 
 
def complexity_gap(result: MethodResult) -> float:
    """How much worse a method is on global attire than on Western attire.
 
    This is the number I actually cared about. A method can post a strong
    average and still be useless to an Indian apparel brand if the entire
    average is carried by t-shirts.
    """
    return result.ssim_western - result.ssim_global
 
 
def report(results: list[MethodResult]) -> None:
    for r in results:
        print(
            f"{r.name:<22} "
            f"FID {r.fid:>5.1f}  "
            f"SSIM-W {r.ssim_western:.2f}  "
            f"SSIM-G {r.ssim_global:.2f}  "
            f"Occ {r.occlusion_acc:.0%}  "
            f"gap {complexity_gap(r):.2f}"
        )
 
 
if __name__ == "__main__":
    report(RESULTS)
    best = min(RESULTS, key=complexity_gap)
    print(f"\nsmallest complexity gap: {best.name} ({complexity_gap(best):.2f})")

The complexity_gap function is the honest summary of the whole research effort. SDXL with ControlNet carries a gap of 0.37 between the two halves of the eval set. Ours is 0.09. Closing that was the research problem, and we closed most of it with two LoRA experts, one trained on draping physics across sarees, hanfus, kimonos and haute couture, the other on occlusion and depth, merged into the base model at 0.6 and 0.4 rather than swapped at runtime.

Against the commercial try-on products in our space, Alphabake and FashnAI, our comparison was qualitative: images side by side, no scores. Our own writeup notes that some of the competitor outputs we compared against were generated from lower quality garment and model inputs, which is exactly the caveat you should be suspicious of when a company writes it about a competitor. I believed our results were better and I still do. I also know what a fair comparison would have required, and we didn't run it.

What the benchmark could not tell us

A benchmark answers the question you encoded into it. Ours encoded: given a model image and a garment image, does the resulting image look right, particularly for garments the field had been ignoring. It answered that well.

It said nothing about whether a brand's creative director would approve the output on a Tuesday. Nothing about how many images a catalogue team needs before the tool changes their workflow instead of adding to it. Nothing about integration effort, procurement cycles, or whether the person championing us internally would still be in that role in six months. Nothing about what happens when a well-funded competitor ships something merely adequate with a Shopify plugin already built.

I knew all of this in the abstract. What I underestimated was how completely those factors would dominate. Model quality is not the binding constraint on our outcome. That sentence has taken me most of this year to be able to write plainly, because every instinct I have as a researcher pushes toward the belief that a better model wins, and in this market it simply isn't the variable under tension.

The variable under tension was money and time, and the two are the same variable.

The arithmetic of being outspent

ai.fashion has raised $3.6 million. alphabake has raised$500K. Those are the two figures I had in front of me while writing our technical documentation, and they were sobering in a way that a benchmark table is not.

Funding buys things that don't show up in FID. It buys a sales team that can sit through a six-month procurement cycle at a large apparel group without the company dying in month four. It buys the plugin work, the SDKs, the design partner hand-holding, the second and third attempt at positioning. It buys the ability to be wrong for a while.

We have signed deals with real names. Aditya Birla Fashion Group is not a logo you put on a slide speculatively, and multiple Mensa Brands labels are working with us. I'm not going to put numbers or volumes next to those, partly because I won't publish them and partly because the number that matters was never in a contract. It's the distance between when we could get revenue at scale and when we run out of room to operate. Larger players have entered the space and the runway is short, and those two facts arrived close enough together that arguing about which one is decisive doesn't help anyone.

I don't think we made one identifiable mistake that is ending the company. That would be an easier story to tell. What I think happened is that we spent our strongest asset, research speed, on the problem where we were most capable rather than the problem where we were most exposed, and by the time the exposure was obvious the option to fix it needed capital we didn't have.

The part that was worth doing regardless

Most open-source diffusion models and the try-on datasets built around them are heavily skewed toward Western casual wear. T-shirts, jeans, hoodies: garments with simple topology, close to a tube around the body. A saree is not that. It depends on pleats, on gravity, on the pallu falling over the shoulder in a specific way, and a model without that prior renders it as a flat texture painted onto skin. Standard models classify a kimono as a bathrobe, because that's the nearest thing in their training distribution.

That's a dataset bias problem wearing a computer vision costume. It affects a very large number of people who happen not to be well represented in the images scraped off the Western internet, and the fashion industry's answer for those people has been that the technology doesn't work well yet.

We showed it can be fixed cheaply. Not by retraining a foundation model, which costs an amount neither we nor anyone in our position has, but by training small cultural adapters on a few thousand curated images each and merging them into the backbone. Five thousand images of draping. Three thousand of hands on hips and jewellery over fabric. Rank 32, alpha 16, a learning rate of 1e-4 on cosine annealing, a script anyone can run. The result closed most of a 0.37 gap.

That finding doesn't belong to AuraX and it doesn't stop being true because we're stopping. The paper is written, the method is described, and the merge is four lines of arithmetic on top of a public model. If it gets picked up by someone with more runway than we had, I'll take that outcome. It's the best available version of a company not making it.

What holds up

Naming the failure mode was worth more than solving it. "The Complexity Gap" and "the Amputated Hand" gave us a way to talk to brands about what was broken, and a brand that recognises its own problem in your vocabulary listens differently. The names travelled further than the metrics did.

Splitting the eval set was the single best methodological decision we made. Had we reported one averaged SSIM across 500 images, we would have posted a fine number, believed our own progress, and shipped something Indian brands would have rejected on sight.

A benchmark win is evidence about a model, not evidence about a business, and I let myself blur those. When our numbers came back strong, I read it as a signal about our position in the market. It wasn't one. It was a signal about our position in a table.

I'd still choose the research problem. Given a year and the same choice, I would work on non-Western garment rendering again, because it was underserved, tractable, and nobody with more resources was doing it seriously. What I'd change is how much of my attention went to the model versus how much went to the years of oxygen a company needs in order to sell one.

Last week I put the saree case back through base Flux Fill, without our experts merged in, to check that I hadn't been generous to ourselves in the writeup. The pallu still comes out as a flat printed panel stuck to the shoulder, with no fold and no shadow where the fabric gathers at the waist. The gap we named is exactly where we found it. Our weights close it, and they're sitting on a drive in Hyderabad along with a 500-image evaluation set that took months to assemble, and I haven't decided yet what to do with either.