Flux-VTON+: Hybrid Flux Inpainting and Multi-LoRA Expert Fusion

Virtual Try-On (VTON) has long suffered from a 'Complexity Gap' when applied to non-Western or structurally complex garments like Sarees and Kimonos. I designed the Flux-VTON+ framework to address this gap by synergizing the Flux Fill diffusion architecture with Flux Redux structural guidance. This comprehensive pipeline solves persistent challenges in high-fidelity garment transfer via a Multi-LoRA Expert Fusion strategy that injects specialized knowledge of draping physics and complex occlusion handling directly into the diffusion process.

I architected the pipeline to utilize Segment Anything Model 2 (SAM2) for precision masking and Flux Redux for structural style injection. A core innovation was my Multi-LoRA Expert Fusion technique, which mathematically merges a Draping Physics LoRA and an Occlusion & Depth LoRA into the base UNet. To overcome memory limitations while maintaining fidelity on global garments, I also developed a dynamic Context Window mechanism, ensuring high-resolution processing without exceeding typical VRAM constraints.

In rigorous technical evaluations against baseline SDXL and base Flux models, Flux-VTON+ achieved a state-of-the-art FID of 18.5 and an SSIM of 0.85 on complex Global garments (up from 0.45 for SDXL). The system proved highly successful at handling complex draping physics (e.g., Saree pleats) and occlusions (e.g., hands crossing the garment), dramatically outperforming industry competitors like Alphabake and FashnAI in VTON benchmarks with an 85% overall success rate.