JanKanta/ltx-2.3-22b-ic-lora-lens_remover

LensRemover — LTX-2.3-22B IC-LoRA for Lens-Flare & Optical-Artifact Removal

LensRemover is a video-to-video IC-LoRA for LTX-2.3-22B that removes lens flares, veiling glare,lens dirt and another optical artifacts from footage while keeping the underlying scene intact. You feed it the flared clip as the conditioning reference, and it regenerates a clean version of the same shot.

It is a dual-encoding LoRA — a single model that works on both sRGB / Rec.709 and ARRI LogC3 inputs, selected at inference with a trigger word.

I used Slava Sexton - OCIO Management in the workflow to handle the colors. It’s the best I𝗢/𝗖𝗼𝗹𝗼𝗿 𝗺𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁 for Comfy out there.

𝗖𝗼𝗺𝗳𝘆𝗨𝗜-𝗢𝗖𝗜𝗢: https://github.com/SlavaSexton/ComfyUI-OCIO

🎬 Showcase

All clips below are on a holdout scene the model never saw in training, shown as INPUT (flare) | LensRemover (clean) side-by-side.

sRGB / Rec.709

ARRI LogC3

The flare is removed in both color encodings, and the composition, subject and motion are preserved — no identity drift.

🚀 How to Use It

This is a V2V IC-LoRA: the conditioning is a reference video (the flared clip), not an image.

1. Load base LTX-2.3-22B + this LoRA lens-eraiser-ltx23-ic-lora.safetensors (strength 1.0).
2. Set the reference video = your flared clip.
3. Prompt with the trigger word that matches your input's color encoding:
   • sRGB / Rec.709 input → start the prompt with rgb,
   • ARRI LogC3 input → start the prompt with logc3,
4. Add a short scene description after the trigger, e.g. rgb, a woman walking through an autumn forest.
5. Generate → you get the cleaned version of the shot.

Tip: For LogC3 / EXR pipelines, encode your footage to LogC3 first (matching the training curve), run with the logc3 trigger, then decode back. Because sRGB ↔ LogC3 is a reversible curve, one LoRA covers both worlds.

🧩 Workflow

A ready-to-run ComfyUI workflow is included: LensRemover_comfyui_workflow.json.

• Base model: ltx-2.3-22b-dev + this IC-LoRA (rank 32, α 32)
• Reference conditioning at full resolution (downscale_factor = 1)
• Recommended working resolution: 960×544, 97 frames, 24 fps (the training bucket)
• Text encoder: Gemma-3-12B

📐 Model Details

Base model	LTX-2.3-22B (dev)
Type	IC-LoRA, video-to-video
Rank / Alpha	32 / 32
Target modules	attn1/attn2 (q, k, v, out) + ff.net
Conditioning	reference video (flared → clean), reference strength 1.0
Resolution	960×544 × 97 frames @ 24 fps
Trigger words	rgb (sRGB/Rec.709), logc3 (ARRI LogC3)
File	lens-eraiser-ltx23-ic-lora.safetensors (~327 MB)

📊 Dataset & Training

• 100 paired clips = 50 unique scenes, each in two color encodings (sRGB/Rec.709 and ARRI LogC3). Each pair maps a flared clip → its clean target.
• Dual-encoding recipe: the LogC3 pairs reuse the caption of their sRGB twin, with a split trigger word baked into the caption (rgb, … / logc3, …) so one model learns to route by encoding.
• 98 pairs trained, 1 scene held out in both encodings for validation.
• 3000 steps on a single 80 GB GPU, bf16, gradient checkpointing.
• Captions generated with a multimodal model and lightly cleaned.

The model converges early — flare removal is already effective within a few hundred steps — and stays stable through 3000 with no over-fitting or drift.

⚠️ Limitations

• Latent Space: Encoded dates are not the same as the output in terms of details — they need to be polished in post-production, but a wide range of dates are there, and they are pretty close to the input.
• Flare variety: trained on a specific set of added optical effects; unfamiliar real-lens flares may generalize less well.
• Resolution: trained at 960×544 — for larger footage, downscale to a working resolution (or tile) and match the aspect.I tested it on FullHD/2K and it is quite stable in 1 step, but 4K is quite unstable.
• Generative model: this regenerates the shot guided by the reference; on high-value footage expect the possibility of subtle content changes. Full-resolution reference conditioning keeps fidelity high.

🙏 Attribution

Built on LTX-2.3-22B by Lightricks, trained with the official LTX-2 IC-LoRA training pipeline. Released under the LTX-2 Community License — please follow the base model's terms.

Feedback welcome — if you try it on your footage, let me know how it holds up.