text_both_base_0727Checkpoint epoch001-global_step15000, on 5 held-out val.jsonl clips.
Every clip has generated audio as well as video — unmute to judge the full result.
a video with {object};
the model must put the object and its sound back.a video without {object}; the model must take it out.--prepend-ref, stripped again before decoding);
no first frame forces it off (--no-prepend-ref).add
direction was found to overshoot at --cfg-scale 5.0 — the latents leave
the VAE's range and decode to a flat pink wash, monotonically in cfg and across seeds,
while remove survives it. This matrix is generated at cfg 1.0.with first frame is out-of-distribution for this checkpoint.
Training used ref_prepend_warmup_steps=5000 with
p_ref = max(0, 1 - step/5000), so from step 5,000 onward the anchor was
stripped from both target and source latents. At step 15,000 this checkpoint trained
anchor-free for 10,000 of its 15,000 steps — no first frame is the mode
that matches training.