generative scene extrapolation - benchmark

DL3DV ctx18 - 20-clip extrapolation benchmark

Observe the first 18 frames of a real DL3DV walkthrough with known camera poses, then generate the continuation along the ground-truth trajectory (frames 18-80, 63 extrapolation frames) into partially-unseen regions. All methods run the identical protocol; metrics are scored on the extrapolated region only.

context = 18 framestargets = 63 (18-80) 20 clips - 832x48011 methods incl SEVA, LVSM & CameraAnything

Leaderboard - extrapolation slice, sorted by PSNR, best per column highlighted

Method	PSNR	SSIM	LPIPS	FID	TSED	MEt3R	n
SEVAStable Virtual Camera - generative NVS diffusion [NEW]	16.06	0.484	0.503	65.9	0.649	0.1130	20
LVSMLarge View Synthesis Model - regression transformer [NEW]	15.62	0.485	0.593	121.8	0.673	0.1108	20
FrameCrafterI2V-14B + LoRA warp-inpaint	14.95	0.409	0.463	43.1	0.516	0.1751	20
VACE-14B (ours)warp + inpaint baseline	14.28	0.422	0.540	81.4	0.509	0.1359	20
NVS-Solvertraining-free diffusion prior	14.15	0.419	0.621	78.4	0.368	0.0577	20
GEN3Cexplicit 3D cache + render	14.13	0.440	0.628	114.2	0.297	0.1209	20
FlexWorld3DGS + video diffusion	14.12	0.432	0.607	90.2	0.309	0.1145	20
Lyra-2AR + 3D memory	13.18	0.400	0.658	79.9	0.393	0.0991	20
trajcrafter_gtpose	12.34	0.366	0.644	78.3	0.416	0.1269	20
TrajCrafterdiffusion NVS - 3-DOF orbit	11.91	0.351	0.680	85.2	0.324	0.1428	20
CameraAnythingrefilming w/ arbitrary camera control (ECCV'26) [NEW-EXT]	11.35	0.348	0.655	77.3	0.424	0.1075	20
MCSDF (ours, prev ckpt)memory-cond. DF - pre-retrain, refreshing	11.35	0.330	0.681	132.3	0.440	0.2065	20

PSNR/SSIM/LPIPS over the extrapolated region; FID = generation realism; TSED = 2-view SfM inlier fraction; MEt3R = multi-view consistency. Raw PSNR is low because this is generation, not reconstruction. SEVA leads PSNR; FrameCrafter leads LPIPS & FID. MCSDF is our method at a pre-retrain checkpoint (a new-objective model is training; this row refreshes when it lands).

CameraAnything audit (2026-07-29). Its last-place PSNR with best-tier MEt3R (0.107) and worst rotation error (43.8°) is a task-domain result, not a broken adapter - we verified the pose chain at three levels: (1) code - their pipeline consumes our OpenCV c2w with no inverse/transpose, relativized to observed frame 0; (2) numerical - independently recomputed Plücker rays match their function to 2.2e-02 max abs (half-pixel grid offset; a convention flip would be O(1)); (3) behavioral - two mirrored synthetic yaw trajectories (±40°, no translation) produce cleanly mirrored camera motion in exactly the direction the matrices dictate, and their own 8 shipped examples all obey their trajectories. The failure mode is that CameraAnything is trained for refilming (new viewpoints on content that stays visible); once the target view leaves the observed frustum it has no geometric evidence and invents a plausible different room. Fairness caveat: translations are similarity-scaled to max‖t‖=2.0 (their training range); doubling to 4.0 on a held-out clip gained +0.45 dB and increased motion magnitude without fixing the route, so a full rescaled rerun would move this row to roughly 11.8 - not enough to change the ordering. Panel note: the per-clip comparison videos now show CameraAnything in place of Lyra-2 (Lyra-2's scores remain in the leaderboard above; the video grid holds 9 method panels + GT).

Per-clip comparison - method grid + camera trajectory

12aa6a12145314fb_c1

context 0-17generated 18-80

method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80, marker = playback position.