generative scene extrapolation - benchmark
DL3DV ctx18 - 20-clip extrapolation benchmark
Observe the first 18 frames of a real DL3DV walkthrough with known camera poses,
then generate the continuation along the ground-truth trajectory (frames 18-80, 63 extrapolation
frames) into partially-unseen regions. All methods run the identical protocol; metrics are scored on
the extrapolated region only.
context = 18 frames targets = 63 (18-80)
20 clips - 832x480 9 methods incl SEVA & LVSM
Leaderboard - extrapolation slice, sorted by PSNR, best per column highlighted
Method PSNR SSIM LPIPS FID TSED MEt3R n SEVA Stable Virtual Camera - generative NVS diffusion [NEW] 16.06 0.484 0.503 65.9 0.649 0.1130 20 LVSM Large View Synthesis Model - regression transformer [NEW] 15.62 0.485 0.593 121.8 0.673 0.1108 20 FrameCrafter I2V-14B + LoRA warp-inpaint 14.95 0.409 0.463 43.1 0.516 0.1751 20 VACE-14B (ours) warp + inpaint baseline 14.28 0.422 0.540 81.4 0.509 0.1359 20 NVS-Solver training-free diffusion prior 14.15 0.419 0.621 78.4 0.368 0.0577 20 GEN3C explicit 3D cache + render 14.13 0.440 0.628 114.2 0.297 0.1209 20 FlexWorld 3DGS + video diffusion 14.12 0.432 0.607 90.2 0.309 0.1145 20 Lyra-2 AR + 3D memory 13.18 0.400 0.658 79.9 0.393 0.0991 20 TrajCrafter diffusion NVS - 3-DOF orbit 11.91 0.351 0.680 85.2 0.324 0.1428 20 MCSDF (ours, prev ckpt) memory-cond. DF - pre-retrain, refreshing 11.35 0.330 0.681 132.3 0.440 0.2065 20
PSNR/SSIM/LPIPS over the extrapolated region; FID = generation realism; TSED = 2-view
SfM inlier fraction; MEt3R = multi-view consistency. Raw PSNR is low because this is generation ,
not reconstruction. SEVA leads PSNR; FrameCrafter leads LPIPS & FID. MCSDF
is our method at a pre-retrain checkpoint (a new-objective model is training; this row refreshes when it lands).
Per-clip comparison - method grid + camera trajectory
12aa6a12145314fb_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 1c2554841888ab34_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 1c2554841888ab34_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 23cf048933459b1f_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 23cf048933459b1f_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 24b70d178abebbb0_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 24b70d178abebbb0_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 377430f9813379c5_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 3ab050170ad955b3_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 3ab050170ad955b3_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 3fa142f449c51c3e_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 3fa142f449c51c3e_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 408f0740fbf4395c_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 408f0740fbf4395c_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 42ac4b19a6ec45ee_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 42ac4b19a6ec45ee_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 4f0eb45686bc4a10_c0
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 4f0eb45686bc4a10_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 53b39a7ec355b8f3_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame 552646ce086e79d5_c1
context 0-17 generated 18-80
method comparison - GT + 9 methods (SEVA, LVSM, FrameCrafter, VACE-14B, GEN3C, NVS-Solver, FlexWorld, Lyra-2, TrajCrafter); context 18 frames, extrapolate 18-80. Bar: green = context 0-17 / orange = generated 18-80 , marker = playback position. camera trajectory - observed 0-17 vs target 18-80, colored by frame Scene-Extrapolation benchmark - ctx18. Companion dataset:
scene-extrapolation-viz .
Videos loop on play; trajectories colored by frame index (observed 0-17 vs target 18-80).
