Redirect4D-Bench: A Scalable Benchmark for Camera Redirection of Monocular Dynamic Videos with Pseudo-4D Ground Truth

Motivation

Camera redirection takes a monocular source video and replays the same dynamic event along a target camera trajectory.

High video-metric scores do not imply successful camera redirection. On the wolf comparison below, per-column CLIP/VBench winners (red) are spread across the baselines, yet none of those winning videos completes the task. Only FreeOrbit4D follows the requested trajectory while keeping the wolf intact, matching the human verdict.

Source video (wolf, yaw −80, scale 0.9)

4D reconstruction (drag/scroll/play; blue = target trajectory, red = source-video trajectory)

Method	Output	CLIP ↑			VBench ↑							Human Evaluation
Method	Output	T	F	V	SC	BG	TF	MS	AQ	IQ	OC	Human Evaluation
ReCamMaster		0.314	0.967	0.920	0.898	0.938	0.944	0.970	0.569	0.715	0.313	❌Does not follow trajectory
TrajectoryCrafter		0.284	0.948	0.802	0.733	0.903	0.933	0.964	0.440	0.741	0.265	❌Distorted geometry
GEN3C		0.288	0.951	0.864	0.708	0.919	0.948	0.982	0.546	0.660	0.293	❌Distorted geometry
FreeOrbit4D		0.304	0.954	0.856	0.825	0.920	0.932	0.977	0.531	0.565	0.293	✅Looks good!

CLIP-T/F/V: text, adjacent-frame, and source-video consistency. VBench: SC subject, BG background, TF temporal-flickering, MS motion-smoothness, AQ aesthetic, IQ imaging, OC overall consistency. Per-column best in green. The CLIP/VBench winners scatter across all four methods, while the human verdict points to the only one that completes the redirection.

Construction Pipeline

Construction is fully training-free, so the benchmark scales as more source clips are added. Source clips are curated from in-the-wild YouTube videos: category-level queries fetch candidates, which are split into shot-level clips and turned into object-centric crops with Grounded-SAM masks. Each clip's RGB + mask drives a reconstruction stack that recovers geometry-complete foreground 4D point clouds and a ViPE background. Target trajectories then render depth and rough masks; a frozen prompt feeds the Wan generator, and MaskRefine produces the pseudo-GT target mask used for evaluation.

Metric Protocol

Object fidelity uses SAM3 propagation to extract masks from each submitted RGB video and compares them to the dataset target pseudo-GT mask. Camera-pose accuracy reconstructs the camera path from each generated video with a pinned reconstruction stack and compares it to the requested target trajectory. CLIP, FID/FVD, and VBench are reported alongside.

Data Case

Each Redirect4D-Bench track ships with a source RGB clip, the reconstructed 4D point cloud, and per-trajectory pseudo-GT mask + depth. The interactive viewer overlays the foreground/background point cloud, the target trajectory (blue), and the original source-video trajectory (red).

Source RGB · Camel

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Deer

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #1

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #2

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Tiger

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Cow

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #3

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #4

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #5

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Cat

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #6

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Robot #7

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Source RGB · Deer #2

4D reconstruction (interactive)

Target pseudo-GT mask

Target rendered depth

Comparison

Four representative cases, each scored with the full CLIP + VBench stack alongside Redirect4D-Bench's Camera accuracy, Object fidelity, and Subject localization metrics. Per-column best in green. Click a case to switch.

Camel Bear Deer Robot Cow Pig

Camel · yaw 120 / pitch -20

TrajectoryCrafter

ReCamMaster

GEN3C

FreeOrbit4D

Method	Traditional video metrics										Human	Redirect4D-Bench metrics
	CLIP ↑			VBench ↑								Camera acc. ↓		Object fid. ↑		Subject loc. ↑
	T	F	V	SC	BG	TF	MS	AQ	IQ	OC		RotErr	TransErr	D	R	cIoU	R·cIoU
TrajectoryCrafter	0.311	0.949	0.846	0.796	0.907	0.933	0.968	0.498	0.748	0.223	❌	3.99	0.15	1.000	0.044	0.493	0.022
ReCamMaster	0.317	0.983	0.935	0.931	0.911	0.962	0.984	0.507	0.709	0.219	❌	27.31	0.73	1.000	0.756	0.095	0.071
GEN3C	0.301	0.949	0.849	0.814	0.939	0.958	0.984	0.498	0.685	0.215	❌	7.10	0.16	1.000	0.178	0.507	0.090
FreeOrbit4D	0.304	0.934	0.858	0.874	0.934	0.952	0.982	0.554	0.703	0.223	✅	0.79	0.03	1.000	1.000	0.919	0.919

Method	Traditional video metrics	Human	Redirect4D-Bench metrics
TrajectoryCrafter	0.323	0.955	0.864	0.773	0.925	0.896	0.956	0.429	0.652	0.310	❌	4.12	0.27	1.000	0.444	0.636	0.283
ReCamMaster	0.323	0.979	0.945	0.931	0.943	0.948	0.982	0.549	0.648	0.294	❌	31.93	0.84	1.000	1.000	0.402	0.402
GEN3C	0.342	0.945	0.853	0.779	0.932	0.911	0.964	0.471	0.571	0.305	❌	23.95	0.67	1.000	0.800	0.622	0.497
FreeOrbit4D	0.298	0.963	0.816	0.852	0.953	0.932	0.973	0.516	0.628	0.279	✅	3.53	0.16	1.000	1.000	0.981	0.981

Method	Traditional video metrics	Human	Redirect4D-Bench metrics
TrajectoryCrafter	0.311	0.944	0.789	0.778	0.869	0.875	0.940	0.418	0.697	0.253	❌	3.05	0.16	1.000	0.733	0.480	0.352
ReCamMaster	0.300	0.974	0.833	0.839	0.869	0.901	0.957	0.472	0.690	0.294	❌	32.94	0.90	1.000	1.000	0.072	0.072
GEN3C	0.310	0.939	0.830	0.807	0.899	0.896	0.959	0.430	0.552	0.280	❌	10.17	0.11	1.000	0.689	0.548	0.378
FreeOrbit4D	0.310	0.933	0.874	0.830	0.913	0.872	0.931	0.466	0.693	0.285	✅	4.20	0.09	1.000	1.000	0.886	0.886

Method	Traditional video metrics	Human	Redirect4D-Bench metrics
TrajectoryCrafter	0.300	0.962	0.794	0.793	0.887	0.924	0.974	0.394	0.691	0.266	❌	4.84	0.56	1.000	0.167	0.588	0.098
ReCamMaster	0.323	0.952	0.879	0.817	0.888	0.937	0.981	0.479	0.685	0.329	❌	19.47	1.03	1.000	1.000	0.528	0.528
GEN3C	0.319	0.952	0.858	0.806	0.928	0.927	0.983	0.516	0.575	0.332	❌	3.47	0.35	1.000	0.917	0.691	0.633
FreeOrbit4D	0.316	0.949	0.848	0.843	0.905	0.910	0.972	0.467	0.680	0.352	✅	3.28	0.34	1.000	1.000	0.922	0.922

Method	Traditional video metrics	Human	Redirect4D-Bench metrics
ReCamMaster	0.290	0.981	0.817	0.882	0.853	0.928	0.966	0.509	0.529	0.296	❌	21.49	0.62	1.000	1.000	0.051	0.051
TrajectoryCrafter	0.312	0.966	0.861	0.843	0.877	0.919	0.958	0.501	0.692	0.277	❌	4.46	0.42	1.000	0.733	0.649	0.476
GEN3C	0.289	0.957	0.817	0.818	0.868	0.918	0.964	0.480	0.524	0.243	❌	8.15	0.29	1.000	0.511	0.711	0.363
FreeOrbit4D	0.324	0.951	0.885	0.849	0.918	0.922	0.962	0.508	0.572	0.296	✅	3.24	0.05	1.000	1.000	0.942	0.942

Method	Traditional video metrics	Human	Redirect4D-Bench metrics
ReCamMaster	0.319	0.967	0.845	0.863	0.875	0.940	0.980	0.568	0.685	0.277	❌	21.67	0.59	1.000	1.000	0.086	0.086
TrajectoryCrafter	0.336	0.968	0.875	0.796	0.916	0.920	0.975	0.515	0.669	0.303	❌	3.86	0.59	1.000	0.778	0.728	0.566
GEN3C	0.325	0.965	0.883	0.841	0.927	0.925	0.983	0.536	0.601	0.302	❌	5.59	0.35	1.000	0.822	0.742	0.610
FreeOrbit4D	0.344	0.960	0.886	0.858	0.918	0.942	0.981	0.530	0.698	0.338	✅	2.81	0.34	1.000	1.000	0.934	0.934

CLIP-T/F/V: text, adjacent-frame, source-video consistency. VBench: SC subject, BG background, TF temporal-flickering, MS motion-smoothness, AQ aesthetic, IQ imaging, OC overall consistency. Camera acc.: RotErr (°), TransErr (m), lower is better. Object fid.: D detection rate, R recognition rate. Subject loc.: cIoU mean cMaskIoU on detected frames, R·cIoU recall-weighted across all frames.

BibTeX

If you find Redirect4D-Bench useful, please cite our work.

@misc{cao2026redirect4dbench,
  title  = {Redirect4D-Bench: A Scalable Benchmark for Camera Redirection of Monocular Dynamic Videos with Pseudo-4D Ground Truth},
  author = {Wei Cao and Hao Zhang and Jiapeng Tang and Yulun Wu and Yingying Li and Ning Yu and Shenlong Wang and Yaoyao Liu},
  year   = {2026},
}