R2SGen: Can Constraint Solving Improve Real-to-Sim Driving Scenarios?
The demand for synthetic training data is hindered by the sim-to-real gap, as current data-driven and LLM-based generators often produce physically implausible scenarios. To address this, we propose R2SGEN, a real-to-sim generation approach.
·
28:36
|
Can constraint solving turn a recorded traffic scene into useful simulation data without inheriting an LLM's geometric mistakes? This episode clarifies what R2SGen's guarantees mean, where its search improvements matter, and what remains unproven about sim-to-real transfer. We examine three questions: why merging equivalent search histories can matter more than choosing a different search order; whether lower program cost captures useful scenario abstraction; and how uncertainty in upstream perception changes the meaning of a verified output. The discussion focuses on static autonomous-driving scenario programs built from structured real-world observations, rather than a physical robot deployment. We connect the work to Scenic, ScenicNL, ChatScene, earlier scenario-querying research, and TrafficGen to distinguish reconstruction, probabilistic authoring, and learned traffic generation. The central limitation is the gap between consistency with supplied geometry and evidence of useful downstream transfer. We also outline concrete reproduction checks and an evaluation plan covering simulator validity, sampling yield, targeted failure preservation, and held-out performance. If your team is building synthetic-data or regression-testing infrastructure, what would you need to measure before trusting a mathematically consistent scene generator?
