Joint audio-video generation has advanced significantly in visual quality and audiovisual synchronization, yet precise control over shot transitions and dialogue timing has remained elusive. This limitation hinders script-driven content creation, where timing errors can disrupt narrative coherence.
To address this, researchers at Hugging Face introduce Temporal Context Routing (TCR), a new module that maps structured script timing onto the shared temporal axis of video and audio generation. By routing each prompt's guidance to the corresponding positions in both modalities, TCR ensures that both video and audio follow the script's timeline, not just each other.
In evaluations on 200 test scripts, TCR reduced Shot Boundary Mean Absolute Error (MAE) by 96%, from 1.11 seconds to 0.042 seconds, and raised Dialogue Accuracy at 0.5 seconds from 28.3% to 84.1%. These improvements were achieved while maintaining visual quality and audiovisual synchronization comparable to baselines.
A user study further showed that participants preferred TCR across all five evaluated dimensions. The project page is expected to be released soon.