Inserting audio into video
AES/EBU digital audio interfaces

When inserting a digital audio stream into a digital video stream, the first difficulty that arises is the difference in structure between the two streams. While both appear to be a succession of fixed-rate bits, derived from the fixed-rate sampling of an analog signal, the underlying temporal structure is very different. The video signal is made up of lines, separated by line break intervals. After digitization, this structure is respected, and the digital stream alternates between a packet of bytes corresponding to a video line and a packet of empty bytes, corresponding to the line returns. It is into these empty packets that the audio signal is inserted.

A second difficulty arises from the relationship between video frames and audio samples. The ratio between the number of video lines per second and the number of audio samples per second is not a whole number....