UNFINISHED: RTP, RTCP, RTSP, SRTP, HLS, MPEG-DASH, RTMP, FML

Originally posted on 2024-01-25

So many protocols to move bits around. But it doesn’t need to be pure pain anymore. Don’t bother searching the web for a description of each of these. For general use you can just stick with the basics below.

RTP (real-time protocol)

Usually runs over UDP and carries the actual audio and video streams. It can run over TCP as well but it’s not as common since usually you’ll want to be able to drop payloads if the network conditions change.

RTCP (real-time control protocol)

Monitors the health of the RTP streams by reporting statistics about the stream with sender reports and receiver reports. Participants that send media via RTP send sender reports that include (among other things) the number of packets sent and the number of octets sent. Receiver reports should be sent by all participants periodically, even if they don’t send media via RTP, and they include (among other things) cumulative number of packets lost and interarrival jitter.

RTSP (real-time streaming protocol)

Looks sort of like HTTP and is used to set up a stream. You tell the server what stream you want with RTSP, then you can send start, stop, fast forward, rewind, etc. commands with RTSP. The server will deliver the actual video over RTP. You can embed RTP in RTSP for situations where the network/firewall is blocking you but this is only recommended if you’re using RTSP over TCP. If you’re using RTSP over TCP you should probably only use it for recorded streams, not live video.

HLS

MPEG-DASH

RTMP (real-time messaging protocol)

A proprietary, TCP-based streaming format. Don’t use it.