Streaming Protocols Deep Dive: 14 Technologies, Their Limits, and Selection Guide

🔊

These 14 streaming technologies don’t operate in the same layer. A complete live streaming pipeline typically follows:

Camera/Encoder → Ingest (RTMP / SRT / RIST / WebRTC-WHIP / WebTransport) → Transcode → Delivery (HLS / LL-HLS / DASH / LL-DASH / HESP / HTTP-FLV) → Viewers

Start by identifying which segment of the pipeline you’re working on, not by asking “which is better.”

mermaid
flowchart TD
    A["Camera/Encoder"] --> B["Ingest Layer<br/>RTMP · SRT · RIST<br/>WebRTC · WebTransport"]
    B --> C["Transcode/Packaging<br/>CMAF · ABR · Multi-bitrate"]
    C --> D["Delivery Layer<br/>HLS · LL-HLS · DASH<br/>HTTP-FLV · WebRTC · HESP"]
    D --> E["Viewer End"]
    style A fill:#2196F3,color:#fff
    style B fill:#FF9800,color:#fff
    style C fill:#9C27B0,color:#fff
    style D fill:#4CAF50,color:#fff
    style E fill:#2196F3,color:#fff

The diagram above shows a five-segment structure for a standard live streaming pipeline. The ingest layer moves signals from cameras to the platform, while the delivery layer distributes processed streams to viewers. RTSP (camera control) and NDI (LAN production) operate outside the main pipeline, each working independently.

Core tradeoff: lower latency conflicts with higher scale and better compatibility—you must balance for your scenario.

Quick Selection: How to Choose

Your NeedSelection
Real-time bidirectional interaction in browsers (meetings / co-hosting / cloud gaming)WebRTC (SFU), or WebTransport + WebCodecs custom pipeline
PC / Android low-latency live streaming (surveillance / e-commerce)HTTP-FLV (flv.js), 1–3s
Full-platform large-scale low-latency deliveryLL-HLS / LL-DASH, or go aggressive with HESP (sub-second)
First hop for broadcaster push (ingest)RTMP (general-purpose) or SRT / RIST (weak network / long distance)
Surveillance / camera controlRTSP (with ONVIF); legacy frame-level viewing uses MJPEG; studio production uses NDI
Frame-by-frame processing in browsers (editing / inference)WebCodecs for encoding/decoding, connect to transport layer yourself
Production-grade large-scale live streamingTypically dual-path: WebRTC for co-hosting + HLS/FLV for distribution

Comparison Matrix

Delivery / Playback / Real-Time Interaction Layer (9 Technologies)

Latency refers to typical “end-to-end (glass-to-glass)” live streaming latency. WebCodecs is a client-side encoding/decoding building block that adds almost no latency itself—actual latency depends on the paired transport layer.

This table has many columns; scroll horizontally on mobile devices.

DimensionWebCodecsWebRTCWebTransportHTTP-FLVMJPEGHLSLL-HLSMPEG-DASHHESP
RoleEncoding/decoding building blockReal-time transportLow-latency transportFLV over HTTPContinuous JPEGAdaptive segmentationHLS low-latencyOpen adaptiveSub-second adaptive
Typical LatencyDepends on transport100–500ms~65ms contribution1–3sFrame-level <200ms15–30s2–5s10–20s~0.5–1s
Scale/ConcurrencyUnlimited (pure client)Requires SFU clusterRequires self-built serviceGood (HTTP)Poor (bandwidth explodes)Excellent (CDN)Good (CDN)Excellent (CDN)Good (CDN)
Browser CompatibilityChrome/FF130+/Safari16.4+Native on all platforms2026 BaselineRequires flv.js (iOS none)Almost universal supportSafari native + hls.jshls.js1.1+/iOS14+Requires dash.js (iOS none)THEO/Dolby player
Adaptive BitrateImplement yourselfSimulcast/SVCImplement yourselfRequires extra devNoneNativeNativeNativeInstant switching
Bandwidth EfficiencyHigh (hardware decode)Medium (multiple streams)HighMedium (fixed bitrate)Extremely low (≈10×)HighHighHighHigh (large GOP)
Implementation ComplexityHighHighHighMediumExtremely lowLowMedium-highMediumMedium-high
CDN CacheableNoNo (direct connection)PartialNoYesYesYesYes
Typical ScenariosEditing / frame-by-frameMeetings / co-hostingBrowser ingestLow-latency liveSurveillance / legacyVOD / large-scaleLow-latency large-scaleOpen standard deliverySub-second large-scale

Ingest / Contribution / Professional Network Layer (5 Technologies)

This layer solves “how to get signals from cameras/encoders to the platform and transcoders”—typically not for direct playback by end viewers.

DimensionRTMPSRTRISTRTSPNDI
RoleIngest/push streamingPublic network contributionPublic network contributionSurveillance controlLAN production
UnderlyingTCPUDP+ARQUDP+ARQRTP (commands+RTP)Ethernet/IP
Typical Latency2–5sSub-second (<2s)Sub-second ~1s0.1–0.5s (LAN)~16ms (1 frame)
Packet Loss ResilienceTCP retransmissionARQ strongARQ+multi-pathDepends on networkLAN stable
EncryptionRequires RTMPSAES-128/256DTLS+AESNone by defaultOptional
Ecosystem/CompatibilityEncoder全覆盖GrowingRelatively newCameras widespread600+ vendors
Public NetworkYesYesYesNo (LAN)No (LAN)
Typical ScenariosLive streaming first hopLong distance / weak networkEnterprise multi-vendorSurveillance / PTZStudio routing

End-to-End Latency Tiers

Audience-Facing Delivery / Playback Latency

TechnologyLatency
HESP~0.5–1s
WebRTC (SFU)~0.3s
WebTransport + WebCodecs~0.3–0.5s
MJPEGFrame-level ~0.2s
HTTP-FLV~1.5s
LL-DASH~2–3s
LL-HLS~3s
MPEG-DASH~15s
HLS (classic)~20s

Ingest / Contribution / LAN Latency (First Hop)

TechnologyLatency
NDI (LAN)~0.016s
RTSP (LAN)~0.2s
SRT~0.5s
RIST~0.6s
WebTransport (contribution)~65ms
RTMP (ingest)~3s

NDI / RTSP are limited to LAN only and cannot serve as public network delivery. RTMP’s “latency” refers to ingest latency—reaching viewers still requires HLS/DASH secondary packaging.

Deep Dive by Technology

WebCodecs

  • What it is: W3C standard API that directly exposes browser’s built-in audio/video encoders/decoders (VideoEncoder / VideoDecoder / AudioEncoder / AudioDecoder / ImageDecoder) to JS. It doesn’t handle demuxing/muxing—you need mp4box.js, webm-muxer to handle containers. Enables web pages to process media frame-by-frame (VideoFrame / EncodedVideoChunk), instead of treating MediaStream as a black box.
  • Core mechanism: Async, off-main-thread, can be placed in Web Worker; connects to media tracks via Insertable Streams (MediaStreamTrackProcessor / MediaStreamTrackGenerator); can freely combine with WebTransport and WebRTC.
  • Advantages: Directly invokes system hardware encoding/decoding—720p decoding is about 10–20× faster than FFmpeg.wasm (specific multiplier varies by load and codec), with lower memory footprint; frame-by-frame control suits video editing, frame-by-frame processing, ML inference, filters, frame extraction; reached Baseline in 2026 (Chrome/Edge 94+, FF 130+, Safari 16.4+).
  • Limitations: Not a transport protocol—must build transport/packaging layer yourself; codec coverage depends on OS; encoding parameters far fewer than FFmpeg; Insertable Streams fragmented (Firefox lacks MediaStreamTrackProcessor, Safari 18+ only added).
  • Root cause: Designed to handle only the encoding/decoding layer, leaving transport/packaging/signaling to developers—flexibility means no out-of-the-box live streaming. Compatibility fragmentation stems from spec evolving by interface rather than by browser.
  • Best for: In-browser video editing, real-time filters, frame-by-frame processing, ML inference, custom streaming pipelines.
  • Not for: Simple needs to just pull a stream and play directly (use HLS/FLV instead).

WebRTC

  • What it is: Browser-native real-time A/V communication standard based on SRTP/DTLS, over UDP, with built-in congestion control. Three topologies: Mesh (P2P), SFU (Selective Forwarding Unit), MCU (Multipoint Control Unit). WHIP (RFC 9725) is the mature path for browser contribution signaling.
  • Core mechanism: End-to-end direct connection (or forwarded via SFU), bidirectional, stateful; uses Simulcast/SVC for multi-bitrate tiers.
  • Advantages: Native support, zero plugins, end-to-end <300ms; mandatory encryption; bidirectional + DataChannel; SFU is the best balance point for 5–100+ participants.
  • Limitations: Cannot be cached by CDNs, scaling requires piling up servers; signaling is O(N²), Mesh collapses beyond 4 participants; about 40–60% of enterprise networks are forced through TURN relay (monthly bandwidth bill for ten-thousand streams can reach tens of thousands of dollars); SFU hits physical CPU ceilings.
  • Root cause: Built for peer-to-peer real-time, not designed for “one-to-millions broadcast.” Latency advantage comes from “no buffering, uncachable, per-connection independence”—the same feature becomes a dead end in large-scale distribution without edge caching, pressure all falls back on origin/SFU.
  • Best for: Video meetings, co-hosting, cloud gaming, online education.
  • Not for: Ten-thousand viewer unidirectional live streaming (use HLS/FLV for dual-path).

WebTransport

  • What it is: W3C API, modern transport layer based on HTTP/3 (QUIC/UDP). Provides two channel types: “reliable streams” and “unreliable datagrams,” multiplexed, no head-of-line blocking. It’s transport not media—video must connect to WebCodecs or MoQ (Media over QUIC) yourself.
  • Core mechanism: QUIC connection migration (Wi-Fi→cellular without breaking), 0-RTT connection setup; WHIP-over-WebTransport is the browser contribution signaling path; MoQ builds CDN-style fan-out on top (still draft). Safari 26.4+ is Baseline across all platforms as of March 2026.
  • Advantages: Sub-second, low overhead, connection migration without stream interruption; no head-of-line blocking; simpler than WebRTC (no ICE/STUN/TURN); Baseline on all platforms.
  • Limitations: No built-in media stack—must pair with WebCodecs / MoQ; directly fails when UDP/443 is filtered (no TCP fallback); pure client→server, no NAT traversal; MoQ still draft, large-scale fan-out not mature.
  • Root cause: Only moves bytes—video encoding/decoding must connect yourself. QUIC brings low latency and migration, but UDP-only means enterprise network filtering is a hard ceiling (WebRTC can at least fall back to TURN-over-TCP). Large-scale distribution must wait for MoQ to mature.
  • Best for: Browser-first custom ingest, self-managed servers, sub-second contribution goals.
  • Not for: Needs CDN large-scale fan-out, networks that strictly filter UDP.

HTTP-FLV

  • What it is: FLV container delivered via HTTP chunked transfer, browsers use flv.js (MSE-based) to real-time demux into fMP4. Low-latency alternative to RTMP + Flash-free.
  • Core mechanism: Server maintains long connection streaming FLV tags; latency comes from GOP length, server I-frame cache, and buffers on both ends.
  • Advantages: Low latency (1–3s, optimized ~1s), fast first screen; simple implementation; good PC/Android support; mature domestic ecosystem.
  • Limitations: iOS Safari completely unsupported (no MSE), must fall back to HLS; fixed bitrate, no native ABR; no persistent caching; must use H.264 + AAC/MP3.
  • Root cause: iOS dead end comes from Apple never implementing MSE in mobile Safari, and flv.js is entirely built on MSE—platform strategy not technical flaw. Fixed bitrate because FLV single-stream packaging, no multi-bitrate index. Essentially “use long connection to bypass segmentation” trading for low latency, also losing HLS’s caching and adaptability.
  • Best for: PC/Android low-latency live streaming, surveillance monitoring, e-commerce interaction.
  • Not for: iOS H5, needs full-platform consistency, large-scale scenarios requiring ABR.

MJPEG

  • What it is: Video treated as a sequence of independent JPEG images, pushed frame-by-frame via HTTP multipart/x-mixed-replace. Each frame is a complete image with no inter-frame compression.
  • Core mechanism: Frame-by-frame independent encoding/decoding, browsers natively support this MIME, no decoding library needed.
  • Advantages: Extremely simple, almost any device can directly view; frame-by-frame independent, error recovery easy; extremely low decoding CPU; extremely low frame-level latency.
  • Limitations: Bandwidth explosion, about 10× H.264 at same quality; almost no audio; extremely high storage cost; largely replaced in consumer space.
  • Root cause: Uses only intra-frame compression, abandons inter-frame temporal redundancy, trading simplicity and low latency for 10× bandwidth penalty. Survives only in narrow scenarios where “compatibility > efficiency” and “frame independence > cost” (legacy cameras, medical imaging, embedded surveillance).
  • Best for: IP cameras / old surveillance, medical / scientific imaging, embedded devices needing frame-level precise access.
  • Not for: High resolution, high frame rate, bandwidth-sensitive or long-term storage modern applications.

HLS

  • What it is: Apple’s 2009 HTTP Live Streaming. Server slices .ts (or fMP4/CMAF) segments, paired with .m3u8 index; Safari native, others rely on hls.js.
  • Core mechanism: Segmentation + index + player buffer. Latency = accumulation of encoding → wait for complete segment → CDN → player buffer.
  • Advantages: Full platform compatibility; native ABR; fits CDN, can scale to millions of concurrent users; mature DRM/time-shift/recording ecosystem.
  • Limitations: High latency (15–30s); not suited for real-time interaction; unidirectional broadcast; relatively complex deployment.
  • Root cause: Puts reliability and scalability before latency—must wait for entire segment (6–10s) to finish encoding + pre-buffer multiple segments, eating 20s+. Segmentation design enables CDN caching, ABR, cheap scaling to millions of viewers—cost is latency. Built for TV-grade reliable distribution, not real-time interaction.
  • Best for: VOD, large-scale live streaming, event distribution not sensitive to latency, full-platform scenarios requiring ABR.
  • Not for: Co-hosting, interaction, any <5s latency scenario.

LL-HLS

  • What it is: Apple’s 2020 low-latency HLS extension, preserving all advantages while pushing latency to 2–5s. Backward compatible.
  • Core mechanism (key pillars): ① Partial Segments (EXT-X-PART, cut segments into 200–400ms CMAF chunks, produce and send simultaneously); ② Blocking Playlist Reload (player pre-reserves next playlist version with _HLS_msn); ③ Preload Hints (EXT-X-PRELOAD-HINT,提前告知下一片 URL); ④ Playlist Delta Updates (CAN-SKIP-UNTIL, incremental updates) and Rendition Reports (EXT-X-RENDITION-REPORT, cross-variant reports). First three directly reduce latency, last two optimize ABR switching and multi-bitrate coordination.
  • Advantages: 2–5s latency still via standard HTTP/CDN; preserves ABR, caching, DRM; backward compatible; 2026: hls.js 1.1+, Shaka 4.0+, iOS 14+ native support.
  • Limitations: Full pipeline must support (packager, origin, each CDN POP, player); if CDN treats partials as independent cache items, gains are lost; still hard to stably achieve <1s; unidirectional broadcast.
  • Root cause: Didn’t颠覆 HLS’s HTTP model, just split “wait for full segment” into “produce and send simultaneously,” preserving CDN dividends also inheriting HTTP distribution floor—each chunk still makes one request to edge node, physically hard to reach sub-second. Real pit is in ops: partial + chunked + request merging must span entire pipeline, any misconfigured node falls back to classic HLS.
  • Best for: Large-scale low-latency live streaming, already using HLS and want to reduce latency without overthrowing architecture.
  • Not for: Sub-second interaction, conservative environments where CDN doesn’t support partial segments.

MPEG-DASH / LL-DASH

  • What it is: ISO/IEC 23009-1 international standard (MPEG alliance), same principle as HLS but open. Uses .mpd (XML) + .m4s (fragmented MP4), codec-agnostic. LL-DASH uses CMAF chunked to push latency to 2–3s, with QUIC can reach sub-2s.
  • Core mechanism: Segmentation + MPD + client ABR (dash.js / Shaka). After sharing CMAF with HLS, can “package once, distribute via dual protocols.”
  • Advantages: Open standard no lock-in; codec freedom; native ABR, CDN-friendly; sharing CMAF with HLS improves storage/cache efficiency.
  • Limitations: iOS/Safari doesn’t natively support (needs dash.js/Shaka); default segmentation 2–4s still too long; MPD/multi-DRM config slightly complex; encryption mode (CENC) differs from HLS (CBCS), often导致 “package once” falling back to “package twice.”
  • Root cause: Flexibility is double-edged sword—openness buys ecosystem fragmentation, Apple chose not to natively support to promote HLS. CMAF lets both share fragments, “choose one” becomes “need both”; but encryption mode difference remains real friction point. Suitable for non-Apple-first, needs codec freedom, multi-DRM scenarios.
  • Best for: Non-Apple-first, multi-DRM, needs codec freedom, Android/Chrome large-scale delivery.
  • Not for: Pure iOS native playback (unless using dash.js).

HESP

  • What it is: HTTP adaptive streaming proposed by THEO (now Dolby OptiView), IETF draft (draft-theo-hesp), promoted by HESP Alliance. Uses two streams to achieve sub-second: Init stream (each packet independent samples, can start from any packet) + Continuation stream (HTTP Range to pull subsequent).
  • Core mechanism: Init stream lets decoder initialize from any packet at any time, bypassing GOP boundary limitations; therefore can use large GOP (10–12s) maintaining compression efficiency without affecting startup/switching. ABR can switch at any moment, zapping <100ms. Only needs CDN support for HTTP CTE + Range.
  • Advantages: Sub-second latency (~0.5–1s) + fast zapping (<100ms); large GOP saves ~20% bandwidth vs LL-HLS; ABR switches anytime without stuttering on weak networks; CDN compatible.
  • Limitations: Ecosystem relatively new, requires dedicated player; needs dedicated packager + dual-stream storage; standard still draft; narrower deployment than LL-HLS.
  • Root cause: Uses Init stream to bypass GOP boundary limitations, large GOP (compression efficient) and low latency/instant ABR can coexist, while LL-HLS forced small GOP for low latency (sacrificing compression). Cost is dual streams and dedicated components, ecosystem not yet widespread.
  • Best for: Sub-second large-scale delivery, interactive live streaming, OTT needing fast zapping.
  • Not for: Conservative environments not wanting to introduce dedicated packager/player.

RTMP

  • What it is: Adobe (originally Macromedia) 2002 TCP protocol, port 1935. Flash-era artifact, now only lives as ingest standard: encoders push to platform, platform then repackages into HLS/DASH/WebRTC. RTMPS=TLS, RTMPT=HTTP tunneling.
  • Core mechanism: Long connection TCP, AMF handshake, chunk streaming; simple URL + stream key push streaming. Enhanced RTMP (v2) starts supporting HEVC/AV1/VP9/HDR.
  • Advantages: Encoders universally support (OBS/vMix/hardware/mobile); platforms universally accept (Twitch/YouTube/FB/TikTok); firewall-friendly can fall back to 80/443; extremely simple to set up.
  • Limitations: Browsers cannot directly play (Flash dead); TCP retransmission raises latency on packet loss; no native encryption (needs RTMPS); single video single audio, no Simulcast; codecs somewhat outdated.
  • Root cause: Survives only because “ingest-side ecosystem stickiness”—industry-wide encoders default to RTMP output, replacement needs coordinating millions of devices; playback side already migrated to HLS/DASH/WebRTC. Essentially ingest standard not delivery standard. TCP nature on public internet relies on retransmission for reliability on packet loss, also paying latency cost. “Outdated but irreplaceable” is result of asynchronous evolution between ingest and playback sides.
  • Best for: Live streaming ingest first hop, legacy encoders, strict firewall environments.
  • Not for: Browser playback, sub-second interaction (use SRT/WHIP instead).

SRT

  • What it is: Haivision open-source UDP reliable transport, ARQ + forward buffering for packet loss recovery, AES-128/192/256 native encryption, codec-agnostic. Designed for low-latency reliable transport over unpredictable public networks.
  • Core mechanism: UDP foundation + ARQ selective retransmission + adjustable buffering; can be consumed by CDN/OTT.
  • Advantages: Stable under public network packet loss, sub-second latency; native AES encryption; codec-agnostic; CDN/OTT friendly, adoption rising.
  • Limitations: Ecosystem newer than RTMP, some old devices don’t support; configuration more complex than RTMP (buffer/latency tuning); UDP/443 may be filtered; not a playback protocol, needs repackaging.
  • Root cause: Self-built reliability (ARQ) on UDP rather than relying on TCP, both fast and packet-loss resilient, core to replacing RTMP for long-distance contribution. Cost is needs self-building/configuration and relies on UDP reachability. Solves public network long-distance reliable contribution, not end-to-end playback, still needs to cooperate with backend delivery.
  • Best for: Broadcast-grade long-distance, overseas live streaming, weak network/mobile backhaul contribution.
  • Not for: Browser direct playback, pure legacy environments.

RIST

  • What it is: Open standard led by VideoLAN (VSF TR-06 series technical recommendations), UDP reliable transport. Emphasizes cross-vendor interoperability, strong congestion control (BCA) and multi-path, positioned for enterprise/broadcast-grade contribution.
  • Core mechanism: DTLS + AES; ARQ + selective retransmission; multi-path intelligent scheduling; can carry audio/video/subtitles/timecode.
  • Advantages: Open and free, cross-vendor interoperability; strong congestion control, multi-path; enterprise private network/multi-location backhaul friendly; native DTLS+AES.
  • Limitations: Ecosystem even more immature, tools/docs less than RTMP; high technical barrier (retransmission/multi-path/serialization); high onboarding cost for small/medium teams; not a playback protocol.
  • Root cause: Same solution as SRT for public network reliable UDP, but emphasizes open standard + multi-path + strong congestion control, positioned for enterprise/broadcast interoperability. Cost is young ecosystem, steep learning curve—complex design is exactly for high reliability, also raises deployment threshold. Suitable for multi-vendor, enterprise private networks, cannot bind to single vendor needs.
  • Best for: Multi-vendor interoperability, enterprise private networks, multi-path long-distance contribution.
  • Not for: Small/medium teams quick onboarding, legacy devices.

RTSP

  • What it is: RFC 2326 application-layer “network remote control” protocol (RTSP 1.0; 2.0 is RFC 7826, not yet widely deployed). Doesn’t send media itself, only sends control commands (OPTIONS/DESCRIBE/SETUP/PLAY/PAUSE/TEARDOWN), real media goes over RTP. Often paired with ONVIF for device discovery and PTZ control.
  • Core mechanism: Commands and media separated; typically runs over LAN UDP, sub-second latency. IP cameras widely support, browsers don’t natively support.
  • Advantages: Sub-second latency, real-time PTZ control; IP cameras/NVRs widely support; codec freedom (H.264/H.265); precise control.
  • Limitations: Browsers don’t natively support, need dedicated player; no encryption by default; firewall may block ports; unstable on public network, no device discovery (relies on ONVIF).
  • Root cause: Is media remote control protocol not transport protocol—only sends commands, real media goes over RTP; designed for closed LAN, public network/encryption not goals. Browsers don’t directly support because never designed for web playback. Going public/cloud must go through media server converted to RTMP/HLS.
  • Best for: Surveillance monitoring, PTZ control, LAN real-time viewing.
  • Not for: Public network live streaming, browser direct playback.

NDI

  • What it is: NewTek (now Vizrt) video-over-IP protocol, runs studio signals over standard Ethernet. mDNS auto-discovery plug-and-play. Full quality about 130–150 Mbps/1080p60, compressed version NDI|HX about 10–20 Mbps (H.264/H.265).
  • Core mechanism: Standard Ethernet + auto-discovery; uses existing network to replace SDI/HDMI cabling. 600+ vendors, 2000+ products support.
  • Advantages: Sub-frame latency (~16ms); plug-and-play zero config; uses existing network replacing dedicated video matrix saves cabling; cross-brand interoperability.
  • Limitations: LAN only, not public network delivery; high bandwidth (full quality needs gigabit/10-gigabit); depends on managed switches + QoS; proprietary ecosystem (though open SDK).
  • Root cause: Born for studio internal IP-ization, sacrificing public network reachability for extreme low latency and zero config; high bandwidth not a problem on LAN, not feasible on public network. Complementary not replacement for SDI/HDMI—cameras go SDI into encoder, encoder outputs NDI internal routing, finally still goes RTMP/SRT to cloud. Is production domain protocol, not distribution domain protocol.
  • Best for: Live production, studio routing, console video walls, education/enterprise PTZ.
  • Not for: Public network distribution, cross-Internet transmission.

Root Cause Overview: Why “Low Latency” and “Large Scale” Are Fundamentally Mutually Exclusive

  1. Latency comes from four-segment accumulation: Encoding + packaging/segmentation + CDN propagation + player buffer. To reduce latency each segment must be cut (small GOP, small segmentation, small buffer), but smaller buffers more fear jitter, reliability drops.
  2. Cacheable ⇄ Real-time mutually exclusive: HLS/FLV/DASH/LL-HLS go via HTTP can be cached by CDN edges, cheaply scaling to millions of viewers; WebRTC/WebTransport each stream unique and bidirectional/direct connection, uncachable, scaling only by piling servers.
  3. Compression efficiency ⇄ Simplicity mutually exclusive: MJPEG abandons inter-frame compression trading for extreme simplicity and low latency; H.264/HEVC use temporal redundancy trading for bandwidth. No free lunch that’s fast, efficient, and simple.
  4. Platform strategy determines compatibility: iOS no MSE → flv.js ineffective; Safari late support for WebCodecs/WebTransport; Apple promotes native HLS. Many “technical limitations” are vendor roadmap choices.
  5. Abstraction layer misalignment: WebCodecs is encoding/decoding building block, RTMP/SRT is ingest, NDI is LAN production, HLS/DASH is delivery. Comparing latency across different layer technologies is a false question—must first locate pipeline position.
  6. Full pipeline bottleneck: LL-HLS/HESP gains depend on weakest link (packager→origin→each CDN POP→player). Any node not supporting partial/dual streams, latency falls back to classic HLS.
  7. Reliability comes from redundancy: SRT/RIST self-build ARQ on UDP trading for reliability, TCP (RTMP) relies on retransmission trading for reliability—all pay latency. Public network unpredictable, reliability and low latency must trade off.
  8. GOP boundary constraints: Players need key frames to start playback, LL-HLS forced small GOP for low latency (sacrificing compression), HESP uses Init stream to bypass → large GOP can also achieve sub-second. This is fundamental difference between same-layer protocols.

Selection Guide

mermaid
flowchart TD
    A["Need bidirectional<br/>real-time interaction?"] -->|"Yes"| B["WebRTC (SFU)"]
    A -->|"No"| C["Viewer scale?"]
    C -->|"Under 10k<br/>PC/Android"| D["HTTP-FLV<br/>1-3s"]
    C -->|"Millions full-platform"| E["Latency floor?"]
    E -->|"≥10s"| F["HLS / DASH"]
    E -->|"2-5s"| G["LL-HLS / LL-DASH"]
    E -->|"<1s"| H["HESP"]
    style B fill:#4CAF50,color:#fff
    style D fill:#2196F3,color:#fff
    style F fill:#FF9800,color:#fff
    style G fill:#2196F3,color:#fff
    style H fill:#9C27B0,color:#fff

The diagram above covers mainstream delivery layer scenarios. Ingest layer selection is simpler:

  • Broadcaster push first hop: General-purpose/legacy → RTMP; weak network/long distance/overseas → SRT; enterprise multi-vendor/multi-path → RIST; browser contribution → WebRTC-WHIP or WebTransport.
  • Surveillance cameras / studio: Camera control → RTSP (with ONVIF); legacy frame-level → MJPEG; LAN production routing → NDI.
  • Frame-by-frame processing in browsers: WebCodecs for encoding/decoding, connect to packaging and transport yourself.
  • Can accept dual-path architecture: Production-grade large-scale live streaming often does WebRTC (interactive co-hosting) + HLS/FLV (large-scale distribution) dual-path, each taking strengths.

Reference Sources

  • Chrome for Developers — Video processing with WebCodecs
  • MDN / Can I Use — WebCodecs and WebTransport browser support (2026 Baseline)
  • W3C — WebTransport; IETF — QUIC (RFC 9000), HTTP/3 (RFC 9114)
  • IETF — WHIP (RFC 9725)
  • Ant Media — WebRTC Topology (Mesh/SFU/MCU), WebRTC Scalability 2025
  • Apple Developer — Enabling Low-Latency HLS
  • Bitmovin — Fundamentals of LL-DASH and LL-HLS
  • Streaming Media — LL-HLS and LL-DASH sub-3-second latency
  • IETF — HESP draft (draft-theo-hesp); Dolby OptiView — What is HESP
  • VSF — RIST TR-06 series technical recommendations (TR-06-1/2/3)
  • IETF — RTSP (RFC 2326 / RFC 7826)
  • Dacast — RTMP in 2026 (ingest standard), SRT comparison
  • LiveAPI — RTSP vs RTMP; ONVIF — RTSP and device discovery
  • Epiphan — NDI Streaming 2025
  • ForaSoft — RTMP glossary, WebTransport/WHIP entries