Adaptive Streaming on Player Side

I beleive this is how the player switches from one resolution to another on HLS, but I have not found any documentation describing this process on the internet to verify.
First of all, simply switching from, e.g. HighRes0008.ts to LowRes0008.ts, doesn’t work.
1. Most often, HighRes0008.ts desn’t start with a IDR, which make descoding this segment impossible.
2. Even it starts with IDR, they may not be lined up on the frame level.
How the player switch:
1. In preparation to switch to lower resolution, player traces backward on the low resolution play list until it reaches an IDR.
2. based on the PTS on the high resolution playlist, it decides the PTS on the low resolution playlist, which is being switched to. Assuming the destination frame number ID is frame# 481.
3. decodes the low resolution playlist, from IDR to frame# 481, and renders that frame when time comes.
.ts segment contains PES, which in turn contains ES. The PTS is in the PES header.
PCR is only at the .ts layer, which I don’t think is needed in this use case, as long as PTS for audio and video are based on same PCR.
The same layer structure applies to audio too: TS – PES – ES (e.g. hold AAC data)



Why: to simply router configuration, so that only one fixed port number, 554, needs to accept connection.
How: Server sends video in the response of PLAY command. For each channel the format is like this:
[ Dolloar sign $, one byte channel ID, two bytes to say length of data, stream data itself ]
Channel ID is defined in SETUP response for that channel, e.g. RTP/AVP/TCP;unicast;interleaved=0-1, where 0 is RTP and 1 is RTCP. The stream data is sent as binary.
Head up: If RTSP wants RTP to be TCP based, RTP must be embedded into PLAY response, and therefore no such thing as RTP TCP port.

Server Push

There are two kinds of server pushes, which I call the true and the fake.

The Fake: I see it for Chrome to play MJPEG. It’s more of a browser feature. The server keeps sending jpg pictures and browser play video by replacing it’s picture one after another. It doesn’t need HTTP/2. Keywords: multipart x-mixed-replace boundary

The True: client receives resources without requesting. HTTP/2 required.
for Nginx, it has to be 1.13.9 and up, which was released this year 2018.

Crossdomain Access

The problem: when a web page from site A access files on site B, browser complains such as crossdomain access denied.
What it is really complaining is that browser, or plugin, doesn’t want to load resource fro site B unless B gives explicit permission.
When web based video player and video content are on different servers, this complaint happens.
Tools used for testing: JW Player 7.8.4 premium version, “primary”:”flash” vs “primary”:”html5″
The fix depends on the player mode: HTML5 vs flash.
For Flash, error message is “Cannot load M3U8: crossdomain access denied (2048)” The fix is creating /crossdomain.xml with content like:
<site-control permitted-cross-domain-policies=”all”/>
<allow-access-from domain=”*” to-ports=”*”/>
For HTML5, error message is “Cannot load M3U8: Crossdomain access denied”. The fix is adding Access-Control-Allow-Origin in the HTML header, e.g by editing nginx.conf.