Add declareInitSegment method to the SegmentBuffer abstraction

This commit adds the `declareInitSegment` and `freeInitSegment` methods
to the RxPlayer's SegmentBuffer abstractions (which is the part of the
code handling the operations on SourceBuffers, such as pushing media and
init segments).

The short term idea is to improve the handling of initialization
segments in the SegmentBuffer.
Until now, each pushed media segment lead to a check that the
initialization segment it relies on is the same than the last one
pushed. To be able to perform that check, a caller need to communicate
again the initialization segment's data each time a chunk is pushed to
the buffer.

This check can be performed efficiently in most cases because we first
check init segment's data equality by reference, which, by pure luck,
should be equal in most cases in the current code. In cases where it
isn't the same reference however, it can lead to a byte-per-byte check,
which should not be an issue in terms of performance in most cases, but
is still an ugly specificity which could be handled in a more optimal
and understandable way.

This commit now allows the definition of a `initSegmentUniqueId`, an
identifier for initialization segments, on SegmentBuffers.
Any pushed segments can then refer to its associated init segment by
indicating which `initSegmentUniqueId` it is linked to.

The SegmentBuffer will ensure behind the hood that the right
initialization segment is pushed before pushing media segments, like
before, excepted that this can now be done just by comparing this
`initSegmentUniqueId` - it also means that the caller is no more
required to keep in memory the data of the loaded initialization
segment, the `SegmentBuffer` is already doing that.
Previously, the initialization segment's data was kept by the
`RepresentationStream`, the abstraction choosing which segments
to load (which is part of the reasons why the reference mostly never
changed).

The declaration and "freeing" of init segment is done through a
`declareInitSegment`/`freeInitSegment` pair of methods on a
`SegmentBuffer`. This sadly means that memory freeing for the
initialization segment is now manual, whereas we just relied on garbage
collection when the initialization segment was directly used.

---

Though mostly, the long term benefit is to implement the hybrid-worker
mode that we plan to have in the future, where buffering is performed in
a WebWorker (thus improving concurrence with an application, with the
goal of preventing both UI stuttering due to heavy player tasks and
rebuffering due to heavy UI tasks).

In the currently-planned long term worker features we would have thus
the following modes:

  - full worker: where both the rebuffering logic and MSE API are called
    in a WebWorker, allowing to avoid UI and media playback blocking
    each other to some extent

    This however requires the
    [MSE-in-Worker](w3c/media-source#175)
    feature to be available in the browser AND it also implies a more
    complex API, notably some callbacks (`manifestLoader`,
    `segmentLoader` and `representationFilter`) which will have to be
    updated.

  - hybrid mode: The buffering logic is mainly performed in a WebWorker
    but MSE API are still in the main thread. This allows e.g. to not
    fight for CPU with the UI to know which segments to download and to
    avoid blocking the UI when the Manifest is being parsed.

    Though the UI blocking could still mean that a loaded segment is
    waiting to be pushed in that mode.

    Because here MSE APIs may have to be called through `postMessage`-style
    message passing, the previous logic of communicating each time the
    same initialization segment each time a segment was pushed, with no
    mean to just move that data (in JavaScript linguo, to "transfer" it)
    was considerably worst than before.
    Relying on a short identifier instead seems a better solution here.

  - normal mode: The current mode where everything stays in main thread.

However it should be noted that all of this long term objective is still
in an higly experimental phase, and the gains are only theoretical for
now.

Author: peaBerberian

src/core/segment_buffers/implementations/audio_video/audio_video_segment_buffer.ts

@@ -21,11 +21,8 @@ import {
 import config from "../../../../config";
 import log from "../../../../log";
 import { getLoggableSegmentId } from "../../../../manifest";
-import areArraysOfNumbersEqual from "../../../../utils/are_arrays_of_numbers_equal";
 import assertUnreachable from "../../../../utils/assert_unreachable";
-import { toUint8Array } from "../../../../utils/byte_parsing";
 import createCancellablePromise from "../../../../utils/create_cancellable_promise";
-import hashBuffer from "../../../../utils/hash_buffer";
 import noop from "../../../../utils/noop";
 import objectAssign from "../../../../utils/object_assign";
 import TaskCanceller, {
@@ -139,20 +136,26 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
   private _pendingTask : IAVSBPendingTask | null;
 
   /**
-   * Keep track of the of the latest init segment pushed in the linked
-   * SourceBuffer.
+   * Keep track of the unique identifier of the  of the latest init segment
+   * pushed to the linked SourceBuffer.
    *
-   * This allows to be sure the right initialization segment is pushed before
-   * any chunk is.
+   * Such identifiers are first declared through the `declareInitSegment`
+   * method and the corresponding initialization segment is then pushed through
+   * the `pushChunk` method.
+   *
+   * Keeping track of this allows to be sure the right initialization segment is
+   * pushed before any chunk is.
    *
    * `null` if no initialization segment have been pushed to the
    * `AudioVideoSegmentBuffer` yet.
    */
-  private _lastInitSegment : { /** The init segment itself. */
-                               data : Uint8Array;
-                               /** Hash of the initSegment for fast comparison */
-                               hash : number; } |
-                             null;
+  private _lastInitSegmentUniqueId : string | null;
+
+  /**
+   * Link unique identifiers for initialization segments (as communicated by
+   * `declareInitSegment`) to the corresponding initialization data.
+   */
+  private _initSegmentsMap : Map<string, BufferSource>;
 
   /**
    * @constructor
@@ -174,8 +177,9 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
     this._sourceBuffer = sourceBuffer;
     this._queue = [];
     this._pendingTask = null;
-    this._lastInitSegment = null;
+    this._lastInitSegmentUniqueId = null;
     this.codec = codec;
+    this._initSegmentsMap = new Map();
 
     const onError = this._onPendingTaskError.bind(this);
     const reCheck = this._flush.bind(this);
@@ -198,6 +202,20 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
     });
   }
 
+  public declareInitSegment(
+    uniqueId : string,
+    initSegmentData : unknown
+  ) : void {
+    assertDataIsBufferSource(initSegmentData);
+    this._initSegmentsMap.set(uniqueId, initSegmentData);
+  }
+
+  public freeInitSegment(
+    uniqueId : string
+  ) : void {
+    this._initSegmentsMap.delete(uniqueId);
+  }
+
   /**
    * Push a chunk of the media segment given to the attached SourceBuffer, in a
    * FIFO queue.
@@ -229,12 +247,12 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
     infos : IPushChunkInfos<unknown>,
     cancellationSignal : CancellationSignal
   ) : Promise<void> {
-    assertPushedDataIsBufferSource(infos);
+    assertDataIsBufferSource(infos.data.chunk);
     log.debug("AVSB: receiving order to push data to the SourceBuffer",
               this.bufferType,
               getLoggableSegmentId(infos.inventoryInfos));
     return this._addToQueue({ type: SegmentBufferOperation.Push,
-                              value: infos },
+                              value: infos as IPushChunkInfos<BufferSource> },
                             cancellationSignal);
   }
 
@@ -350,7 +368,7 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
    * @param {Event} err
    */
   private _onPendingTaskError(err : unknown) : void {
-    this._lastInitSegment = null; // initialize init segment as a security
+    this._lastInitSegmentUniqueId = null; // initialize init segment as a security
     if (this._pendingTask !== null) {
       const error = err instanceof Error ?
                       err :
@@ -447,7 +465,7 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
           const error = e instanceof Error ?
             e :
             new Error("An unknown error occured when preparing a push operation");
-          this._lastInitSegment = null; // initialize init segment as a security
+          this._lastInitSegmentUniqueId = null; // initialize init segment as a security
           nextItem.reject(error);
           return;
         }
@@ -557,15 +575,17 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
       this._sourceBuffer.appendWindowEnd = appendWindow[1];
     }
 
-    if (data.initSegment !== null &&
-        (hasUpdatedSourceBufferType || !this._isLastInitSegment(data.initSegment)))
+    if (data.initSegmentUniqueId !== null &&
+        (hasUpdatedSourceBufferType ||
+         !this._isLastInitSegment(data.initSegmentUniqueId)))
     {
       // Push initialization segment before the media segment
-      const segmentData = data.initSegment;
+      const segmentData = this._initSegmentsMap.get(data.initSegmentUniqueId);
+      if (segmentData === undefined) {
+        throw new Error("Invalid initialization segment uniqueId");
+      }
       dataToPush.push(segmentData);
-      const initU8 = toUint8Array(segmentData);
-      this._lastInitSegment = { data: initU8,
-                                hash: hashBuffer(initU8) };
+      this._lastInitSegmentUniqueId = data.initSegmentUniqueId;
     }
 
     if (data.chunk !== null) {
@@ -576,56 +596,37 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
   }
 
  /**
-  * Return `true` if the given `segmentData` is the same segment than the last
+  * Return `true` if the given `uniqueId` is the identifier of the last
   * initialization segment pushed to the `AudioVideoSegmentBuffer`.
-  * @param {BufferSource} segmentData
+  * @param {string} uniqueId
   * @returns {boolean}
   */
-  private _isLastInitSegment(segmentData : BufferSource) : boolean {
-    if (this._lastInitSegment === null) {
+  private _isLastInitSegment(uniqueId : string) : boolean {
+    if (this._lastInitSegmentUniqueId === null) {
       return false;
     }
-    if (this._lastInitSegment.data === segmentData) {
-      return true;
-    }
-    const oldInit = this._lastInitSegment.data;
-    if (oldInit.byteLength === segmentData.byteLength) {
-      const newInitU8 = toUint8Array(segmentData);
-      if (hashBuffer(newInitU8) === this._lastInitSegment.hash &&
-          areArraysOfNumbersEqual(oldInit, newInitU8))
-      {
-        return true;
-      }
-    }
-    return false;
+    return this._lastInitSegmentUniqueId === uniqueId;
   }
 }
 
 /**
  * Throw if the given input is not in the expected format.
  * Allows to enforce runtime type-checking as compile-time type-checking here is
  * difficult to enforce.
- * @param {Object} pushedData
+ * @param {Object} data
  */
-function assertPushedDataIsBufferSource(
-  pushedData : IPushChunkInfos<unknown>
-) : asserts pushedData is IPushChunkInfos<BufferSource> {
+function assertDataIsBufferSource(
+  data : unknown
+) : asserts data is BufferSource {
   if (__ENVIRONMENT__.CURRENT_ENV === __ENVIRONMENT__.PRODUCTION as number) {
     return;
   }
-  const { chunk, initSegment } = pushedData.data;
   if (
-    typeof chunk !== "object" ||
-    typeof initSegment !== "object" ||
-    (
-      chunk !== null &&
-      !(chunk instanceof ArrayBuffer) &&
-      !((chunk as ArrayBufferView).buffer instanceof ArrayBuffer)
-    ) ||
+    typeof data !== "object" ||
     (
-      initSegment !== null &&
-      !(initSegment instanceof ArrayBuffer) &&
-      !((initSegment as ArrayBufferView).buffer instanceof ArrayBuffer)
+      data !== null &&
+      !(data instanceof ArrayBuffer) &&
+      !((data as ArrayBufferView).buffer instanceof ArrayBuffer)
     )
   ) {
     throw new Error("Invalid data given to the AudioVideoSegmentBuffer");

src/core/segment_buffers/implementations/image/image_segment_buffer.ts

@@ -38,6 +38,22 @@ export default class ImageSegmentBuffer extends SegmentBuffer {
     this._buffered = new ManualTimeRanges();
   }
 
+  /**
+   * @param {string} uniqueId
+   */
+  public declareInitSegment(uniqueId : string): void {
+    log.warn("ISB: Declaring initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
+  /**
+   * @param {string} uniqueId
+   */
+  public freeInitSegment(uniqueId : string): void {
+    log.warn("ISB: Freeing initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
   /**
    * @param {Object} data
    * @returns {Promise}

src/core/segment_buffers/implementations/text/html/html_text_segment_buffer.ts

@@ -137,6 +137,22 @@ export default class HTMLTextSegmentBuffer extends SegmentBuffer {
     this.autoRefreshSubtitles(this._canceller.signal);
   }
 
+  /**
+   * @param {string} uniqueId
+   */
+  public declareInitSegment(uniqueId : string): void {
+    log.warn("ISB: Declaring initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
+  /**
+   * @param {string} uniqueId
+   */
+  public freeInitSegment(uniqueId : string): void {
+    log.warn("ISB: Freeing initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
   /**
    * Push text segment to the HTMLTextSegmentBuffer.
    * @param {Object} infos

src/core/segment_buffers/implementations/text/native/native_text_segment_buffer.ts

@@ -66,6 +66,22 @@ export default class NativeTextSegmentBuffer extends SegmentBuffer {
     this._trackElement = trackElement;
   }
 
+  /**
+   * @param {string} uniqueId
+   */
+  public declareInitSegment(uniqueId : string): void {
+    log.warn("ISB: Declaring initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
+  /**
+   * @param {string} uniqueId
+   */
+  public freeInitSegment(uniqueId : string): void {
+    log.warn("ISB: Freeing initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
   /**
    * @param {Object} infos
    * @returns {Promise}

src/core/segment_buffers/implementations/types.ts

@@ -87,6 +87,13 @@ export abstract class SegmentBuffer {
     this._segmentInventory = new SegmentInventory();
   }
 
+  public abstract declareInitSegment(
+    uniqueId : string,
+    initSegmentData : unknown
+  ) : void;
+
+  public abstract freeInitSegment(uniqueId : string) : void;
+
   /**
    * Push a chunk of the media segment given to the attached buffer, in a
    * FIFO queue.
@@ -96,7 +103,8 @@ export abstract class SegmentBuffer {
    * pushed.
    *
    * Depending on the type of data appended, the pushed chunk might rely on an
-   * initialization segment, given through the `data.initSegment` property.
+   * initialization segment, which had to be previously declared through the
+   * `declareInitSegment` method.
    *
    * Such initialization segment will be first pushed to the buffer if the
    * last pushed segment was associated to another initialization segment.
@@ -106,7 +114,7 @@ export abstract class SegmentBuffer {
    * reference).
    *
    * If you don't need any initialization segment to push the wanted chunk, you
-   * can just set `data.initSegment` to `null`.
+   * can just set the corresponding property to `null`.
    *
    * You can also only push an initialization segment by setting the
    * `data.chunk` argument to null.
@@ -230,12 +238,16 @@ export type IBufferType = "audio" |
  */
 export interface IPushedChunkData<T> {
   /**
-   * The whole initialization segment's data related to the chunk you want to
+   * The `uniqueId` of the initialization segment linked to the data you want to
    * push.
+   *
+   * That identifier should previously have been declared through the
+   * `declareInitSegment` method and not freed.
+   *
    * To set to `null` either if no initialization data is needed, or if you are
    * confident that the last pushed one is compatible.
    */
-  initSegment: T | null;
+  initSegmentUniqueId : string | null;
   /**
    * Chunk you want to push.
    * This can be the whole decodable segment's data or just a decodable sub-part