Update dependency twilio-video to v2

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
twilio-video (source)	^1.18.1 -> ^2.0.0

---

Release Notes

twilio/twilio-video.js (twilio-video)

v2.32.1

Compare Source

====================

Bug Fixes

• Fixed an issue where video tracks would freeze when the video element was offscreen in a document Picture-in-Picture window.
• Fixed an issue where LocalVideoTrack.restart() would fail on some devices when a video processor was active.
• Fixed an issue where SDP munging would generate duplicated payload types, causing SDP negotiation failures with Chrome 137+.

v2.32.0

Compare Source

====================

New Features

Real-Time Transcriptions Support

Added real-time transcription capabilities for group rooms, allowing the capture and display of real-time captions during video calls.

Usage

To receive transcriptions, set the receiveTranscriptions option to true when connecting to a room. Then, subscribe to the transcription event to receive real-time transcription data:

import { connect } from 'twilio-video';

const room = await connect(token, {
  name: 'my-room',
  receiveTranscriptions: true
});

room.on('transcription', (transcriptionEvent) => {
  console.log(`${transcriptionEvent.participant}: ${transcriptionEvent.transcription}`);
});

See the Transcription's documentation for more details.

v2.31.0

Compare Source

=======================

New Features

Document Picture-in-Picture API Support

Previously, when Client Track Switch Off Control was set to auto, video tracks were automatically switched off—even if they were visible and rendered in a Document Picture-in-Picture (PiP) window. This caused issues where users could see the video element, but the track itself was disabled. In this new SDK version, video tracks will remain active and continue to play when displayed in a PiP window.

v2.30.0

Compare Source

========================

New Features

WebRTC Overrides (Beta)

The SDK now enables you to override WebRTC APIs with the following options. If your environment supports WebRTC redirection—such as Citrix HDX's WebRTC redirection technologies—your application can leverage this new beta feature to enhance media quality in those environments. For more details, please refer to the JavaScript WebRTC Overrides documentation.

ConnectOptions

• rtcConfiguration - An optional RTCConfiguration object passed to the RTCPeerConnection constructor.
• RTCPeerConnection - Allows overriding the native RTCPeerConnection class.

ConnectOptions and CreateLocalTracksOptions

• getUserMedia - Overrides the native MediaDevices.getUserMedia API.
• enumerateDevices - Overrides the native MediaDevices.enumerateDevices API.
• MediaStream - Overrides the native MediaStream class.
• mapMediaElement - Callback triggered after a media track is attached to an audio or video element.
• disposeMediaElement - Callback triggered after a media track is detached from an audio or video element.

Bug Fixes

• Fixed a bug where a Chrome iOS Participant's local audio (Krips Noise Cancellation enabled) did not recover after disconnecting a bluetooth headset. (VIDEO-13010)

v2.29.0

Compare Source

================================

Changes

Video Processor V3 support (Beta)

• AddProcessorOptions.outputFrameBufferContextType = 'bitmaprenderer' is now supported on Safari and Firefox. (VBLOCKS-3643, VBLOCKS-3644)
• AddProcessorOptions.inputFrameBufferType now has a new value videoframe. On browsers that support VideoFrame, the inputFrameBuffer argument of VideoProcessor.processFrame() will be a VideoFrame. On other supported browsers, it will be an HTMLVideoElement.
• AddProcessorOptions.outputFrameBufferContextType now has a new value bitmaprenderer. Currently, this is only supported for Chromium-based browsers. On other supported browsers, it falls back to 2d.
• Patched the build script to work around the issue: markdown-it/linkify-it#111.

v2.28.2

Compare Source

==========================

Bug Fixes

• Fixed a bug in Desktop Safari 18 and all iOS browsers on iOS 18 where cloning a disabled MediaStreamTrack would incorrectly set the enabled property to true instead of preserving the original disabled state. This ensures track cloning behavior matches the MediaStreamTrack specification and works consistently across browsers. Bug report: https://bugs.webkit.org/show_bug.cgi?id=281758

v2.28.1

Compare Source

========================

Bug Fixes

• Previously, a Chrome iOS 17 Participant's local audio (Krisp noise cancellation enabled) did not recover after foregrounding the browser following the playing of a YouTube video (or some other application which requires microphone permissions). We work around this by permanently disabling the Krisp noise cancellation upon foregrounding the browser. (VIDEO-13006)

v2.28.0

Compare Source

===========================

Bug Fixes

• Fixed a bug where a Chrome iOS 17 Participant's local and remote media (Krisp Noise Cancellation enabled) failed to recover after interacting with a PSTN call in full-screen mode. (VIDEO-13011)
• Fixed a bug where a Chrome iOS 17 Participant's audio with Krisp Noise Cancellation did not recover after interacting with a system popup. (VIDEO-13012)
• Fixed a bug where a Chrome iOS 17 Participant's audio with Krisp Noise Cancellation did not recover after invoking Siri. (VIDEO-13013)

v2.27.0

Compare Source

=======================

Changes

VideoTrack.addProcessor now works on browsers that support OffscreenCanvas as well as HTMLCanvasElement. When used with
@​twilio/video-processors v2.0.0, the Virtual
Background feature will work on browsers that supports WebGL2.
See VideoTrack.addProcessor and
@​twilio/video-processors v2.0.0 for details.

Example

import { createLocalVideoTrack } from 'twilio-video';
import { Pipeline, VirtualBackgroundProcessor } from '@​twilio/video-processors';

const virtualBackgroundProcessor = new VirtualBackgroundProcessor({
  pipeline: Pipeline.WebGL2,
  // ...otherOptions
});

await virtualBackgroundProcessor.loadModel();

const videoTrack = await createLocalVideoTrack({
  width: 640,
  height: 480,
  frameRate: 24
});

videoTrack.addProcessor(processor, {
  inputFrameBufferType: 'video',
  outputFrameBufferContextType: 'webgl2',
});

v2.26.2

Compare Source

==========================

Changes

• Starting from version 110, Chrome will no longer support the iSAC audio codec. The SDK will now log a warning to the console
  whenever an audio or a video codec that is specified in ConnectOptions.preferredAudioCodecs and ConnectOptions.preferredVideoCodecs
  is not supported by the browser. (VIDEO-12494)

Bug Fixes

• Fixed a bug on Chrome versions 112+ where Room.getStats() did not reject the returned Promise when an exception was
  raised while accessing WebRTC stats that due to a TypeError caused by trying to read from the now-removed RTCMediaStreamTrackStats. (VIDEO-12534)

v2.26.1

Compare Source

=========================

Bug Fixes

• Fixed a bug that manifests on Chrome versions 112+ where Room.getStats() raises an unhandled exception due to a
  TypeError caused by trying to read from the now-removed RTCMediaStreamTrackStats. Instead, the SDK now reads from
  the RTCMediaSourceStats. (VIDEO-12411)
• Fixed an error in the type definition for the attach() method of AudioTrack and VideoTrack. (VIDEO-12242)
• Fixed an error in the type definition for createLocalAudioTrack(). (VIDEO-12383)

v2.26.0

Compare Source

==========================

New Features

• The LocalAudioTrack and
  LocalVideoTrack classes now provide a
  new boolean property called isMuted, which lets you know if the audio or video source is currently providing raw media
  samples. The classes also emit muted and unmuted events if the value of isMuted toggles. The application can use
  these APIs to detect temporary loss of microphone or camera to other applications (ex: an incoming phone call on an iOS device),
  and update the user interface accordingly. (VIDEO-11360)

• The Room class provides a new method called refreshInactiveMedia,
  which restarts any muted local media Tracks, and plays any inadvertently paused HTMLMediaElements that are attached to
  local and remote media Tracks. This is useful especially on iOS devices, where sometimes your application's media may
  not recover after an incoming phone call. You can use this method in conjunction with the local media Track's isMuted
  property described previously to recover local and remote media after an incoming phone call as shown below. (VIDEO-11360)

Vanilla JS

html

<button id="refresh-inactive-media" disabled>Refresh Inactive Media</button>

js

const { connect } = require('twilio-video');

const room = await connect('token', { name: 'my-cool-room' });

const $refreshInactiveMedia = document.getElementById('refresh-inactive-media');
$refreshInactiveMedia.onclick = () => room.refreshInactiveMedia();

const [{ track: localAudioTrack }] = [...room.localParticipant.audioTracks.values()];
const [{ track: localVideoTrack }] = [...room.localParticipant.videoTracks.values()];

const isLocalAudioOrVideoMuted = () => {
  return localAudioTrack.isMuted || localVideoTrack.isMuted;
}

const onLocalMediaMutedChanged = () => {
  $refreshInactiveMedia.disabled = !isLocalAudioOrVideoMuted();
};

[localAudioTrack, localVideoTrack].forEach(localMediaTrack => {
  ['muted', 'unmuted'].forEach(event => {
    localMediaTrack.on(event, onLocalMediaMutedChanged);
  });
});

React

src/hooks/useLocalMediaMuted.js

import { useEffect, useState } from 'react';

export default function useLocalMediaMuted(localMediaTrack) {
  const [isMuted, setIsMuted] = useState(localMediaTrack?.isMuted ?? false);

  useEffect(() => {
    const updateMuted = () => setIsMuted(localMediaTrack?.isMuted ?? false);
    updateMuted();

    localMediaTrack?.on('muted', updateMuted);
    localMediaTrack?.on('unmuted', updateMuted);

    return () => {
      localMediaTrack?.off('muted', updateMuted);
      localMediaTrack?.off('unmuted', updateMuted);
    };
  }, [localMediaTrack]);

  return isMuted;
}

src/components/room.js

import useLocalMediaMuted from '../hooks/useLocalMediaMuted';

export default function Room({ room }) {
  const [{ track: localAudioTrack }] = [...room.localParticipant.audioTracks.values()];
  const [{ track: localVideoTrack }] = [...room.localParticipant.videoTracks.values()];

  const isLocalAudioMuted = useLocalMediaMuted(localAudioTrack);
  const isLocalVideoMuted = useLocalMediaMuted(localVideoTrack);
  const isLocalMediaMuted = isLocalAudioMuted || isLocalVideoMuted;

  const refreshInactiveMedia = () => {
    room.refreshInactiveMedia();
  };

  return (
    <>
      ...
      {isLocalMediaMuted && <Button onClick={refreshInactiveMedia}>
        Refresh Inactive Media
      </Button>}
      ...
    </>
  );
}

v2.25.0

Compare Source

==========================

New Features

Auto-switch default audio input devices

This release adds a new feature that preserves audio continuity in situations where end-users change the default audio input device.
A LocalAudioTrack is said to be capturing audio from the default audio input device if:

• it was created using the MediaTrackConstraints { audio: true }, or
• it was created using the MediaTrackConstraints { audio: { deviceId: 'foo' } }, and "foo" is not available, or
• it was created using the MediaTrackConstraints { audio: { deviceId: { ideal: 'foo' } } } and "foo" is not available

In previous versions of the SDK, if the default device changed (ex: a bluetooth headset is connected to a mac or windows laptop),
the LocalAudioTrack continued to capture audio from the old default device (ex: the laptop microphone). Now, a LocalAudioTrack
will switch automatically from the old default audio input device to the new default audio input device (ex: from the laptop microphone to the headset microphone).
This feature is controlled by a new CreateLocalAudioTrackOptions
property defaultDeviceCaptureMode, which defaults to auto (new behavior) or can be set to manual (old behavior).

The application can decide to capture audio from a specific audio input device by creating a LocalAudioTrack:

• using the MediaTrackConstraints { audio: { deviceId: 'foo' } }, and "foo" is available, or
• using the MediaTrackConstraints { audio: { deviceId: { ideal: 'foo' } } } and "foo" is available, or
• using the MediaTrackConstraints { audio: { deviceId: { exact: 'foo' } } } and "foo" is available

In this case, the LocalAudioTrack DOES NOT switch to another audio input device if the current audio input device is no
longer available. See below for the behavior of this property based on how the LocalAudioTrack is created. (VIDEO-11701)

const { connect, createLocalAudioTrack, createLocalTracks } = require('twilio-video');

// Auto-switch default audio input devices: option 1
const audioTrack = await createLocalAudioTrack();

// Auto-switch default audio input devices: option 2
const audioTrack1 = await createLocalAudioTrack({ defaultDeviceCaptureMode: 'auto' });

// Auto-switch default audio input devices: option 3
const [audioTrack3] = await createLocalTracks({ audio: true });

// Auto-switch default audio input devices: option 4
const [audioTrack4] = await createLocalTracks({ audio: { defaultDeviceCaptureMode: 'auto' } });

// Auto-switch default audio input devices: option 5
const room1 = await connect({ audio: true });

// Auto-switch default audio input devices: option 6
const room2 = await connect({ audio: { defaultDeviceCaptureMode: 'auto' } });

// Disable auto-switch default audio input devices
const room = await createLocalAudioTrack({ defaultDeviceCaptureMode: 'manual' });

Limitations

• This feature is not enabled on iOS as it is natively supported.
• Due to this WebKit bug, MacOS Safari Participants may lose their local audio after switching between default audio input devices two-three times.
• This feature is not supported on Android Chrome, as it does not support the MediaDevices.ondevicechange event.

v2.24.3

Compare Source

=========================

Bug Fixes

• Fixed a bug where iOS Safari Participant could not hear or see others after switching back from YouTube picture-in-picture mode. (VIDEO-11352)
• Fixed a bug where iOS Safari Participant could not hear others after switching from recording an audio message in a messenger app. (VIDEO-11354)
• Fixed a bug where iOS Safari Participant could not hear or see others after watching a video in another browser tab. (VIDEO-11356)
• Fixed a bug where iOS Safari Participant sometimes could not hear or see others after finishing an incoming call in full screen mode. (VIDEO-11359)

v2.24.2

Compare Source

===========================

Bug Fixes

• Fixed a bug where sometimes, a MediaClientRemoteDescFailedError was raised when a Chrome Participant who had enabled
  Adaptive Simulcast (ConnectOptions.preferredVideoCodecs = 'auto') tried to publish a camera Track after publishing a
  <canvas> Track. (VIDEO-11516)
• Fixed an issue where the Krisp Noise Cancellation fails to load in an application where the content security policy
  directives default-src self unsafe-eval are used. (VIDEO-11537)

v2.24.1

Compare Source

==========================

Bug Fixes

• Fixed a bug where sometimes a runtime error was raised on iOS devices as shown below. (VIDEO-11263)

  Unhandled Runtime Error: TypeError: null is not an object (evaluating 'el.paused')
  

• The LocalTrackOptions type definition now contains logLevel as an optional property. (VIDEO-10659)
• Fixed an issue where the import keyword was causing problems in webpack and typescript projects. (VIDEO-11220)

v2.24.0

Compare Source

========================

New Features

• The support for twilio approved 3rd party noise cancellation solutions is now generally available.

Bug Fixes

• Fixed an issue where input media track was not stopped, after localAudioTrack.stop() when using noiseCancellation (VIDEO-11047)
• Added versioning support for noise cancellation plugin. This SDK will require noise cancellation plugin to be version 1.0.0 or greater. (VIDEO-11087)

v2.23.0

Compare Source

======================

New Features

• This release adds private beta support for 3rd party noise cancellation solution. You need to host twilio approved 3rd party plugin on your web server to enable noise cancellation. Please fill out this form to request access to the 3rd party plugin.

Once you get the access to the plugin, You can install it from npm with:

npm install <noise_cancellation_plugin>

Once installed, you need to host the contents of ./node_modules/<noise_cancellation_plugin>/dist/ from your web server. We recommend that you add plugin version number to the hosted path to ensure that browser does not use stale version when its updated. You need to pass the path to the hosted files to twilio-video sdk when creating audio track as shown in the example below. The example below assumes that you have hosted the files at /noise_cancellation_plugin/1.0.0/dist on your web server.

const { connect, createLocalAudioTrack } = require('twilio-video');

// create a local audio track and have it use
// @​twilio/krisp-audio-plugin for noise cancellation processing.
const localAudioTrack = await Video.createLocalAudioTrack({
  noiseCancellationOptions: {
    vendor: 'krisp',
    sdkAssetsPath: '/noise_cancellation_plugin/1.0.0/dist'
  }
});

// publish the track to a room
const room = await connect( token, {
  tracks: [localAudioTrack]
  // ... any other connect options
});

// you can enable/disable noise cancellation at runtime
// using noiseCancellation interface exposed by localAudioTrack
function updateNoiseCancellation(enable: boolean) {
  const noiseCancellation = localAudioTrack.noiseCancellation;

  if (noiseCancellation) {
    enable ? noiseCancellation.enable() : noiseCancellation.disable();
  }
}

NOTE: If your application is using the default-src self content security policy directive, then you should add
another directive unsafe-eval, which is required for the Krisp Audio Plugin to load successfully.

v2.22.2

Compare Source

======================

Changes

• isSupported flag now returns false if the browser does not support the Unified Plan SDP format. (VIDEO-10307)

  The following is a list of browsers with Unified Plan as the default SDP format.

  • Chrome 72+
  • Safari 12.1+
  • Firefox 38+

v2.22.1

Compare Source

======================

Bug Fixes

• The encoding of audio and screen share Tracks are prioritized in Chrome and Safari, thereby more gracefully degrading
  their quality in limited network conditions. (VIDEO-10212)

v2.22.0

Compare Source

====================

New Features

This release include the Media Warnings API (Beta) to help surface media related warning events on the SDK whenever the media server is not able to detect media from a published audio or video track.

Example

const room = await connect('token', {
  notifyWarnings: [ 'recording-media-lost' ]
  // Other connect options
});

Array.from(room.localParticipant.tracks.values()).forEach(publication => {
  publication.on('warning', name => {
    if (name === 'recording-media-lost') {
      console.log(`LocalTrack ${publication.track.name} is not recording media.`);

      // Wait a reasonable amount of time to clear the warning.
      const timer = setTimeout(() => {
        // If the warning is not cleared, you can manually
        // reconnect to the room, or show a dialog to the user
      }, 5000);

      publication.once('warningsCleared', () => {
        console.log(`LocalTrack ${publication.track.name} warnings have cleared!`);
        clearTimeout(timer);
      });
    }
  });
});

API Definitions

ConnectOptions

• notifyWarnings - An array of warnings to listen to. By default, this array is empty and no warning events will be raised. Possible warning values include:

  • recording-media-lost - Raised when the media server has not detected any media on the published track that is being recorded in the past 30 seconds. This usually happens when there are network interruptions or when the track has stopped.

Events

The SDK raises warning events when it detects certain conditions. You can implement callbacks on these events to act on them, or to alert the user of an issue. Subsequently, "warningsCleared" event is raised when conditions have returned to normal.

• LocalTrackPublication.on('warning', callback(name)) - Raised when the published Track encounters a warning.

• LocalTrackPublication.on('warningsCleared', callback()) - Raised when the published Track cleared all warning.

• LocalParticipant.on('trackWarning', callback(name, publication)) - Raised when one of the LocalParticipant's published tracks encounters a warning.

• LocalParticipant.on('trackWarningsCleared', callback(publication)) - Raised when one of the LocalParticipant's published tracks cleared all warning.

• Room.on('trackWarning', callback(name, publication, participant)) - Raised when one of the LocalParticipant's published tracks in the Room encounters a warning.

• Room.on('trackWarningsCleared', callback(publication, participant)) - Raised when one of the LocalParticipant's published tracks in the Room cleared all warning.

v2.21.3

Compare Source

====================

Bug Fixes

• Fixed an issue where the generated API documentation has a missing search bar. (VIDEO-10199)

v2.21.2

Compare Source

=====================

Bug Fixes

• Fixed an issue where publishing a video track sometimes caused a failure with "Unhandled exception: Client is unable to create or apply a local media description". (VIDEO-9511)
• Fixed an issue where the dimensionsChanged event was not firing when the track dimensions first became available. (VIDEO-3576)
• Removed references to node dependencies that causes build errors on Angular and Vue. (VIDEO-9282)
• Fixed an issue where incorrect device was detected when using iPad in Desktop Website mode. (VIDEO-8282)

v2.21.1

Compare Source

=======================

Bug Fixes

• Fixed the issue where twilio-video.js does not build with the latest version of webpack and vite. (VIDEO-8609)

v2.21.0

Compare Source

======================

New Features

• twilio-video.js now supports WKWebView and SFSafariViewController on iOS version 14.3 or later. The isSupported flag relies partly on the User-Agent string to determine if twilio-video.js officially supports the user's browser. If your application modifies the default value for the User-Agent string, the new value should follow the correct format.

  Additionally, for iOS applications, your application will need to include the camera usage description, microphone usage description and inline media playback in order for the SDK to work on WKWebView.

  Note: As with Safari, WKWebViews only support only one local media track of each kind at a time.

  We also would like to thank @​cbxp for his contribution. (VIDEO-8374)

Known Issue

Some common issues such as interruptions on mobile devices which includes, backgrounding the application, or switching between applications can sometimes cause VideoTracks to go black or AudioTracks to stop.

v2.20.1

Compare Source

=====================
Bug Fixes

• Fixed a bug that was introduced in 2.19.0 where the published LocalVideoTracks of Participants on older iOS versions 14.5 and below did not encode and transmit media. (VIDEO-8770)

v2.20.0

Compare Source

==========================

Changes

The Preflight API (runPreflight), originally released in 2.16.0, has been promoted to GA.

Thank you @​morninng @​eroidaaruqaj #​1622 for your feedback. Based on this feedback, we have made the following changes to runPreflight. (VIDEO-7728)

• The failed event now provides a PreflightTestReport which include partial results gathered during the test. Use this in addition to the error object to get more insights on the failure.

• Signaling and Media Connection errors are now properly surfaced via the failed event.

• PreflightTestReport now includes a progressEvents property. This new property is an array of PreflightProgress events detected during the test. Use this information to determine which steps were completed and which ones were not.

You can learn more about runPreflight usage in the documentation, here.

Other changes in this release includes:

• In October 2019, twilio-video.js started using Unified Plan where available, while also maintaining support for earlier browser versions with Plan B as the default SDP format. With this release, twilio-video.js will now stop supporting the Plan B SDP format and will only support the Unified Plan SDP format. Please refer to this changelog and this public advisory for more related information. (VIDEO-6587)

v2.19.1

Compare Source

=========================

Bug Fixes

• Fixed a bug where media connection was not getting reconnected after a network interruption if participant was not subscribed to any tracks. (VIDEO-8315)
• Fixed a bug where network quality score stops updating after network glitches. (VIDEO-8413)

v2.19.0

Compare Source

=========================

New Features

• This release introduces a new feature Adaptive Simulcast. This opt-in feature can be enabled by setting preferredVideoCodecs="auto" in ConnectOptions. When joining a group room with this feature enabled, the SDK will use VP8 simulcast, and will enable/disable simulcast layers dynamically, thus improving bandwidth and CPU usage for the publishing client. It works best when used along with Client Track Switch Off Control and Video Content Preferences. These two flags allow the SFU to determine which simulcast layers are needed, thus allowing it to disable the layers not needed on publisher side. This feature cannot be used alongside maxVideoBitrate.

If your application is currently using VP8 simulcast we recommend that you switch to this option.

Example:

const { connect } = require('twilio-video');

const room = await connect(token, {
  preferredVideoCodecs: 'auto',
  bandwidthProfile: {
    video: {
      contentPreferencesMode: 'auto',
      clientTrackSwitchOffControl: 'auto'
    }
  }
});

Known Limitations

• Specifying preferredVideoCodecs="auto" will revert to unicast in the following cases:
  • The publisher is using Firefox.
  • The publisher has preferred the H264 codec.
  • The Room is configured to support only the H264 codec.
  • Peer-to-Peer Rooms
• When the participant is being recorded, the SFU will not disable any simulcast layers of the participant's VideoTrack.

Bug Fixes

• Fixed a bug where clientTrackSwitchOffControl and contentPreferencesMode sometimes did not work as expected during network glitches. (VIDEO-7654)

v2.18.3

Compare Source

========================

Bug Fixes

• Fixed a bug where connect was returning a Promise type instead of a CancelablePromise. (VIDEO-7831)
• Fixed a bug where audioLevel, frameRate, and captureDimensions WebRTC stats are returning null on certain browsers. With this release, these stats are now populated whenever they are available. (VIDEO-3600)

v2.18.2

Compare Source

==========================

Bug Fixes

• Fixed a bug where setting clientTrackSwitchOffControl to auto caused the RemoteVideoTracks to get switched off while playing in picture-in-picture mode. Note that this fix does not apply to Firefox as it does not yet implement picture-in-picture APIs. (VIDEO-6677)

v2.18.1

Compare Source

=========================

Changes

• Added some metrics to track the usage of the Preflight API Public Beta (runPreflight). There are no changes to the public APIs in this release. (VIDEO-6891)

v2.18.0

Compare Source

=========================

New Features

• When a LocalParticipant tries to publish a LocalVideoTrack in an Audio Only Group Room,
  it will fail with a RoomTrackKindNotSupportedError. (VIDEO-7242)

Known Issue

In Firefox, although the publishing of a LocalVideoTrack in an Audio Only Group Room fails,
the RoomTrackKindNotSupportedError is not raised. We are actively working on fixing this issue.

v2.17.1

Compare Source

===========================
Bug Fixes

• Fixed a regression in 2.17.0 which caused Chrome screen share tracks to be encoded at lower dimensions. (VIDEO-7000)

v2.17.0

Compare Source

===========================
New Features

• twilio-video.js now supports Chrome on iOS versions 14.3 and above. (VIDEO-5723)

Bug Fixes

• Fixed a bug where the VideoTracks of Safari Participants with VP8 simulcast enabled sometimes had low frame rates. (VIDEO-6263)
• Fixed a bug where the screen share track got restarted as a camera track if it was ended when the application was foreground. (VIDEO-3977)

v2.16.0

Compare Source

========================
New Features

This release includes the Preflight API Public Beta (runPreflight) to help test connectivity with Twilio servers. It can be used to detect issues prior to joining a Video Room or as part of a troubleshooting page.

The API connects two peer connections using Twilio's Signaling and TURN servers. It publishes synthetic audio and video tracks from one participant and ensures that other participant receives media on those tracks. After successfully verifying connectivity, it generates a report with information on the connection.

runPreflight was originally introduced as an experimental API in 2.8.0-beta1 and has been updated based on feedback. In short, usage of the API will now be free of charge.

Example:

const { runPreflight } = require('twilio-video');
const token = getAccessToken();

const preflightTest = runPreflight(token);

preflightTest.on('progress', (progress: string) => {
  console.log('preflight progress:', progress);
});

preflightTest.on('failed', (error: Error) => {
  console.error('preflight error:', error);
});

preflightTest.on('completed', (report: PreflightTestReport) => {
  console.log("Test completed in " + report.testTiming.duration + " milliseconds.");
  console.log(" It took " + report.networkTiming.connect?.duration + " milliseconds to connect");
  console.log(" It took " + report.networkTiming.media?.duration + " milliseconds to receive media");
});

The PreflightTestReport generated by preflightTest on completed provides statistics that can be useful in cases where there is a poor connection. Some of the useful statistics in the report are as follows:

• Packet loss, round trip time, and jitter observed on the connection
• Network timing measurements on the progress events, such as time to connect or to receive media.
• Ice candidates and selected ice candidate pairs

preflightTest emits a failed event to indicate test failures. You can use the PreflightProgress events to better understand where the test failed and refer to
this guide for interpreting common errors.

A few things to note:

• This function uses web audio API's. Browser's autoplay policies sometimes require user action before accessing these APIs. Please ensure that this API is called in response to user action like a button click.
• For testing limitations in available bandwidth, we recommend you use testMediaConnectionBitrate from the RTC Diagnostics SDK.

Bug Fixes

Fixed a bug where the SDK was holding on to internally maintained audio elements longer than needed, now they will be cleaned up once track has started. (VIDEO-6480)

v2.15.3

Compare Source

======================

Bug Fixes

Fixed a bug where the SDK was not cleaning up internally maintained media elements. This causes memory leaks on certain use cases such as reconnecting or republishing to a room (VIDEO-6336).

Additionally, Chrome 92 started enforcing limit on number of WebMediaPlayers. This blocks creation of WebMediaPlayers once the limit is reached - 75 for desktop and 40 for mobile. This SDK update will help prevent running into this limit issue on use cases such as reconnecting or republishing to a room. Please ensure that your application cleans up media elements as well after they are detached.

const elements = track.detach();
elements.forEach(el => {
  el.remove();
  el.srcObject = null;
});

Please be aware that your application may still run into the Chrome's WebMediaPlayers limit for large rooms where participants exceeds this limit.

v2.15.2

Compare Source

======================

Bug Fixes

Fixed a bug where setting clientTrackSwitchOffControl to auto caused the tracks to get switched off aggressively, which resulted in momentary black track during app layout changes (VIDEO-5226).

v2.15.1

Compare Source

=====================

New Features

Updated June 24, 2021

• The Video Processor API has been promoted to GA. There are no changes to the API at this moment and we will continue to improve it on future releases.

Bug Fixes

• Fixed a bug where twilio-video was throwing an exception in a server-side rendering application.

v2.15.0

Compare Source

=====================

Breaking Change on Video Processor API (Beta)

VideoProcessor.processFrame method signature has been changed in order to improve the performance of the Video Processor API. With this update, the output frame buffer is now provided to the processFrame method which should be used to draw the processed frame.

Old signature:

processFrame(inputFrame: OffscreenCanvas)
  : Promise<OffscreenCanvas | null>
  | OffscreenCanvas | null;

New signature:

processFrame(inputFrameBuffer: OffscreenCanvas, outputFrameBuffer: HTMLCanvasElement)
  : Promise<void> | void;

Example:

class GrayScaleProcessor {
  constructor(percentage) {
    this.percentage = percentage;
  }
  processFrame(inputFrameBuffer, outputFrameBuffer) {
    const context = outputFrameBuffer.getContext('2d');
    context.filter = `grayscale(${this.percentage}%)`;
    context.drawImage(inputFrameBuffer, 0, 0, inputFrameBuffer.width, inputFrameBuffer.height);
  }
}

Video.createLocalVideoTrack().then(function(videoTrack) {
  videoTrack.addProcessor(new GrayScaleProcessor(100));
});

Bug Fixes

• Fixed a bug where isSupported was returning true on certain unsupported mobile browsers. With this release, isSupported should now return true only for the browsers supported by twilio-video.js.

• Updated NetworkQualityBandwidthStats documentation to reflect the correct bandwidth units, in bits per second, instead of bytes.

v2.14.0

Compare Source

=====================

New Features

This release contains a significant update to the Bandwidth Profile API. It allows for more efficient use of bandwidth and CPU in multi-party applications. In addition it provides developers with more dynamic control over which video tracks are delivered to the client and the preferred video resolution of the tracks. These capabilities are provided via the Client Track Switch Off Control and Content Preferences settings.

Existing Bandwidth Profile settings will continue to function as before, however we recommend developers update their Bandwidth Profile settings to make use of these new capabilities at their earliest convenience.

Client Track Switch Off Control

• This feature allows subscribers to control whether the media for a RemoteVideoTrack is received or not. Client Track Switch Off Control has two modes of operation:
  • auto (default): The SDK determines whether tracks should be switched off based on document visibility, track attachments, and / or the visibility of video elements.
  • manual: The application requests that individual tracks be switched off or on using the RemoteVideoTrack.switchOff() / switchOn() methods.
• Note: If your application previously set the maxTracks property to limit the number of tracks visible, you should migrate to using clientTrackSwitchOffControl to take advantage of this feature.

Video Content Preferences

• This feature allows subscribers to specify preferences about the media that they receive on a RemoteVideoTrack. Video content preferences has two modes of operation:
  • auto (default): The SDK specifies content preferences based on video element size. A RemoteVideoTrack attached to a video element with larger dimensions will get a higher quality video compared to a RemoteVideoTrack attached to a video element with smaller dimensions.
  • manual: The application specifies the content preferences for individual tracks using RemoteVideoTrack.setContentPreferences().
• Note: If your application previously set the renderDimensions property, you should migrate to using contentPreferencesMode to take advantage of this feature.

Both of these features are available in Group Rooms and are enabled by default if your application specifies Bandwidth Profile Options during connect.

const { connect } = require('twilio-video');
const room = await connect(token, {
  name: 'my-new-room',
  bandwidthProfile: {
    video: {
      /* Defaults to "auto" for both features. Be sure to remove "renderDimensions" and "maxTracks". */
    }
  }
});

Migrating to Attach APIs

The automatic behaviors rely on applications using the attach and detach methods of RemoteVideoTrack. If your application currently uses the underlying MediaStreamTrack to associate Tracks to video elements, you will need to update your application to use the attach/detach methods or use the manual APIs.

Manual Controls

const room = await connect(token, {
  bandwidthProfile: {
    video: {
      contentPreferencesMode: 'manual',
      clientTrackSwitchOffControl: 'manual'
    }
  }
});

When manual controls are used you can operate directly on RemoteVideoTrack to specify preferences. For example, applications can:

1. Force disabling a track.

remoteTrack.switchOff();

2. Enable and request QVGA video.

v2.13.1

Compare Source

=======================

New Features

• The Video Processor API has been promoted to beta. There are no changes to the API at this moment and we will continue to improve it on future releases.

Bug Fixes

• Fixed a bug where Android Firefox Participants sometime failed to publish VP8 VideoTracks in a Group Room. (VIDEO-3736)

v2.13.0

Compare Source

======================

New Features

Video Processor API Pilot (Chrome only)

• You can now register a VideoProcessor with a VideoTrack in order to process its video frames. In a LocalVideoTrack, video frames are processed before being sent to the encoder. In a RemoteVideoTrack, video frames are processed before being sent to the attached <video> element(s). The VideoProcessor should implement the interface shown below. (VIDEO-3560, VIDEO-3561)

  abstract class VideoProcessor {
    abstract processFrame(inputFrame: OffscreenCanvas)
      : Promise<OffscreenCanvas | null>
      | OffscreenCanvas | null;
  }

  A VideoTrack provides new methods addProcessor and removeProcessor which can be used to add and remove a VideoProcessor. It also provides a new property processor which points to the current VideoProcessor being used by the VideoTrack. For example, you can toggle a blur filter on a LocalVideoTrack as shown below.

  import { createLocalVideoTrack } from 'twilio-video';
  
  class BlurVideoProcessor {
    private readonly _outputFrameCtx: CanvasRenderingContext2D;
    private readonly _outputFrame: OffscreenCanvas;
  
    constructor(width: number, height: number, blurRadius: number) {
      this._outputFrame = new OffscreenCanvas(width, height);
      this._outputFrameCtx = this._outputFrame.getContext('2d');
      this._outputFrameCtx.filter = `blur(${blurRadius}px)`;
    }
  
    processFrame(inputFrame: OffscreenCanvas) {
      this._outputFrameCtx.drawImage(inputFrame, 0, 0);
      return this._outputFrame;
    }
  }
  
  // Local video track
  createLocalVideoTrack({
    width: 1280,
    height: 720
  }).then(track => {
    const processor = new BlurVideoProcessor(1280, 720, 5);
    document.getElementById('preview').appendChild(track.attach());
    document.getElementById('toggle-blur').onclick = () => track.processor
      ? track.removeProcessor(processor)
      : track.addProcessor(processor);
  });

  You can also toggle a blur filter on a RemoteVideoTrack as shown below.

  room.on('trackSubscribed', track => {
    if (track.kind === 'video') {
      const { width, height } = track.dimensions;
      const processor = new BlurVideoProcessor(width, height, 3);
      document.getElementById('preview-remote').appendChild(track.attach());
      document.getElementById('toggle-blur-remote').onclick = () => track.processor
        ? track.removeProcessor(processor)
        : track.addProcessor(processor);
    }
  });

v2.12.0

Compare Source

=====================

New Features

100 Participant Rooms Pilot

• In this pilot program developers can connect to a Group Room with Maximum Participants set between 50 and 100.
  A Room created with Max Participants greater than 50 is structured to support a small number of presenters and a large number of viewers. It has the following behavioral differences compared to regular Group Rooms:
  • "participantConnected" event is raised on the Room when a RemoteParticipant
    publishes the first LocalTrack.
  • "participantDisconnected" event is raised on the Room when a RemoteParticipant
    stops publishing all of its LocalTracks.
  • The total number of published Tracks in the Room cannot exceed 16 at any one time. Any attempt
    to publish more Tracks will be rejected with a ParticipantMaxTracksExceededError. (JSDK-3021)

Bug Fixes

• Fixed a bug where calling LocalMediaTrack.restart() logged a warning about PeerConnection being closed in Peer to Peer Rooms. (JSDK-2912)
• Fixed a race condition that sometimes caused switchedOff event for RemoteVideoTrack to not get emitted, which also resulted in wrong value for RemoteVideoTrack.isSwitchedOff property. (VIDEO-3695)

v2.11.0

Compare Source

=========================

• You can now import type definitions for the SDK APIs to your project. Previously, typescript developers relied on definitelyTyped for these definitions. We would like to thank the folks at DefinitelyTyped for maintaining these definitions. Going forward, the definitions will be included in the library and will take precedence over any other type definitions that you may be using. (JSDK-3007)

You can access the types of the public API classes from the Video namespace as shown below

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.