Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix Video Recording delay and improve startRecording() performance #2192

Merged
merged 24 commits into from
Nov 22, 2023

Conversation

mrousavy
Copy link
Owner

@mrousavy mrousavy commented Nov 21, 2023

What

This PR consists of two fixes:

  1. Fixes the delay of Video Recordings where Video buffers and Audio samples were offset by some significant amount causing the audio to not be in sync.
  2. Improves the performance of startRecording() by a lot so it no longer freezes for multiple seconds

Changes

  • perf: Remove AVAssetWriterInputPixelBufferAdaptor in favor of a simple AVAssetWriterInput
  • fix: Synchronize Audio Sample Buffers against the Video Capture Session so the audio track is no longer out of sync in video recordings
  • fix: Use the .videoRecording mode for the AVAudioSession
  • perf: Run Audio Session activation in parallel to speed up startRecording()
  • fix: Set AVAssetWriter.shouldOptimizeForNetworkUse to false
  • chore: Throw CameraErrors instead of custom errors in a few places
  • chore: Simplify the startRecording() function a bit

Tested on

  • iPhone 11 Pro
  • iPhone 15 Pro

Related issues

Copy link

vercel bot commented Nov 21, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
react-native-vision-camera ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 22, 2023 4:53pm

@mrousavy mrousavy marked this pull request as ready for review November 22, 2023 16:50
@mrousavy mrousavy merged commit cf8f3d0 into main Nov 22, 2023
8 checks passed
@mrousavy mrousavy deleted the fix/fix-recording-delay branch November 22, 2023 16:53
isaaccolson pushed a commit to isaaccolson/deliveries-mobile that referenced this pull request Oct 30, 2024
…ance (mrousavy#2192)

* fix: Add more logs

* perf: Init audio AVAssetWriter in parallel

* chore: Log Audio Session activation

* perf: Init Audio Session asynchronously

* chore: Log when category changed

* fix: Revert lazy audio initialization

* Update Podfile.lock

* Pass `sourceFormatHint` to video and audio AVAssetWriter

* fix: Remove `sourceFormatHint` from Video Asset Writer

* Use default options for Audio Asset Writer

* Update Podfile.lock

* Revert "Use default options for Audio Asset Writer"

This reverts commit e575a14.

* Add time logs

* fix: Don't synchronize audio buffers, they are already in sync

* shouldOptimizeForNetworkUse = false

* fix: Only update `latestTimestamp` once video buffer has been written

* perf: Use `AVAssetWriterInput` instead of `AVAssetWriterInputPixelBufferAdaptor`

* fix: Fix Audio not being synchronized with Video

* Remove logs add comments

* Format

* feat: Set `.videoRecording` AVAudioSession mode

* Refactor `startRecording()` a bit

* Format

* chore: Throw error directly instead of double-checking
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🐛 3.6.6: Audio & Video Recording out of sync
1 participant