Disable Server_VAD in VoiceActivityDetectionSettings

[timbowicz]

Hi Stephen, first of all great work with the openai implementation.

I want to mimic push to talk functionality with the realtime streaming model. From what I read in the openai documentation you have to disable the 'Server_VAD' for that and send the audio input manually.

I've done this :

var options = new Options(
    model: Model.GPT4oRealtime,
    instructions: systemPrompt,
    voice:voice,
    turnDetectionSettings: new VoiceActivityDetectionSettings(TurnDetectionType.Disabled));
                
session = await openAI.RealtimeEndpoint.CreateSessionAsync(options, destroyCancellationToken);

But it throws this error :

Exception: [missing_required_parameter] Missing required parameter: 'session.turn_detection.type'. Type: invalid_request_error Parameter: session.turn_detection.type
Utilities.Async.AwaiterExtensions.WithCancellation[T] (System.Threading.Tasks.Task`1[TResult] task, System.Threading.CancellationToken cancellationToken) (at ./Library/PackageCache/[email protected]/Runtime/Async/AwaiterExtensions.cs:163)
OpenAI.Realtime.RealtimeSession.SendAsync[T] (T event, System.Action`1[T] sessionEvents, System.Threading.CancellationToken cancellationToken) (at ./Library/PackageCache/[email protected]/Runtime/Realtime/RealtimeSession.cs:293)
OpenAI.Realtime.RealtimeSession.SendAsync[T] (T event, System.Threading.CancellationToken cancellationToken) (at ./Library/PackageCache/[email protected]/Runtime/Realtime/RealtimeSession.cs:224)
OpenAI.Realtime.RealtimeEndpoint.CreateSessionAsync (OpenAI.Realtime.Options options, System.Threading.CancellationToken cancellationToken) (at ./Library/PackageCache/[email protected]/Runtime/Realtime/RealtimeEndpoint.cs:52)

It looks like you didn't include the possibility to disable Server_VAD in the VoiceActivityDetectionSettings class.

[StephenHodgson]

Thanks Tim for the kind words. I'll look into this and make sure it's working correctly. They updated the API schema this week so maybe related.