Skip to content

Enable full duplex #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
sam-s10s wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Enable full duplex #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a7e3643
README
sam-s10s Jul 13, 2025
3ab0af6
Support full duplex (speak over the agent).
sam-s10s Jul 24, 2025
741a626
ignore build files
sam-s10s Jul 24, 2025
4799e6a
Merge branch 'main' into smx/esp32-s3-touch-lcd
sam-s10s Jul 24, 2025
883d18c
IDF version
sam-s10s Jul 24, 2025
cfc1442
Enabled HALF_DUPLEX by default.
sam-s10s Jul 24, 2025
5682966
Remove defunct code
sam-s10s Jul 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,6 @@ build
sdkconfig
sdkconfig.old
managed_components
.cache
.vscode
*_configuration.json
2 changes: 1 addition & 1 deletion esp32-m5stack-cores3/dependencies.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ dependencies:
idf:
source:
type: idf
version: 5.4.1
version: 5.4.2
m5stack/m5gfx:
component_hash: a44875eb6d6577bddcf1669ad5dec35e1afa425b7ca8e99af6b38505f37840e6
dependencies: []
Expand Down
8 changes: 8 additions & 0 deletions esp32-s3-box-3/src/media.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
#define OPUS_ENCODER_BITRATE 30000
#define OPUS_ENCODER_COMPLEXITY 0

#define HALF_DUPLEX // mute mic when bot speaking

#ifdef HALF_DUPLEX
std::atomic<bool> is_playing = false;
void set_is_playing(int16_t *in_buf, size_t in_samples) {
bool any_set = false;
Expand All @@ -25,6 +28,7 @@ void set_is_playing(int16_t *in_buf, size_t in_samples) {
}
is_playing = any_set;
}
#endif

esp_codec_dev_handle_t mic_codec_dev = NULL;
esp_codec_dev_handle_t spk_codec_dev = NULL;
Expand Down Expand Up @@ -65,7 +69,9 @@ void pipecat_audio_decode(uint8_t *data, size_t size) {
opus_decode(opus_decoder, data, size, decoder_buffer, PCM_BUFFER_SIZE, 0);

if (decoded_size > 0) {
#ifdef HALF_DUPLEX
set_is_playing(decoder_buffer, decoded_size);
#endif
if ((ret = esp_codec_dev_write(spk_codec_dev, decoder_buffer,
decoded_size * sizeof(uint16_t))) !=
ESP_OK) {
Expand Down Expand Up @@ -109,9 +115,11 @@ void pipecat_send_audio(PeerConnection *peer_connection) {
return;
}

#ifdef HALF_DUPLEX
if (is_playing) {
memset(read_buffer, 0, PCM_BUFFER_SIZE);
}
#endif

auto encoded_size = opus_encode(opus_encoder, (const opus_int16 *)read_buffer,
PCM_BUFFER_SIZE / sizeof(uint16_t),
Expand Down