Some help in correctly using the audio player needed (troubleshooting crashes, skeleton code?)

[mbariola]

Hi,

I understand that this may be too much to ask, but I have been trying in the past months to achieve the following

I am experimenting with a large sized esp32-based physical panel to control many things in the house.
This thing has 6 different capacitive buttons. It gives feedback to the user via leds, but also via a BT speaker.
Every time the user touches a capacitive button, a short blink and ideally, a small audio file (like a click sound) is played. There are 3 or 4 different small samples depending on the action and context.

Ideally, the small audio samples (mp3 encoded) were to be saved in SPIFFS, and I had found some contributions showing how to use SPIFFS, and merged it with some other examples from your repos on how to connect to the BT speaker and play them. I think I did not do a great job of it (one was using streams, the other the "old" method of managing the files), but it mostly worked.
Then I wanted to integrate the audio player, as it seemed the easiest way to play the small samples.
Unfortunately, I was not able to figure out whether something was already playing, and in general every time I played a second sample even though the first one was done playing, I'd get a guru meditation.

That was a few months ago, now I am giving it a try again and am trying to piece it all together

I understand that the problem is fully in my code. but I would really appreciate it if you could help me with a skeleton code
to properly connect to a BT speaker and make sure that the connection stays on
to understand better how the audio player works specifically to make sure that I can safely play more than one file, switch between files, starting / stopping one .... knowing the current status of playing
SPIFFS would be nice but I understand that it will probably not be supported, my esp32 hs no SD Card, but I am happy to provide the raw audio data in a big array or similar (even though I am at a dearth of memory so I am using mp3).

I was able to play the test sine wave via AD2P so the whole hardware setup seems to work.

Again, apologies if this is a lot to ask. I am just getting a bit lost with all the moving parts and the slowness of my debug loop. Maybe I should avoid the audioplayer and just figure out how to play a specific sample based on needs? yet, how to properly detect if some other sample is currently playing?

I'm kind of getting a little lost in the treasure trove of all the many ways of achieving this. Thanks for any direction

[pschatzmann]

The fact that you want to use A2DP should drive the architecture: If you look at this example you can see that A2DP is constantly pulling data, so for your project you would return the content of your file piece by piece until it has finished. If there is no file to play you would just return an empty array, to keep A2DP open.

This way your buttons would just open and set the actual file to be played.

I think this is the most efficient way to deal with this, but so far, you will need to forget about mp3 since you need to provide the data in the format that a2dp needs. I would need to extend my api to support directly reading from an encoded file...

by the way: files are also streams so you can also read them with readBytes(uint_t*data, size_t len) like in the example.

[mbariola]

@pschatzmann thanks for the clarifications. I guess I was overcomplicating it.
So, to recap & check if I understood correctly

• I cannot play an mp3 (yet) via a2dp unless I find a way to convert it on the fly, so I have to stick to wav
• I could adapt get_sound_data like this, I seem to understand that result_len is the binary array with the data?

int32_t get_sound_data(Frame* frames, int32_t count) {
    uint8_t *data = (uint8_t*)frames;
    int channels = 1; mono wav, ideally 16bits
    size_t len = count * channels * sizeof(int16_t);
   // I open my wav file on spiffs and get a wav file handle wfh
   // if a "stop" or "pause" input is received, I eventually close the file handle & return 0 (to be interpreted as an empty array)
   // how to implement a "pause" method though? it's not strictly needed in this case, just curious
   // else, I read from wfh
    size_t result_len = wfh.readBytes(data, len);   
    return result_len;
}

• And that's it. the BT connection should stay up for as long as the ESP32 runs

Did I get that right?

[pschatzmann]

I suggest that you try

int32_t get_sound_data(Frame* frames, int32_t count) {
    uint8_t *data = (uint8_t*)frames;
    int framesize = 4;
    size_t result_len = wfh.readBytes(data, count*framesize) / framesize;   
    if (result_len == 0){
       // if no more data, just return silence
       memset(data, 0, count*framesize);
       result_len = count;
    }
    return result_len;
}

And dont't forget: if you open a wav file first read 44 bytes which are the header which must be ignored.

[mbariola]

@pschatzmann , I didn't know about the 44 bytes thing. To skip them, I guess something like this might work? (ugly as it's brittle and uses a global var, but can't think of anything better right now)

bool skipHeader = true;
int32_t get_sound_data(Frame* frames, int32_t count) {
    if(skipHeader) {
        wfh.readBytes(data, 44); // skip the header bytes
        skipHeader = false;
    }
    uint8_t *data = (uint8_t*)frames;
    int framesize = 4;
    size_t result_len = wfh.readBytes(data, count*framesize) / framesize;   
    if (result_len == 0){
       // if no more data, just return silence
       memset(data, 0, count*framesize);
       result_len = count; // should it be count*framesize?
       skipHeader = true; // for next wav
    }
    return result_len;
}

[pschatzmann]

You can store the file directly as binary file, then you would not need to do this trick.
Please note A2DP needs 16bits, 2 channels, 44100 samples/second

[mbariola]

@pschatzmann I did some tests , and this is the best I could come up with

/**
 * @file adc-a2dp.ino
 * @author Phil Schatzmann
 * @brief see https://github.com/pschatzmann/arduino-audio-tools/blob/main/examples/adc-a2dp/README.md
 *
 * @author Phil Schatzmann
 * @copyright GPLv3
 *
 */

#include <Arduino.h>
#include <BluetoothA2DPSource.h>
#include <AudioTools.h>
#include <SPIFFS.h>

BluetoothA2DPSource a2dp_source;

File wfh;

unsigned long lastrun = 0;

bool skipHeader = true;

// Initialize SPIFFS
uint initSPIFFS()
{
    uint spiffsMounted = SPIFFS.begin(true);
    if (!spiffsMounted)
    {
        Serial.println("An error has occurred while mounting SPIFFS");
    }
    Serial.println("SPIFFS mounted successfully");
    return spiffsMounted;
}

int32_t get_sound_data(Frame *frames, int32_t count)
{
    Serial.println("In get_sound_data");
    char *data = (char *)frames;
    if (skipHeader)
    {
        Serial.println("Skipping header");
        wfh.readBytes(data, 44); // skip the header bytes
        skipHeader = false;
    }
    else
    {
        Serial.println("Skipped header");
    }
    
    int framesize = 4;
    size_t result_len = wfh.readBytes(data, count * framesize) / framesize;
    Serial.println(result_len);
    if (result_len == 0)
    {
        // if no more data, just return silence
        memset(data, 0, count * framesize);
        result_len = count * framesize; // should it be count*framesize?
        skipHeader = true;  // for next wav
    }
    return result_len;
}
// Arduino Setup
void setup(void)
{
    // Serial Init
    Serial.begin(460800);
    initSPIFFS();
    delay(150);
    // start the bluetooth
    Serial.println("starting A2DP...");
    wfh = SPIFFS.open("/01.wav", FILE_READ);
    a2dp_source.start("Wuzhi Audio", get_sound_data);
    lastrun = millis();
}

// Arduino loop - repeated processing
void loop()
{
    if(millis() -lastrun > 10000)
    ESP.restart();
}

to do a test, I just play a compatible wav file that is loaded into spiffs. I had to change the declaration of data from uint8_t* to char* as I was getting a compile error.

However, it seems that there is some problem with connecting to the speaker.

this is what I get on the serial monitor

starting A2DP...
20:55:43.239 > [ 840][W][BluetoothA2DPSource.cpp:558] bt_av_hdl_stack_evt(): [BT_AV] Reconnecting to b8:07:8b:3d:70:0f
20:55:43.239 > [ 841][W][BluetoothA2DPCommon.cpp:41] connect_to(): [BT_AV] connect_to to b8:07:8b:3d:70:0f
20:55:43.340 > [ 944][E][BluetoothA2DPSource.cpp:723] bt_app_av_state_connecting(): [BT_AV] bt_app_av_state_connecting unhandled evt 4

your other sketch - playing a sine wave over the speaker - works well, on the same speaker and from the same esp32. this is the working sketch

typedef int16_t sound_t; // sound will be represented as int16_t (with 2 bytes)
uint16_t sample_rate = 44100;
uint8_t channels = 2;                       // The stream will have 2 channels
SineWaveGenerator<sound_t> sineWave(32000); // subclass of SoundGenerator with max amplitude of 32000
GeneratedSoundStream<sound_t> in(sineWave); // Stream generated from sine wave
A2DPStream out = A2DPStream::instance();    // A2DP input - A2DPStream is a singleton!
StreamCopy copier(out, in);                 // copy in to out

// Arduino Setup
void setup(void)
{
  Serial.begin(460800);
  AudioLogger::instance().begin(Serial, AudioLogger::Warning);

  // set the frequency
  sineWave.setFrequency(N_B4);

  // Setup sine wave
  auto cfg = in.defaultConfig();
  cfg.channels = channels;
  cfg.sample_rate = sample_rate;
  in.setNotifyAudioChange(out);
  in.begin(cfg);

  // We send the test signal via A2DP - so we conect to the MyMusic Bluetooth Speaker
  auto cfgA2DP = out.defaultConfig(TX_MODE);
  cfgA2DP.name = "Wuzhi Audio";
  // cfgA2DP.auto_reconnect = false;
  out.begin(cfgA2DP);
  out.setVolume(0.3);
  Serial.println("A2DP is connected now...");
}

// Arduino loop
void loop()
{
  copier.copy();
}

Thanks for your time in pointing me in the right direction, I understand time is precious, so let me know when too much is too much. Maybe if you could point me to how to change the second sketch to accept a wav file, everything might fall into place?
Best

[pschatzmann]

My gut feeling is that your problems come from print statements in get_sound_data: This just uses too much time.
And remove the skipHeader logic as well.
You would just do the following in the setup:
wfh = SPIFFS.open("/01.wav", FILE_READ);
wfh.readBytes(data, 44); // skip the header bytes

[mbariola]

Just tried that, error is the same as above.

New sketch -

#include <Arduino.h>
#include <BluetoothA2DPSource.h>
#include <AudioTools.h>
#include <SPIFFS.h>

BluetoothA2DPSource a2dp_source;

File wfh;

unsigned long lastrun = 0;



// Initialize SPIFFS
uint initSPIFFS()
{
    uint spiffsMounted = SPIFFS.begin(true);
    if (!spiffsMounted)
    {
        Serial.println("An error has occurred while mounting SPIFFS");
    }
    Serial.println("SPIFFS mounted successfully");
    return spiffsMounted;
}

int32_t get_sound_data(Frame *frames, int32_t count)
{
    char *data = (char *)frames;
    int framesize = 4;
    size_t result_len = wfh.readBytes(data, count * framesize) / framesize;
    Serial.println(result_len);
    if (result_len == 0)
    {
        // if no more data, just return silence
        memset(data, 0, count * framesize);
        result_len = count * framesize; // should it be count*framesize?
        skipHeader = true;  // for next wav
    }
    return result_len;
}
// Arduino Setup
void setup(void)
{
    // Serial Init
    Serial.begin(460800);
    initSPIFFS();
    delay(150);
    // start the bluetooth
    Serial.println("starting A2DP...");
    wfh = SPIFFS.open("/01.wav", FILE_READ);
    char hdr[44];
    wfh.readBytes(hdr, 44); // skip the header bytes
    a2dp_source.start("Wuzhi Audio", get_sound_data);
    lastrun = millis();
}

// Arduino loop - repeated processing
void loop()
{
    if(millis() -lastrun > 10000)
    ESP.restart();
}

[pschatzmann]

not sure if this will help, but try

a2dp_source.auto_reconnect(false);
a2dp_source.start("Wuzhi Audio", get_sound_data);

And dont forget to remove the Serial.println in get_sound_data

[mbariola]

Hi @pschatzmann , I tried your suggestion but still no luck. I have been ill in the past days, as soon as I feel better, I'll try to work from the sine generator example and will probably just generate different notes depending on the type of touch, then try to grow from there and see if I can make it play proper sounds. thanks for your help thus far!