Multichannel mp3 player using libhelix

[Son0f4Glitch]

Hi all! I'd like to use libhelix to build a 4 channel mp3 player that will run on an esp32 WROOM 32E. So far I've been trying to add other source channels without success. Has anyone attempted this before? Can anyone point me to the right direction?
Cheers! SOAG =)

[pschatzmann]

I haven't looked into the number of supported tracks for different audio file formats and I am not sure if mp3 is supporting more then 2 tracks.

Anyhow, the ESP32 is very limited and decoding 2 mp3 tracks works just fine - but I have the gut feeling that more will be difficult. Using an uncompressed audio file format e.g. WAV will not have any of this limitations, because we can just copy data.

On the other side I do not see how to output the 4 tracks properly: did you plan to use the 2 I2S ports ?

[Son0f4Glitch]

So far I'm using the analog stereo output, do you see any issues there?
As of now I'm able to load multiple tracks properly, and play them back one at a time. How would you go about playing them simultaneously, as copier won't accept multiple sources as an argument?

[pschatzmann]

I think 4 channels and mp3 will never work! You can't use 1 decoder for multiple inputs because - as far as I know - the decoder relies on the information from the prior segments: so you can't feed it from multiple sources and for 2 decoders there are not enough resources. So the major issue is that mp3 supports max 2 channels.

Here are my recommendations:

• Use a file format which supports 4 channels. I suggest that you start with WAV. There might not be enough space on the microcontroller, so you might need to store the file on a SD drive or somewhere on the internet. PCM WAV is very efficient to process, since we just need to copy data! If you have the WAV working you can consider to use e.g. AAC which should support many channels.
• After decoding you need to reduce the 4 channels to 2 before so that you can output them. This can be done e.g. with a ChannelFormatConverterStream: The current implementation will take input channel 1 to ouput channel 1 and input channels 2-4 in to output channel 2. You might enhance the logic: any contribution from you side will be appreciated. In the initial implementation I had the conversion between 1 and 2 channels in mind!

[Son0f4Glitch]

Unfortunately, using WAV is not an option :/
I managed to get two tracks to play at the same time, but with serious audio glitches and regular interruptions of the audio signal. To achieve that, I run the two files through the same processes in parallel. Everything is fine until I "copy" to two copiers simultaneously (copier1.copy(); copier2.copy()). Once again, I'd like to sum the two audio files before copying them to the output, but I cannot figure out how to do it.

`#include "AudioTools.h"
#include "AudioCodecs/CodecMP3Helix.h"
#include "tester1Red.h"
#include "tester2Red.h"

// audio setup variables
uint16_t sampleRate = 44100;
uint8_t channels = 2;

// load the audio files
MemoryStream tester1(tester1Red_mp3, tester1Red_mp3_len); // load file audio 1
MemoryStream tester2(tester2Red_mp3, tester2Red_mp3_len); // load file audio 2

// setup the audio output
AnalogAudioStream analog1; // select audio engine
AnalogAudioStream analog2;

EncodedAudioStream out1(&analog1, new MP3DecoderHelix()); // route signal through the decoder (this is my transport)
EncodedAudioStream out2(&analog2, new MP3DecoderHelix());

// copy to analog output
StreamCopy copier1(out1, tester1); // copy to output
StreamCopy copier2(out2, tester2);

void setup() {

Serial.begin(115200);
AudioLogger::instance().begin(Serial, AudioLogger::Info);

// update audio info with info from decoder
out1.setNotifyAudioChange(analog1);
out2.setNotifyAudioChange(analog2);

// begin processing
auto config1 = analog1.defaultConfig();
auto config2 = analog2.defaultConfig();

// initialise output channels
analog1.begin(config1);
//analog2.begin(config2);

// start playback
out1.begin();
//out2.begin();

}

void loop() {

if (tester1 || tester2) {
copier1.copy();
copier2.copy();

} else {
auto info1 = out1.decoder().audioInfo();
auto info2 = out2.decoder().audioInfo();
LOGI("The audio rate from the mp3 file is %d", info.sample_rate);
LOGI("The channels from the mp3 file is %d", info.channels);
analog1.end(); // remove to loop the audio file
analog2.end();
stop();
}

}`

[pschatzmann]

You can't use 2 AnalogAudioStream instances. ESP32 only supports one port (which must be 0)! To combine decoded audio please read this documentation.
But I want to repeat: You are running into the wrong direction and from my point of view your approach can't work...
Start simple with a 4 channel wav and if it is working add additional complexity with a lossy codec that supports multiple channels in one file.

From my point of view you just hear the output from one file and the noise and glitches is coming from the fact that you are abusing the codec.

ps if you would use I2S you could use 2 instances, but you would still need to assign them to different ports.

[Son0f4Glitch]

Alrighty thank you very much for your support, you solved a lot of my doubts and gave me a direction to follow. I'll update you!

[TangerineDreamer]

Why not develloping a little sequencer audio player class that read several audio mp3 or wav at a time but use only one instance of audiostream as output.
In sequencer you can place audios in the time range and set volume,effects for each one and for
the mix output...But i'm not advanced enough to do it,it's only a vision of the mind :)

[Son0f4Glitch]

My idea was also to create a sort of mixer that can host multiple tracks in single stereo input channel. However that doesn't seem to be possible with MP3 - or at least I'm not skillfull enough :/ thanks for your contribution though!

[pschatzmann]

track = channel; Stereo has 2 channels, so if you have more input channels then output channels you need to start to mix/combine them

[Son0f4Glitch]

Exactly! That's why I originally tried to sum them and then input the summation to the copier. I couldn't sort it out though. I'm going now through your documentation :)

[pschatzmann]

https://github.com/pschatzmann/arduino-audio-tools/wiki/Converting-the-Number-of-Input-and-Output-Channels