Directional audio issue in games

[RondoRevolution]

I've found this repo and the reddit post a week ago and have been using this config happily for music and YouTube, but recently I decided to game with it, and noticed that whenever I have the Virtual Surround sink selected, some sounds have issues in the two games I tried that are not present when I open it with the Headphone Correction Sink.

More precisely, in Gunfire Reborn character sounds only come from the left channel/ear, it also happens with the initial gun. The only exception is the character Tao, where her character sounds come from both left and right, but duplicated. Skills and other guns sounds seems to be coming correctly.

In Risk of Rain 2 something similar happens. In the character select screen, a lot of characters sound like coming from the left. Without the Virtual Surround sink selected, they sound slightly to the right since the characters are off-center in the screen (tho there are characters where their select sound is centered). But in-game, pod arrival sound, gun sounds, character sounds and skills are all coming only from the left channel/ear.

Here are my config files:

99-eq.conf

  context.modules = [
{   name = libpipewire-module-parametric-equalizer
    flags = [ nofail ]
    args = {
        node.description = "Headphone Correction Sink"
        media.name       = "Headphone Correction Sink"
        equalizer.filepath = "/home/jojo/.local/share/pipewire/eq/kz-crn.txt"
        audio.channels = 2
        audio.position = [ FL FR ]
            capture.props = {
                node.name   = "effect_input.eq6"
                media.class = Audio/Sink
            }
            playback.props = {
                node.name   = "effect_output.eq6"
                node.passive = true
                target.object = alsa_output.pci-0000_0c_00.4.analog-stereo
            }
        }
    }
]

99-virtual-surround.conf

# Headphone surround sink
#
# Copy this file into a conf.d/ directory such as
# ~/.config/pipewire/filter-chain.conf.d/
#
# Adjust the paths to the sofa file to match your system.
#
context.modules = [
    { name = libpipewire-module-filter-chain
        flags = [ nofail ]
        args = {
            node.description = "Virtual Surround"
            media.name       = "Virtual Surround"
            filter.graph = {
                nodes = [
                    {
                        type = sofa
                        label = spatializer
                        name = spFL
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 30.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spFR
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 330.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spFC
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 0.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spRL
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 150.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spRR
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 210.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spSL
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 90.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spSR
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 270.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spLFE
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"    = 180.0
                            "Elevation"  = 0.0
                            "Radius"     = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spTFL
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"   = 30.0
                            "Elevation" = 75.0
                            "Radius"    = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spTFR
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"   = 330.0
                            "Elevation" = 75.0
                            "Radius"    = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        name = spTRL
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        control = {
                            "Azimuth"   = 150.0
                            "Elevation" = 75.0
                            "Radius"    = 75.0
                        }
                    }
                    {
                        type = sofa
                        label = spatializer
                        config = {
                            filename = "/home/jojo/.local/share/pipewire/hrir/EAC_48kHz.sofa"
                        }
                        name = spTRR
                        control= {
                            "Azimuth"   = 210.0
                            "Elevation" = 75.0
                            "Radius"    = 75.0
                        }
                    }
                    { type = builtin label = mixer name = mixTL }
                    { type = builtin label = mixer name = mixTR }
                    { type = builtin label = mixer name = mixBL }
                    { type = builtin label = mixer name = mixBR }
                    { type = builtin label = mixer name = mixL }
                    { type = builtin label = mixer name = mixR }
                ]
                links = [
                    { output = "spFL:Out L"  input="mixBL:In 1"  }
                    { output = "spFL:Out R"  input="mixBR:In 1"  }
                    { output = "spFR:Out L"  input="mixBL:In 2"  }
                    { output = "spFR:Out R"  input="mixBR:In 2"  }
                    { output = "spFC:Out L"  input="mixBL:In 3"  }
                    { output = "spFC:Out R"  input="mixBR:In 3"  }
                    { output = "spRL:Out L"  input="mixBL:In 4"  }
                    { output = "spRL:Out R"  input="mixBR:In 4"  }
                    { output = "spRR:Out L"  input="mixBL:In 5"  }
                    { output = "spRR:Out R"  input="mixBR:In 5"  }
                    { output = "spSL:Out L"  input="mixBL:In 6"  }
                    { output = "spSL:Out R"  input="mixBR:In 6"  }
                    { output = "spSR:Out L"  input="mixBL:In 7"  }
                    { output = "spSR:Out R"  input="mixBR:In 7"  }
                    { output = "spLFE:Out L" input="mixBL:In 8"  }
                    { output = "spLFE:Out R" input="mixBR:In 8"  }
                    { output = "spTFL:Out L" input = "mixTL:In 1" }
                    { output = "spTFL:Out R" input = "mixTR:In 1" }
                    { output = "spTFR:Out L" input = "mixTL:In 2" }
                    { output = "spTFR:Out R" input = "mixTR:In 2" }
                    { output = "spTRL:Out L" input = "mixTL:In 3" }
                    { output = "spTRL:Out R" input = "mixTR:In 3" }
                    { output = "spTRR:Out L" input = "mixTL:In 4" }
                    { output = "spTRR:Out R" input = "mixTR:In 4" }
                    { output = "mixBL:Out" input = "mixL:In 1" }
                    { output = "mixTL:Out" input = "mixL:In 2" }
                    { output = "mixBR:Out" input = "mixR:In 1" }
                    { output = "mixTR:Out" input = "mixR:In 2" }

                ]
                inputs  = [ "spFL:In" "spFR:In" "spFC:In" "spLFE:In" "spRL:In" "spRR:In" "spSL:In" "spSR:In" "spTFL:In" "spTFR:In" "spTRL:In" "spTRR:In"  ]
                outputs = [ "mixL:Out" "mixR:Out" ]
            }
            capture.props = {
                node.name      = "effect_input.spatializer"
                media.class    = Audio/Sink
                audio.channels = 12
                audio.position = [ FL FR FC LFE RL RR SL SR TFL TFR TRL TRR ]
            }
            playback.props = {
                node.name      = "effect_output.spatializer"
                node.passive   = true
                audio.channels = 2
                audio.position = [ FL FR ]
                target.object  = effect_input.eq6
            }
        }
    }
]

[RondoRevolution]

After some testing, it is definitely the config, I tested with a common 7.1 config and it works just fine. Listening closely to the 7.1.4 config, it looks like these games might be assigning some of the sounds to height channels on the left, at least it's what it sounds like in Risk of Rain 2.

I have also tested the 7.1.4 config on Borderlands 2, Aimlabs, Valheim and Halo MCC since the first 2 I tested were both Unity games and I thought it could have been an issue with Unity games specifically, but Aimlabs and Valheim both works fine from the little I tested. Borderlands 2 also seemed to work fine. Halo actually won't output sound to the 7.1.4 config and crashes with a Fatal Error. It works with a standard 7.1 config.

From my testing, it looks like this config is not suited for general gaming and might be instead specifically for Dolby Atmos games, unless there is something that can change in it to make it work properly on these games I had issues.

[DekoDX]

I will investigate further, I've actually started to notice this myself. Thank you.

[RondoRevolution]

Not sure if this is relevant here, and I haven't tested it yet, but it looks like there is some changes in the 2 most recent versions of GE-Proton that addresses spatial audio. Here's the release notes for 10-28.

It does mention height channels, but I'm not sure if the correct way would be to set WINEALSA_CHANNELS=8 or WINEALSA_CHANNELS=12 since it mentions height channels are ignored from the downmixing.

[DekoDX]

Not sure if this is relevant here, and I haven't tested it yet, but it looks like there is some changes in the 2 most recent versions of GE-Proton that addresses spatial audio. Here's the release notes for 10-28.

It does mention height channels, but I'm not sure if the correct way would be to set WINEALSA_CHANNELS=8 or WINEALSA_CHANNELS=12 since it mentions height channels are ignored from the downmixing.

Unfortunately according to the patch author, winealsa has some bugs with anything higher than WINEALSA_CHANNELS=8

Since the channels in wine were the main use case I will provide an alternative 7.1 sink without replacing the current one. I highly encourage raising the issue with upstream(especially if I don't get around to it soon).

[DekoDX]

Worth noting that if you want to restore the old behavior for games that do work. You should be able to set the audio driver back to winepulse with protontricks.

[RondoRevolution]

I see. From my testing, it does fix issues in the games I mentioned before, but I'll keep using the 7.1 config I had setup before for the time being since it's easier than having to test the config for each game.

EDIT: Actually, I could easily add the env vars to my environment instead of reverting back to the 7.1 config, but I'm not sure that's the best way to go about this nor if the issues I reported will still appear when playing Linux native games.

[DekoDX]

Since it appears to be an issue in upstream wine, it's not likely I can fix this on my end.

However I'm not going to close this issue until things are resolved on that end.

[RondoRevolution]

Hey just an update, I have actually been using the 7.1.4 config full time on my machine, and fixing the issues I find with WINEDLLOVERRIDES="winepulse.drv=d" WINEALSA_CHANNELS=12 %command%, just recently found the no audio issue on Sunset Overdrive and fixed it by adding the variable to the game's launch option on Steam. The downside is that it drops the height channels, the 12 I put there doesn't actually work and it is just capped at the max allowed (8 iirc).

But it looks like GE-Proton-10-29 just launched with a PR merged that makes top channels work with the winealsa driver, tho I haven't tested it yet.

[Vyrolian]

Hey! Noticed a mention of my PR. Some explanation - WINEALSA_CHANNELS has to be amount of physical speakers you have, it goes up to 8 (more on that later). Spatial stream decides by itself how many channels to open - it's usually 12 in spatial games. You can see it in pw-top. So with the WINEALSA_SPATIAL=1 flag it downmixes internally in the driver from all the channels the game provided to however many you set in WINEALSA_CHANNELS. So in qjackctl you will notice if you select both spatial and channels variables that the channels that aren't on your hardware are dead. It's because, for instance, with 2 channels selected, i take all 12 from the game and downmix them accordingly to left and right, and disable the rest (there is absolutely no point in keeping them).

This is not pretty, but alsa pulse plugin (yes, even with pipewire it uses alsa pulse plugin because it uses steam runtime libraries) can only properly see up to 8 channels (7.1 surround stream), after that it puts them in AUX channels and they don't work at all for some reason. So there is no point in allowing more than 8 in winealsa channels, and anything above 8 defaults to 2 if selected. More than that, with winealsa and any stream that has more channels than your hardware my downmixer is always required because the alsa pulse plugin doesn't communicate with pipewire properly so there is no downmixing formula applied.

When/if (I dont know whether steamrt 4 uses pipewire) steamrt starts shipping pipewire, I am certain I can make the downmixer work properly in pipewire instead. The default pipewire downmixer does not have top channels however, so unless you do some shenanigans with manual downmixing you can't have top channels.

Addressing the initial issue on the topic - downmixer in both pipewire and the winealsa patch is done correctly, so any issues with games not sounding right are related to spatial translation in wine, which is not perfect.

[RondoRevolution]

@Vyrolian So if I'm using iems with the virtual surround config here, I should still only setup WINEALSA_CHANNELS to 2 since physically it only has 2 channels, which is already happening since I put 12 there as it doesn't support that, right? And then I have to also set WINEALSA_SPATIAL=1 so the height channels are taken into consideration per you PR? Tho wine's spatial translation is not perfect and might be prone to some sort of positioning error or something like that?

[Vyrolian]

I honestly am not very sure what this repo does, creates a virtual sink so the game thinks you have 12 channels and opens a spatial stream?

The winealsa downmixer is for the games that:

1. Create spatial stream by default
2. Refuse to work without spatial audio at all

As an example - GTA V Enhanced. Try launching it with only winealsa and WINEALSA_CHANNELS set to something. Then WINEALSA_CHANNELS will physically block creating a stream with more channels than selected. This is useful for games like Forza Horizon. They create a spatial stream by default, but if you hardcode with a cap with WINEALSA_CHANNELS, it will fallback to a channel-based stream, so you don't need a downmixer at all. WINEALSA_SPATIAL variable lets you create the stream with more channels than WINEALSA_CHANNELS, but it will then downmix so it sounds right.

So if the game is spatial by default you can use winealsa downmixer, but if the game is not spatial by default, you can't force so many channels that it becomes a spatial stream if you use winealsa.

[RondoRevolution]

I honestly am not very sure what this repo does, creates a virtual sink so the game thinks you have 12 channels and opens a spatial stream?

Not exactly. Its a Pipewire config to create a virtual sink for a virtual surround setup to enhance the audio experience by simulating an actual surround setup. It is still 2 channels so not real surround, but Pipewire handles the downmixing with the provided HRTF sofa file.

To give an example of how this enhances audio in games: Do you know when you have positional audio in a game, and you can clearly tell where the sound is coming from, but it only really plays from one side? For example, there's a gun shot or footsteps coming from your back left and you can clearly hear it coming from that, but because the sound in game doesn't propagate it is isolated in that specific position, unlike real life where that sound would also hit you from different angles at different times at different loudness, etc? That's basically what this does, simulate a proper spatial sound via the chosen HRTF file.

It can also enhance the audio experience on basically everything else if you choose an HRTF file that more closely matches your own or that you think sound good, that's why I use it full time on my PC, including for music, movies, youtube, etc.

But I could be explaining this kind of wrong since I'm not very knowledgeable on audio in general.

Pipewire have documentation on this here with provided configuration file examples. The main difference between the files in that link and the ones in this repo, is that the config here have the additional 4 height channels.

[Vyrolian]

Understood. Winealsa downmixer is admittedly hacky and the downmixing should ideally happen in pipewire. Unfortunately with steam runtime constraints this is as good as it can get for now. I made it so winealsa is working properly in every game, in case you want to use it if winepulse crackles. I have also made some fixes for winepulse to eliminate crackling, though they are admittedly a bit hacky (if the idea is to match Windows audio driver behavior), however it doesn't actually cause any issues. I initially put it in the PR together with the downmixer changes, but the fix required some more work.