retpoline and retpoline-external-thunk flags (target modifiers) to enable retpoline-related target features

[azhogin]

-Zretpoline and -Zretpoline-external-thunk flags are target modifiers (tracked to be equal in linked crates).

• Enables target features for -Zretpoline-external-thunk:
  +retpoline-external-thunk, +retpoline-indirect-branches, +retpoline-indirect-calls.
• Enables target features for -Zretpoline:
  +retpoline-indirect-branches, +retpoline-indirect-calls.

It corresponds to clang -mretpoline & -mretpoline-external-thunk flags.

Also this PR forbids to specify those target features manually (warning).

[rustbot]

r? @compiler-errors

rustbot has assigned @compiler-errors.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

[compiler-errors]

r? compiler

[bors]

☔ The latest upstream changes (presumably #136085) made this pull request unmergeable. Please resolve the merge conflicts.

[chenyukang]

r? compiler

[fee1-dead]

r? compiler

[lcnr]

r? @davidtwco maybe 😅

[Darksonn]

I don't think retpoline actually changes the ABI, so it might not need to be a target modifier. On the other hand, as an exploit mitigation, we do still want to avoid cases where you don't enable the mitigation in every compilation unit.

cc @andyhhp do I understand things correctly?

[azhogin]

I don't think retpoline actually changes the ABI, so it might not need to be a target modifier. On the other hand, as an exploit mitigation, we do still want to avoid cases where you don't enable the mitigation in every compilation unit.

I read your post in #116852:

I've investigated this issue in more details. It turns out that the retpoline mitigations do not affect ABI and can be mixed.

Mixing them gives up their advantage, though. Linking together two compilation units that disagree on whether to apply the mitigation may make both vulnerable to what they attempt to mitigate.

That's why I thought it should be a target modifier.

[andyhhp]

I don't think retpoline actually changes the ABI, so it might not need to be a target modifier. On the other hand, as an exploit mitigation, we do still want to avoid cases where you don't enable the mitigation in every compilation unit.

cc @andyhhp do I understand things correctly?

It's important to distinguish the general indirect-thunk transform from retpoline which is one specific transform (and is not effective on any Intel CPUs since Skylake).

retpoline is the specifically transform of an indirect call/jmp instruction into a call; int3; rewrite-ra; ret ROP gadget with the same net effect, but causes prediction to come from the RSB rather than the BTB.

The need to rewrite the return address on the stack necessitates the use of a GPR, and this is observable call/jmp-ee but any non-parameter register will do, so it doesn't impact the main function call ABI.

Even within retpoline itself, there are variations; inline retpolines are an option, but tend to be heavy on the code bloat side, which is why out-of-line retpolines in common sections are often preferred.

The alternatives to repoline are lfence;jmp (used on some Atom processors where retpoline isn't safe. Previously recommended by AMD but later rescinded), and re-inlining the original indirect branch (when hardware is believed safe, which is a contentious point because it can be made safe on AMD, but only "pretty safe" Intel.)

As to separate complication units, you can mix and match, but if any one piece of your whole process isn't taking safety precautions, then you've lost.

[azhogin]

It's important to distinguish the general indirect-thunk transform from retpoline which is one specific transform (and is not effective on any Intel CPUs since Skylake).

Could you, please, recommend the right way to represent this flag according with +retpoline-external-thunk, +retpoline-indirect-branches, +retpoline-indirect-calls target features?

Should it be -Zx86-retpoline=[fixed_variant_of_features1|fixed_variant_of_features2|...] as in harden-sls (#136597)?
Or simple features list like -Zx86-retpoline=external-thunk,indirect-calls?

[Darksonn]

I would think that the best choice would be to use the same name as clang, which is -mretpoline-external-thunk. So that would be -Zretpoline-external-thunk.

[ojeda]

Yeah, personally, I prefer when compiler drivers agree. rustc tends to diverge a fair amount, but at least if "uncommon" flags have the same names it would help a bit.

[azhogin]

• -Zretpoline-external-thunk flag enables:
  +retpoline-external-thunk, +retpoline-indirect-branches, +retpoline-indirect-calls.

• -Zretpoline flag enables:
  +retpoline-indirect-branches, +retpoline-indirect-calls

-Zretpoline-external-thunk also turns on -Zretpoline.

Looks like it corresponds to clang flags.

[rustbot]

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

[WaffleLapkin]

this API seems confusing, at the very least change the doc comment.