Allow custom to act as a Fallback Backend

[bushrat011899]

Objective

As highlighted in #671, there is room for ergonomic improvements in how getrandom handles unsupported platforms. Since getrandom already has a backend which supports external integration, custom, it would be beneficial to end users to have a way to allow 3rd party dependencies to provide a getrandom backend without modifying RUSTFLAGS.

Solution

• Added a new feature, first-party-backends-only, which will throw a compiler error if a first party backend is not available for the current target (either due to misconfiguration or lack of support). This is sem-ver compatible with a patch release.
• Enabled first-party-backends-only by default to preserve the current behaviour, where an unsupported configuration will fail to compile. This is not required for sem-ver compatibility since it would at worst allow compilation where it was already failing. This is done as a conservative guard against a hypothetical supply chain attack.
• Adjusted the backends.rs cfg_if statement to fall back to custom if no other option is available. Note that custom is still included as the first branch to preserve current behaviour.

Intended Use

This fallback would allow HAL-style crates to include a dependency as a way to provide a fallback backend for getrandom. Since it relies on extern linking, defining multiple fallback backends will cause a compile-time error, as will failing to provide one. Consider the contentious wasm_js backend. In order to work around getrandom's use of RUSTFLAGS to enable that backend, uuid just vendored the backend. This is highly undesirable as an outcome, since there is now an increased risk of uuid containing a bug or vulnerability that is fixed in getrandom but not in their copy.

With this fallback functionality, now rust-random or a 3rd party could publish the wasm_js backend for getrandom as a dedicated crate, allowing users or libraries to activate it as they require. This may be a desirable path forward for more experimental backends anyway, as it would allow iteration without changes to getrandom directly.

---

Notes

• Crates will need to be updated to disable default features on getrandom to allow this approach to work, but I don't see many difficulties here since it would be in service of improved ergonomics for their consumers.
• I think there's room to improve the custom backend to allow overriding the u32 and u64 methods, and potentially providing error messages back to the user, but that's a larger breaking change that isn't required for the ergonomic benefits discussed here.

[newpavlov]

This looks like a rehash of the custom backend which we had in getrandom v0.2. We intentionally removed it in v0.3. Please read the previous discussion which led to it.

[bushrat011899]

This looks like a rehash of the custom backend which we had in getrandom v0.2. We intentionally removed it in v0.3. Please read the previous discussion which led to it.

I have read through that issue, and I see this as distinct in that the current custom behaviour is preserved, where it will be chosen over first-party backends when getrandom_backend = "custom", but can be used as a fallback without configuration by end-users without RUSTFLAGS configuration. There is no change in how the custom backend is implemented, it just allows use in the fallback position instead of becoming a compiler error.

Is there a specific concern with this approach beyond its similarity to previous versions of this crate?

[newpavlov]

Firstly, this feature most certainly should not be enabled by default (it would be practically impossible to disable it). Next, there is a danger of users enabling this feature unconditionally in library crates (e.g. for testing on Web WASM), which could cause cryptic linking errors for downstream users. Finally, enabling an opt-in backend in getrandom would still expose an extern function from a custom implementation crate (e.g. imagine library author using a wasm-bindgen-based custom with downstream users also enabling getrandom_backend="wasm_js"). This is quite possible in practice since IIUC you plan to use it as an alternative for the wasm_js opt-in backend, which also would mean additional confusion for downstream users and ecosystem split between those who use a "custom" crate and those who use rely on the cfg-based opt-in backend.

I guess it could work as one potential option for experimentation. I am interested in hearing @josephlr's opinion on it.

As a minor bikeshedding, I think the feature should be named custom-fallback.

[bushrat011899]

Firstly, this feature most certainly should not be enabled by default (it would be practically impossible to disable it).

Assuming by "this feature" you mean support for a fallback implementation, then agreed. This PR explicitly has the fallback disabled by default using a default feature first-party-backends-only. With this feature enabled (which is the default), this PR changes nothing in getrandom.

Next, there is a danger of users enabling this feature unconditionally in library crates (e.g. for testing on Web WASM), which could cause cryptic linking errors for downstream users.

Unlike enabling a feature on getrandom (which is easy to do), this requires explicitly adding either your own implementation or an external dependency. Many libraries already handle this correctly (e.g., Bevy) by using a web or js feature. Regardless, this is a concern for other crates in my opinion, not getrandom. For example, nothing stops a library linking two different versions of llvm-sys, which will also cause a linking compiler error.

I appreciate that Rust does have a problem with conflating Wasm with Web Browser, but I don't believe the current RUSTFLAGS approach will lead to a solution in Cargo or rustc. For example, getrandom still has an std feature which is just as destructive as a js feature to compatibility, and even more pervasive. I believe the language team is thoroughly motivated to solve this problem already.

Finally, enabling an opt-in backend in getrandom would still expose an extern function from a custom implementation crate (e.g. imagine library author using a wasm-bindgen-based custom with downstream users also enabling getrandom_backend="wasm_js"). This is quite possible in practice since IIUC you plan to use it as an alternative for the wasm_js opt-in backend, which also would mean additional confusion for downstream users and ecosystem split between those who use a "custom" crate and those who use rely on the cfg-based opt-in backend.

This would be pretty easy to resolve for the wasm_js case by simply adding not(getrandom_backend="wasm_js") within the hypothetical 3rd party Wasm on the browser backend. And crucially would be controlled by the end user via the existing RUSTFLAGS method. If the user adds getrandom_backend="wasm_js", then they will always get the wasm_js backend provided by getrandom, no matter what a library crate does. If they don't add it (or forget to), then a library can provide a fallback backend and allow compilation to succeed.

I guess it could work as one potential option for experimentation. I am interested in hearing @josephlr's opinion on it.

I appreciate you being open to discussion here. I know this is a very important crate you're protecting, and I do want to make it clear that I appreciate you engaging in the discussion.

As a minor bikeshedding, I think the feature should be named custom-fallback.

Happy to call this a custom-fallback, but I'm not quite sure what would be renamed here, unless you're proposing adding an extra feature flag to enable the fallback?

[newpavlov]

The crate feature should be additive, i.e. enabling the crate future should enable the custom fallback. As I wrote, it will be practically impossible for you to disable the first-party-backends-only feature since it's likely that a dependency in your project's dependency tree will use default features.

This feature then should be enabled by crates which define the "custom" function. Note that it's not sufficient to just add the crate to your dependency tree because of how crates get linked, see the custom backend section docs for more information. In v0.2 we had the register_custom_getrandom! macro to help with that.

[bushrat011899]

The crate feature should be additive, i.e. enabling the crate future should enable the custom fallback. As I wrote, it will be practically impossible for you to disable the first-party-backends-only feature since it's likely that a dependency in your project's dependency tree will use default features.

Ah ok I see what you were referring to now. I agree, I've inverted the feature and renamed it to custom-fallback as requested, and made sure it is not enabled by default. To provide an escape hatch for binaries concerned with a custom fallback being implemented when they do not want one, I have added an extra RUSTFLAG, getrandom_no_fallback. Since it is a RUSTFLAG it is not held to the additive standard that features are, and provides a way for binaries to assert that only their explicitly chosen backend is used.

This feature then should be enabled by crates which define the "custom" function. Note that it's not sufficient to just add the crate to your dependency tree because of how crates get linked, see the custom backend section docs for more information. In v0.2 we had the register_custom_getrandom! macro to help with that.

I believe it actually is sufficient to just include the relevant crate in your dependency graph by explicitly declaring the implementing function as extern "Rust" fn rather than just using #[no_mangle] on its own. See this documentation from critical-section. I could be wrong, but if that's required in end-user code I consider that acceptable, since they will likely have a js/web feature they are enabling anyway.