Share AssetSources between the AssetServer, the asset processor, and the asset processor's internal asset server.

[andriyDev]

Objective

• Previously, we would build the sources up to 3 times with different options: once for the regular asset server, once for the asset processor, and once for the asset processor's internal asset server.
• This is a step towards Allow adding and removing asset sources at runtime. #21758 (since now adding a source to this shared AssetSources will be reflected in all the uses.

Solution

1. Skip all the hot-reloading polling if watch_for_changes is false. If we don't do this, sharing the sources between the asset server and the processor-internal asset server will result in two tasks consuming asset events, so the regular asset server will miss asset events.
2. Move the state of the asset processor into a separate struct that we then Arc. We need to be able to gate the processed asset reader, but we can't create an asset processor without the sources. So instead we allow ourselves to create the state first, so that we can gate on that state, and then create the processor with that state.
3. Split the processed reader into a gated and an ungated form. The gate first blocks on the processor being done initialized, followed by gating on the per-asset lock. However the asset processor needs to iterate through all the directories in order to finish initializing. So just unconditionally gating doesn't work - we hit a deadlock. So we provide access to the ungated processed reader, so that the processor can use that to initialize its state.
4. Finally do the sharing!

One thing I'm starting in this PR is making things more private. For example, it's not clear why ProcessorGatedReader was pub. I've also made AssetSources::gate_on_processor no longer pub. This does mean that users can no longer initialize their own AssetServer properly (e.g., gated on the processor), but I don't think we should really support this - our focus should be on the AssetServer initialized by AssetPlugin, not hypothetical uses where someone wants to insert their own AssetServer.

Testing

• CI

[github-actions]

It looks like your PR is a breaking change, but you didn't provide a migration guide.

Please review the instructions for writing migration guides, then expand or revise the content in the migration guides directory to reflect your changes.

[andriyDev]

I added a migration guide, though I think in a future PR I'm just gonna make these private. I am very skeptical that users are initializing these manually and I think the fact that these are public is more a mistake than a desired feature. There would need to be a really compelling reason to support this and we should probably have a more appropriate "front door" if that's the case.

[kristoff3r]

LGTM, but I haven't tested it

[kristoff3r]

Shouldn't the AssetSourceBuilder functions generate Arcs instead of always converting the Box they make?

[andriyDev]

They could do. I didn't want to change too much since it's not fully necessary, and also returning a box means we own the reader rather than being given a cloned arc they got from somewhere else. I'd say that's malicious, but it lines up!

In the future, we might refactor this even more since I think the builder API mostly only made sense with AssetSourceBuilders which I'm getting rid of in the next PR.

[greeble-dev]

Clicking approve as I couldn't spot any issues and the tests pass for me. Only nitpick is that the PR's explanation for ungated_processed_reader should also be a comment in the code.

[andriyDev]

I included a short blurb about it. I initially didn't know where to put the comment, but I think a comment on the field itself is sufficient. It should prevent people from removing it at least without understanding the intent.