Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Add timestamp in the asset cache flow to enable user-driven cache invalidation for static url support #151

Open
techanon opened this issue Feb 14, 2025 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@techanon
Copy link
Contributor

Is your feature request related to a problem?

[Brain dump wall of text, sorry]

The current way asset loading is implemented is AssetBundle URL + AssetMeta URL + encryption validation (collectively "Asset Definition") are passed to remote clients in order to load an avatar asset that someone else has published. So in order to "update" an Asset Definition, the old one needs to be removed (or ignored) and the new version has to have a URL that is uniquely different from any other prior versions (currently done with a GUID).

This currently doesn't have any way to define some permanent custom URL where the asset can be updated in-place and then declare that an Asset Definition should be invalidated and re-fetched from the URLs (cache invalidation) on remote clients.

The goal would be to support using a static url like
https://mybucket.bcdn.com/somestore/mycustomavatar-GUID-GOES-HERE.BasisEncyptedMeta
that doesn't need to be changed, but simply needs it's returned content updated.

Currently it's a big pain to constantly have to copy and paste the URLs/key every time after a change that needs testing or iterated on quickly.

Describe your preferred solution

To solve this issue, I believe the best method is to include an additional timestamp value of when the asset was last modified, stored in plaintext, and passed along with the rest of the Asset Definition content.
Then whenever the source (eg: a basis client user) declares that an Asset Definition has been modified, instead of having to fetch the Meta URL to check for further information every time the asset needs to be loaded, it can simply "react" to the timestamp changing.
This could be done with an API that is binded to a UI button that only the user who knows the encryption key can trigger. It could also be something that is delivered to clients via the server (should a platform built on basis adopt using this system).

Extending off of this would be easy time-based cache invalidation. By using a last-modified timestamp as a cache id, the application can quickly check all local cache items and delete those that are past some configurable or pre-defined lifespan, after which it would force re-fetch the Asset Definition urls should they ever show up again.

Another scenario that this would help resolve is if two users are using the same avatar Asset Definition, but one of them updates and then propagates, and the other user isn't present to receive the signal for invalidating the cache.
The timestamp can help automatically resolve the "source of truth" issue by having all clients always use the latest timestamp for any Asset Definition it sees.

If it has an older one that is still valid time-wise and has already loaded, the app could force re-download the avatar so it's the latest, but that might be disruptive for the in-the-moment experience of users.
So actually re-loading the avatar in the scene should be only at the behest of the user equipping it, or through normal unload/reload functionality.
This would probably be best served by just having a reload avatar button in the UI, which of course is mainly just a UI problem rather than a functional one.

Describe any considered alternatives

The primary alternative would be to embed some unique or sequential version identifier in the AssetMeta file, which would have to be fetched by the application client every time it gets an avatar Asset Definition, in order to ensure it has the latest copy.
This can pile up quickly as a bunch of excess network calls that will slow down the avatar loading period, which is being intentionally avoided with the current implementation. So it's not strictly advised.

Another option would be to add handling for cache-related header data on a HEAD call to the AssetMeta file. But this offers limited support due to the need for a compliant CDN that supports the necessary header information.
But, ALSO this still runs into the +1 net call for every avatar on load excess that is trying to be avoided.

Additional Context

It might not be a bad idea to embed the last-modified timestamp within the AssetMeta as an additional layer of validation for what is the "latest copy", though this timestamp would be different from the propagated one UNLESS when a user actions an update, the Meta is fetched, parsed and validated for correctness first, then the embedded timestamp would be the one that is propagated as the "source of truth", which if the timestamp is newer than the cached one, grab the associated AssetBundle url as well and update as needed.

Very much a "came to me as I was typing it out" kind of thing. Details of this last bit need some refinement.

@techanon techanon added the enhancement New feature or request label Feb 14, 2025
@neon-glowstick
Copy link
Contributor

I like this idea a lot. I imagine the flow will be something like this.

  1. I switch into my avatar (or reload it)
  2. I request the newest timestamp for the avatar from the server.
  3. I compare the timestamp against my local cache. If its newer, I download the avatar.
  4. I send an avatar change message to the game server with the timestamp and it is broadcast to all connected players.
  5. You receive an avatar change message along with a timestamp.
  6. You compare the timestamp against your local cache. If the timestamp is newer, you download the avatar from the server.

Only the user that changes into an avatar makes the extra call to the server to compare the timestamp. Everyone else trusts the timestamp they get along with the avatar change message.

I think the timestamp can go in the meta file as it becomes part of the asset definition. Its the versioning of the avatar. And if only the user changing into the avatar needs to check the server timestamp it doesn't need to be plaintext.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants