Why glTF asset labels didn't work for me

[greeble-dev]

When I started using Bevy I mostly copy and pasted the examples. I wanted to load skinned meshes and animations from glTFs, so I used GltfAssetLabel because that's what the examples did.

After trying this out for a while, I decided to drop GltfAssetLabel and switch to loading entire Gltf asset - something almost none of the examples do.

I'm writing this down because "don't do what the examples do" seems like a bad place to be in. Also I get the sense that there's some design uncertainties around sub-assets, so maybe this feedback will help.

Background

A glTF asset contains sub-assets like animation clips and meshes. The sub-assets are labelled, and many examples use this to select parts of a glTF file.

Here's some code adapted from the animated_mesh_control example:

let animation_clips = [
    asset_server.load(GltfAssetLabel::Animation(0).from_asset("models/animated/Fox.glb")),
    asset_server.load(GltfAssetLabel::Animation(1).from_asset("models/animated/Fox.glb")),
    asset_server.load(GltfAssetLabel::Animation(2).from_asset("models/animated/Fox.glb")),
];

let graph = AnimationGraph::from_clips(animation_clips);

let scene = asset_server.load(GltfAssetLabel::Scene(0).from_asset("models/animated/Fox.glb"));

There's an alternative - load the Gltf directly, wait for it to finish, then pull the sub-assets out of the Gltf struct:

let gltf_handle: Handle<Gltf> = asset_server.load("models/animation/Fox.glb");

... wait a bit ...

if Some(gltf) = assets.get(&gltf_handle) {
    let animation_clips = [
        gltf.animations[0],
        gltf.animations[1],
        gltf.animations[2],
    ];
    
    let graph = AnimationGraph::from_clips(animation_clips);

    let scene = gltf.scenes.first();
}

After a while I came to prefer this approach, even though it's clunkier at first glance. A more realistic example is here: https://github.com/greeble-dev/bevy_mod_skinned_aabb/blob/0af9e6ac1495ac7bfd4e2582cd09ea79e6080409/examples/showcase.rs#L111

So, what's wrong with labelled sub-assets?

Problem 1: Performance Traps

Loading a sub-asset via GltfAssetLabel requires loading the whole glTF, including the work of turning the glTF into Bevy assets. GltfLoader::load always returns a complete Gltf, and then AssetServer drops everything but the requested sub-asset.

This may not be surprising to someone familiar with the asset system, but it was a surprise to me. I assumed that loading a single animation would do less work.

Even worse, loading N different sub-assets from the same glTF requires loading the whole glTF N times. Here's what happens with the animated_mesh_control example that loads three animations and a scene:

If I use labels to load all the animations from a glTF containing a hundred animations, that means ten thousand AnimationClips are created and 99% of them are instantly destroyed.

However, if I load a Gltf directly then I know it's only loaded once. And while I still can't load just one part, I'm not misled into thinking I am.

Problem 2: Labels Are Limited

GltfAssetLabel identifies sub-assets by index. This makes sense, because it's how glTF identifies things internally. Many glTF sub-assets can also have names, but they're optional and not required to be unique.

However, indices are awkward. I will give assets names if I want to refer to them individually, so I can load animation "run", not animation "17". Indices are also fragile - deleting an animation can change the indices of other animations.

(There have been a few efforts to support names in GltfAssetLabel, but none have yet succeeded. The latest is #16529.)

A deeper issue is that labels can't express many needs. For example, there's no way to select both a mesh and the material used by that mesh. I can spawn an entire scene or a select a single asset, but nothing in-between.

If I load a Gltf then I get a HashMap<Box<str>, Handle<AssetType> for all the named assets. I can also find relations between sub-assets, like looking up a GltfPrimitive to get both the mesh handle and material handle.

Problem 3: Pre-Spawning Is Not That Useful

One advantage of GltfAssetLabel is that I can start loading sub-assets and immediately assign them to an entity. So I can spawn a player entity and start loading their mesh in one step, and the mesh will appear when it's ready.

But how useful is this in practice? I don't want my game start with a blank screen and have assets pop in when ready. As well as looking bad, how can the player jump over a gap if their jump animation hasn't loaded? Or what if it works fine on my computer, then falls over on a customer's slower computer?

If I load a Gltf then I don't get to pre-spawn easily. But for me that's not a limitation, and it's easier to wait for one asset than to wait for several.

Summary

When I started using Bevy, GltfAssetLabel seemed like the obvious solution. But I quickly ran into problems.

GltfAssetLabel might have worked for me if: 1) my assets were small or performance was not a concern, and 2) my use cases fit the limitations. Or I might have required them if I needed easy pre-spawning.

Loading the Gltf seemed clunkier and more limiting at first glance, but I practice it was a much better fit.

[PatrickChodowski]

That is very valuable information, thank you for sharing.

ekhm...