Model Costs and Cached Tokens

[Leon0402]

What feature would you like to be added?

Previously there where fields client.total_usage_summary and planner.client.actual_usage_summary with the amount of tokens and the costs. There is a class

@dataclass
class RequestUsage:
    prompt_tokens: int
    completion_tokens: int

but I think apart from the logic being flawed (see #4769, #4719) it also lacks important fields. Most notably the costs and the cached tokens.

I think this should also be mentioned in the Migration Guide.

Why is this needed?

Has been there previously in v2 and seems very useful to have in v4 as well.

[ekzhu]

Thanks @Leon0402 for the issue. yes, it is important. We are planning to address this after the 0.4.0. For now, let's targeting 0.4.1 for this one.

#4769 and #4719 must be resolved before tackling this one.