You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary:
To 2.1
Unfortunately, I can't remove the old version of the crate. The problem is that it appears in the [public API](https://docs.rs/tiktoken-rs/latest/tiktoken_rs/struct.CoreBPE.html#method.new) of `tiktoken_rs`, which we also import. That crate has a PR to upgrade its dependency [here](zurawiki/tiktoken-rs#95) but that has not been merged. Alternatives here are:
1. Fork `tiktoken_rs`. This seems worse.
2. It's strictly speaking possible to use generics combined with some other tricks to avoid needing to name v1 of `FxHashMap` in fbsource. However, that makes that code extraordinarily hard to change and work with, and doesn't seem all that clearly motivated.
If there's something else I should be doing here, all ears
Reviewed By: zertosh
Differential Revision: D68064854
fbshipit-source-id: bd1c96abbc0ebaa178a55826bc85c61eb35e8301
0 commit comments