refactor(attn): let the bucketer own the batched decode buckets - #810

refactor(attn): let the bucketer own the batched decode buckets#810

sducouedic wants to merge 4 commits into

torch-spyre:maintorch-spyre/spyre-inference:mainfrom

sducouedic:bucketer-owns-num-seqs-laddersducouedic/spyre-inference:bucketer-owns-num-seqs-ladderCopy head branch name to clipboard