Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds NDCG metrics as a part of adding RecSys metrics in #2631
This PR builds on the work done in #3054 and #2632 (much thanks to ili0820 and kamalojasv181!)
The primary changes left at this stage as per my understanding would be around vectorising parts of the DCG computation as previously discussed here and here, and then docs.
I've considered different ways to address that, namely using
torch.vmap
(but I think that will be better to pursue once pytorch/pytorch#124423 is resolved); a batched vectorisation and a full one. Right now I think the batch-based vectorisation could be a good way to go since a full vectorisation approach could be a bit too complex (/overkill?) here, the main cause of complexity being different numbers of unique values across samples. Let me know what you think! I'm still in the early stages of working at this level and so open to feedback is all I am 🤠