-
Notifications
You must be signed in to change notification settings - Fork 565
[Feature] support min_p_sampling #2872
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
Thanks for your contribution! |
@@ -282,6 +286,7 @@ def forward_cuda( | |||
sampled_token_ids=next_tokens, | |||
logprobs_tensors=logprobs_tensors, | |||
) | |||
self.step+=1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
delete it
""" | ||
min_p_sampling | ||
""" | ||
if paddle.count_nonzero(min_p_arr)==0: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pre-commit all files
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
咱们的pre-commit失效了吗
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
单测需要和小算子组合比对正确性
test/layers/test_min_p.py
Outdated
# limitations under the License. | ||
|
||
|
||
import matplotlib.pyplot as plt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
单测不比依赖matplotlib
644ac9d
to
13d4cdd
Compare
功能描述
我们参考了flashinfer的实现(感谢),实现了min_p_from_prob,支持min_p以张量的形式传入,既支持gpu kernel的形式,也支持paddle散op的形式
使用方式
服务方式请求:
离线方式: