The max tokens of generated answer by ChatGPT is 768？ Is the restricted by proxy or openai? #1033
Replies: 1 comment 7 replies
-
|
For V3, proxy is not used. |
Beta Was this translation helpful? Give feedback.
-
|
The max tokens should be 4096 - the number of tokens in prompt |
Beta Was this translation helpful? Give feedback.
-
|
yes, I also see that in the api document. However, I got max 768 tokens by V1 method. I am much confused........ |
Beta Was this translation helpful? Give feedback.
-
|
I'll check this out |
Beta Was this translation helpful? Give feedback.
-
|
I'm not hitting any token limit. It might be that the model was fine tuned to speak less. |
Beta Was this translation helpful? Give feedback.
-
|
I asked it to count numbers to infinity and it reached the same amount with both manual max_tokens and max_tokens unset |
Beta Was this translation helpful? Give feedback.
-
RT
Beta Was this translation helpful? Give feedback.
All reactions