Skip to content

issues Search Results · repo:BlinkDL/RWKV-LM language:Python

Filter by

239 results
 (75 ms)

239 results

inBlinkDL/RWKV-LM (press backspace or delete to remove)

BlinkDL/RWKV-LM
RWKV4 cuda

Hello, I am trying to use the RWKV4 model to process a sequential pkl dataset. However, when I use the CUDA kernel, I encounter an error UnicodeDecodeError: gbk codec can t decode byte 0xaa in position ...
  • dyx0209
  • Opened 
    yesterday
  • #300

BlinkDL/RWKV-LM
AttributeError: 'types.SimpleNamespace' object has no attribute 'n_head'

Код: import torch from rwkv.model import RWKV from rwkv.utils import PIPELINE, PIPELINE_ARGS # Путь к модели MODEL_PATH = rrwkv7-g1-0.4b-20250324-ctx4096 # Инициализация модели на CPU model = RWKV(model=MODEL_PATH, ...
  • lev-Kunitsa
  • 1
  • Opened 
    4 days ago
  • #299

BlinkDL/RWKV-LM
Question about Pretraining

Hello! I wonder if RWKV7 used the sequence packing strategy during pre-training? If so, do the samples need to be masked from each other?
  • necrophagists
  • Opened 
    4 days ago
  • #298

BlinkDL/RWKV-LM
How to apply GPRO

How to apply GPRO methods to further training of the rwkv model
  • raymondzml
  • 1
  • Opened 
    on Feb 23
  • #293

BlinkDL/RWKV-LM
rwkw-r1?

is there any plans of releasing a reasoner model?
  • Kreijstal
  • 1
  • Opened 
    on Feb 22
  • #292

BlinkDL/RWKV-LM
rwkv-4 RWKV_TimeMix如何分割序列，没有看到cuda_run接受任何sequence切分的标志？

RWKV_TimeMix中在序列维度上进行操作，在进行训练时训练数据常常是首尾相接的，序列之间需要隔断分开进行处理，例如flashattention会接收一个序列开始位置的输入，RUN_CUDA似乎没有，是如何实现的
  • Bsdnbo
  • 1
  • Opened 
    on Feb 11
  • #291

BlinkDL/RWKV-LM
RWKV-6 and newer

To bring more awareness and adoption of RWKV, would it be possible to get benchmark scores on the Huggingface LLM leaderboard or on the model cards itself (For RWKV-6 and newer)? https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/ ...
  • DIGist
  • 1
  • Opened 
    on Feb 9
  • #290

BlinkDL/RWKV-LM
请问RWKV7 ChannelMix去除sigmoid了吗？会有什么影响？

作者您好，一直在关注RWKV， RWK7开始ChannelMix好像去除了sigmoid RWKV6: k = self.key(xk) k = torch.relu(k) ** 2 kv = self.value(k) return torch.sigmoid(self.receptance(xr)) * kv RWKV7: ...
  • myhub
  • 1
  • Opened 
    on Feb 6
  • #288

BlinkDL/RWKV-LM
BPE Tokenizer

Is it possible use bpe tokenizer instead rwkv_vocab_v20230424 in the next model? I tried rwkv model in Thai language. It look good but it is very slow because Thai is character level for rwkv_vocab_v20230424. ...
  • wannaphong
  • 2
  • Opened 
    on Jan 28
  • #286

BlinkDL/RWKV-LM
rwkv-7 代码和模型不一致

rwkv_v7_demo.py : args.vocab_size = 50304 01.b 实际：65536 raise RuntimeError( Error(s) in loading state_dict for {}:\n\t{} .format( RuntimeError: Error(s) in loading state_dict for RWKV: Missing key(s) ...
  • qxde01
  • 1
  • Opened 
    on Jan 26
  • #285
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue search results · GitHub