swissbert/pretraining at master · ZurichNLP/swissbert · GitHub

Name		Name	Last commit message	Last commit date
parent directory ..
data_preprocessing		data_preprocessing
fairseq_additions		fairseq_additions
training_scripts		training_scripts
README.md		README.md
requirements.txt		requirements.txt
swissdox.py		swissdox.py
tokenization.py		tokenization.py
utils.py		utils.py

README.md

Code for adapting X-MOD on our pre-training corpus.

The corpus itself is not released.

Dependencies

See requirements.txt

Data preprocessing

See data_preprocessing/prepare_data.ipynb

Training

See training_scripts/swissbert_new_vocab.sh