Skip to content

First Part of the project (Tokenization)

Molly-Mitton Clementrequested to merge
dev into main

Here you'll find tthe first part of the project, including the tokenizers we implemented, instructions on how to train them, and the trained artifacts (weights/checkpoints) generated after trainiing.

Merge request reports