@maximelabonne Nice! We're going to release our training datasets in the coming weeks, including human curated preference data - would be happy to share more with you ahead of time
@digitalix The spark boast 1 Petaflops sparse NVFP4, or 512 teraflops dense. Can you find any configuration with any framework where you can reach that training a model?
We've come a long way from the initial announcement of the Sovereign LLM project - excited to showcase our results.
We've written up a tutorial sharing how you can train your own Sovereign LLM using Nvidia Nemotron, NeMo Framework, and DGX Cloud Lepton: https://t.co/1Y65PdMOdp
Great work together with @nvidia
Beyond excited to announce the official release of:
π Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs π₯
Dicta-LM 3.0 is a powerful open-weight collection of LLMs with full Hebrew support.
View the full announcement here: https://t.co/FxPr1j6yBI
π§΅
The models were trained on a cluster of over 100 H200 GPUs, on NVIDIA DGX Cloud Lepton. All training was done using the NVIDIA NeMo Framework and the NVIDIA NeMo-RL library. We are extremely grateful to NVIDIA and their technical teams, who made this all possible!
Beyond excited to announce the official release of:
π Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs π₯
Dicta-LM 3.0 is a powerful open-weight collection of LLMs with full Hebrew support.
View the full announcement here: https://t.co/FxPr1j6yBI
π§΅