🚀 Introducing ChronoGPT-instruct
Excited to share the latest instruction-tuned ChronoGPTs, joint work with @hesongrun, @AsafManela, and @JimmyCMWu . A series of high-performance, chronologically consistent, instruction-following LLMs designed to eliminate lookahead bias.
🚀 Introducing ChronoBERT: A Chronologically Consistent Language Model ⏳📜
Excited to share ChronoBERT, a joint work with @LinyingLyu , @AsafManela , and @jimmywucm ! Our new pre-trained language model ensures no lookahead bias and strong language understanding! 🧠✨
nanoGPT speedrun: Nice work from @kellerjordan0 adapting the nanoGPT/llmc PyTorch training code into a benchmark training a 124M Transformer to a fixed validation loss target. Current SOTA is 3.8X more token-efficient training (2.7B vs. 10B tokens)