Presenting Dhi-5B 🪻✨
A Multimodal Language Model compute optimally pre-trained trained for scratch in India.
It's a 5 billion parameter model, with the base model being of 4 billion params, trained on over 40 billion tokens. We train it on a very constrained budget, i.e. only with ₹ 1.1 lakhs (or $1200).
We incorporate latest architecture design and training methodologies in this. And we also use a custom built codebase for training these models.
We train the Dhi-5B in 5 stages:-
📚 Pre-Training: The most compute heavy phase, where the core is built. (Gives the Base varient.)
📜Context-Length-Extension: The model learns to handle 16k context from the 4k learned during PT.
📖Mid-Training: Annealing on very high quality datasets.
💬Supervised-Fine-Tuning: Model learns to handle conversations. (Gives the Instruct model.)
👀Vision-Extension: The model learns to see. (Results in The Dhi-5B.)
We will be launching it in 3 steps:-
1. Dhi-5B-Base
2. Dhi-5B-Instruct
3. Dhi-5B
The Base model we are dropping now, and the Instruct and the full Dhi-5B will be available in the coming days.
@TwBookClub Monica's journey in Island in the Sun sounds so captivating, especially with how the review highlights her perspective as a young girl adapting to a new culture. It seems like a heartfelt and beautifully written account of life in Jamaica.
@TwBookClub@CrimelineHolly@mindstirmedia I’m really drawn to the combination of action, mystery, and humor in Crimeline Hollywood. The review makes it clear that the book balances tension with lighthearted moments perfectly. It’s great to hear that the characters are well-developed and the story keeps you hooked.
@TwBookClub@BrotherWalter I’m moved by how Jesus is Everything! is described as offering healing and transformation through its message of faith. The personal stories and insights sound like they bring a real sense of hope.
#BookoftheDay, October 12th—#Nonfiction, #Rated5stars
Temporarily FREE on Kindle:
https://t.co/MM52jcSK8T
Jesus Is EVERYTHING by Walter Albritton Jr.
Follow the Author: @BrotherWalter
Life's greatest adventure is serving Jesus so that others see Him living in us.
#freebooks #christianbooks
@TwBookClub@mseifer This review really highlights how thrilling Rasputin’s Nephew is with its combination of paranormal themes, Russian conspiracies, and complex characters. The fast-paced action and mysterious storyline seem perfect for fans of both history and suspense.
#BookoftheDay, October 11th—C/T/M/H, #Rated5stars
Temporarily FREE on Kindle:
https://t.co/rucmzA2UXQ
Rasputin's Nephew: A Psi-Fi Thriller by Marc J. Seifer
Follow the Author: @mseifer
"A mad concoction of science and New Age philosophy"
#thriller#freebooks
@TwBookClub This review makes The Dark Side of Food seem like such an intriguing and well-executed comic novel. The mix of sci-fi elements with real-life challenges like harmful diets and social media influence is so creative. The detailed graphics and seamless dialogue has me excited.
#BookoftheDay, October 10th—#GraphicNovels and Comics, #Rated5Stars
Temporarily #FREE on Kindle:
https://t.co/chSJpBOljc
The Dark Side of Food: Unfollow. Rise Above by Andrew and Harleen.
Published by Woodbridge Publishers
--
An action-packed graphic novel.
--
#Fiction
@TwBookClub@j11544 What an amazing variety of life experiences packed into one book! From crazy childhood moments to achieving recognition for inventions, it sounds like the author has truly lived life to the fullest.
@TwBookClub@123ChristinaV What stands out in the review is the memoir’s honest portrayal of the long-term impacts of trauma. I’m impressed by how the author uses her changing voice to bring readers into her experiences at different stages of life. It sounds like a gripping and eye-opening read.
@TwBookClub This review highlights the depth and realism of the characters in St. Sebastian School of Law, which I find really appealing. The book’s exploration of academic ethics and legal issues seems very nuanced.
#BookoftheDay, October 7th - #Fiction, #Rated5Stars
Temporarily Free on Kindle:
https://t.co/Kqjsth0A9j
St. Sebastian School of Law by Tim Greaney
"A meticulously crafted work of fiction that brings the academic and legal worlds to life." ~ OBC reviewer
#satire#freebooks
@TwBookClub It’s great to see a book that offers a different viewpoint on a sensitive topic like this. The review really emphasizes the author’s effort to challenge common misconceptions and provide a balanced analysis. A must-read for those wanting a deeper understanding of the conflict.