Excited to see MegaBeam-Mistral-7B on the RULER long-context leaderboard! Probably the only 7B model in the top 10...
Weights - https://t.co/qzX1sIkauF
RULER leaderboard - https://t.co/ZKAVlAY0Ul
Excited to share the large-context (512K tokens) LM https://t.co/qzX1sIkauF
- Achieved 88.7 on the RULER benchmark, comparable to Llama3.1-8B (88.3) and Llama3.1-70B (89.6).
- Scored 34.9 on the InfiniBench
- 100% on the Needle-In-A-Haystack (to cover the base)
- Apache 2.0
@AshleyAitken@AWS Thank you so much Ashley for coming ! Loved your thought provoking question! Was great to see you and we shall catch up with you soon!
Our solution to the Amazon Last Mile Routing Research Challenge is now available as open source. Also ready to share today is our arXiv preprint on route optimisation with sequence models and RL.
https://t.co/BCVE6JWFKW
https://t.co/nBNdrTYBsK
2 types of developers: those who read the Stack Overflow question first, and those who skip past it and go straight to the answer
i have literally never met the former
At @RealAAAI, @geoffreyhinton recalls his advisor’s advice: “reading rots the mind”. Only once you have figured how you would solve a problem, then read the literature.
👇What I learned recently about Python, CUDA, and/or @spacy_io multi-processing.
1. Python lack of multi-threading is super-annoying but there're crutches that can still be useful.
2. You need to call multiprocessing.set_start_method('spawn') or your CUDA is gonna be screwed up