Peng Boris Akebuon

Software Engineer @Traleor Python, GraphQl, Go, and Typescript Developer instereted in AI/ML.
YouTube:

[Tech Culture Africa @techcafrica]

•REDIET ABEBE

Abebe is an Ethiopian computer scientist. Also an assistant professor at the University of California.

She is the Co-founder of Black Researchers.

[Jim Fan @DrJimFan]

We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive - truly open, frontier research that empowers all. It makes no sense. The most entertaining outcome is the most likely.

DeepSeek-R1 not only open-sources a barrage of models but also spills all the training secrets. They are perhaps the first OSS project that shows major, sustained growth of an RL flywheel.

Impact can be done by "ASI achieved internally" or mythical names like "Project Strawberry".
Impact can also be done by simply dumping the raw algorithms and matplotlib learning curves.

I'm reading the paper:

> Purely driven by RL, no SFT at all ("cold start"). Reminiscent of AlphaZero - master Go, Shogi, and Chess from scratch, without imitating human grandmaster moves first. This is the most significant takeaway from the paper.
> Use groundtruth rewards computed by hardcoded rules. Avoid any learned reward models that RL can easily hack against.
> Thinking time of the model steadily increases as training proceeds - this is not pre-programmed, but an emergent property!
> Emergence of self-reflection and exploration behaviors.
> GRPO instead of PPO: it removes the critic net from PPO and uses the average reward of multiple samples instead. Simple method to reduce memory use. Note that GRPO was also invented by DeepSeek in Feb 2024 ... what a cracked team.

[DeepSeek @deepseek_ai]

🚀 DeepSeek-R1 is here!

⚡ Performance on par with OpenAI-o1
📖 Fully open-source model & technical report
🏆 MIT licensed: Distill & commercialize freely!

🌐 Website & API are live now! Try DeepThink at https://t.co/v1TFy7LHNy today!

🐋 1/n

[Peng Boris Akebuon @itz_omen]

The greenest year so far, let's keep committing.

[Tech in Ghana @TechInGhana]

Peng Boris is a technology leader with expertise in artificial intelligence and SaaS development. As Co-founder and CTO, he has worked with American, German, and African companies to build scalable SaaS products.

Passionate about empowering the tech community, Peng has trained more than 1,000 software engineers across Cameroon, Ghana, Nigeria, and Kenya through workshops and mentorship programs.

Additionally, he runs a YouTube channel that provides tutorials and industry insights, highlighting his commitment to knowledge sharing and community growth.

#techingh24 #14thEdition #synergy

[Highscore @highscore]

🔔 New Steam Game #Giveaway [Ends in 48 hours]

🎮Hogwarts Legacy🎮
(Gifted through Steam, Metascore: 83)

To enter:
- Retweet 🔃
- Tag 2 friends ⬇️
- Follow @highscore 😊

We'll pick a winner on Saturday, August 31st 🎉

Steam Page: https://t.co/zZfJJDD4xK

[Highscore @highscore]

60 seconds in GTA: San Andreas #gta #gtasanandreas #videogames

[Peng Boris Akebuon @itz_omen]

@marianne_makes Sorry 😐 to hear, that sucks

[OpenAI @OpenAI]

We're sharing progress toward understanding the neural activity of language models. We improved methods for training sparse autoencoders at scale, disentangling GPT-4’s internal representations into 16 million features—which often appear to correspond to understandable concepts.

https://t.co/tTRPztmra1

[Highscore @highscore]

Parsec vs. Highscore latency benchmark test 👀🔥

Thanks @cgbattlehub for the video!

https://t.co/pMaWrKn61E

[Guilherme Penedo @gui_penedo]

We have just released 🍷 FineWeb: 15 trillion tokens of high quality web data.
We filtered and deduplicated all CommonCrawl between 2013 and 2024.
Models trained on FineWeb outperform RefinedWeb, C4, DolmaV1.6, The Pile and SlimPajama!

[Devendra Chaplot @dchaplot]

We just released Mixtral-8x22B-v0.1 and Mixtral-8x22B-Instruct-v0.1:
- Free to use under Apache 2.0 license
- Outperforms all open models
- Native function calling
- Masters English, French, Italian, German and Spanish.
- Seq_len = 64K

https://t.co/SCG8s06Dbl

[Aran Komatsuzaki @arankomatsuzaki]

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

- Presents MiniCPM family, including MiniCPM-DPO, MiniCPM-MoE and MiniCPM-128K
- MiniCPM 2B performs on par with Phi-2 and outperforms Mistral and Gemma

https://t.co/fRRwQbXtKD

[Andrej Karpathy @karpathy]

Highly amusing update, ~18 hours later:

llm.c is now down to 26.2ms/iteration, exactly matching PyTorch (tf32 forward pass). We discovered a bug where we incorrectly called cuBLAS in fp32 mathmode 🤦‍♂️. And ademeure contributed a more optimized softmax kernel for very long rows (50,257 elements per row, in the last logits layer).

But the fun doesn’t stop because we still have a lot of tricks up the sleeve. Our attention kernel is naive attention, not flash attention, and materializes the (very large) preattention and postattention matrices of sizes (B, NH, T, T), also it makes unnecessary round-trips with yet-unfused GeLU non-linearities and permute/unpermute inside our attention. And we haven’t reached for more optimizations, e.g. CUDA Graphs, lossless compressible memory (?), etc.

So the updated chart looks bullish :D, and training LLMs faster than PyTorch with only ~2,000 lines of C code feels within reach. Backward pass let’s go.

[Women Techmakers Buea #IWD25 @WTMBuea]

Not forgetting @DjangoCameroon was in the room with us today🔥💙 presented by @yokwejuste

If you’re a Django developer or enthusiast then you should definitely think about checking out and joining this community🎊

#ImpactTheFuture
#IWD2024
#WTMBueaIWD2024

[Highscore @highscore]

Thanks @CloudDosage for the feature!

[Zengyi Qin @qinzytech]

Training LLMs can be much cheaper than previously thought.

0.1 million USD is sufficient for training LLaMA2-level LLMs🤯

While @OpenAI and @Meta use billions of dollars to train theirs, you can also train yours with much less money.

Introducing our open-source project JetMoE: https://t.co/TG2JA612AM

A thread 🧵

[Peng Boris Akebuon @itz_omen]

@NinaTakang Nice pictures 🖼️

[Peng Boris Akebuon @itz_omen]

Introducing @bunjavascript v1.1 - a huge update with native Windows support

https://t.co/OPdegVM8cQ

[Women Techmakers Buea #IWD25 @WTMBuea]

Pick a side 🌚

⏰Tell a friend to tell a friend! ⏰
Let's make it an unforgettable experience together!✨😊

#ImpactTheFuture
#IWD2024
#WTMBueaIWD2024