Bhaksar Boruah

[Bhaksar Boruah @boruahbhaskar]

Wishing everyone a joyful, safe, and proud Fourth of July! 🎆🇺🇸

#July4 #IndependenceDay #USA250

3/3

[Bhaksar Boruah @boruahbhaskar]

and its enduring commitment to the Constitution, the rule of law, and individual liberty.
From incredible BBQ and tacos to stunning parks, rivers, and endless opportunities—there’s so much to love. Thank you, America, for welcoming me and allowing me to be part of your story. 2/3

[Bhaksar Boruah @boruahbhaskar]

Happy 250th Independence Day, America! 🇺🇸

After more than 10 years of calling the United States home, I’ve grown to deeply appreciate this remarkable country. I admire the spirit of its people, the freedoms it protects, its respect for innovation and talent.
1/3

[Bhaksar Boruah @boruahbhaskar]

@matt_elevenlabs 11B

[Bhaksar Boruah @boruahbhaskar]

@calcsam BOOK

[Bhaksar Boruah @boruahbhaskar]

তুমি এতিয়া বহু দূৰৈৰ এখন ঠাইলৈ গুছি গৈছা, য’ৰ পৰা ঘূৰি আহিব নোৱাৰা। কিন্তু আমাৰ বাবে তুমি সদায় ধ্ৰুৱতৰা দৰে জিলিকি থাকিবা, আমাৰ পথ দেখুৱাই। দা, তুমি আমাৰ হৃদয়ত চিৰকাল জীয়াই থাকিবা। শান্তিৰে থাকা, আমাৰ প্ৰিয় জুবিন দা।চিৰদিনৰ বাবে কৃতজ্ঞ, তোমাৰ এজন অনুৰাগী

[Bhaksar Boruah @boruahbhaskar]

জুবিন দাৰ প্ৰতি এজন অনুৰাগীৰ শ্ৰদ্ধাঞ্জলি 
জুবিন দা,তোমাৰ গানৰ লগত মোৰ এনে এক ব্যক্তিগত সম্পৰ্ক আছিল, সঁচাকৈয়ে শব্দৰে বৰ্ণনা কৰিব নোৱাৰো। গান শুনা আৰম্ভ কৰাৰ পৰা প্ৰতিদিনে তোমাৰ গান নুশুনাকৈ এদিনো যোৱা নাই। তুমি মোৰ সৈতে এনেকৈ জড়িত হৈ আছিলা যে

[Bhaksar Boruah @boruahbhaskar]

যিয়ে আমাৰ প্রত্যেকৰে হৃদয়ত স্থান দখল কৰিছিল। তোমাৰ গানবোৰ কেৱল সুৰ নহয়, সেয়া আমাৰ জীৱনৰ স্মৃতি, আমাৰ আৱেগৰ প্ৰতিফলন। তুমি আমাৰ হৃদয়ৰ গভীৰতাত সদায় জীয়াই থাকিবা। তোমাৰ গানবোৰ প্ৰজন্মৰ পিছত প্ৰজন্মই উদযাপন কৰিব।

[Rock Photography @Photomusicrock]

Last photo of Black Sabbath alongside members of Metallica, Aerosmith, Pantera, Van Halen and more Rock and Heavy Metal legends before their final concert called 'Back To The Beginning' on July 5, 2025.

[UNESCO @UNESCO]

🔴 BREAKING!

New inscription on the @UNESCO #WorldHeritage List: Moidams – the Mound-Burial System of the Ahom Dynasty, #India 🇮🇳.

➡️https://t.co/FfOspAHOlX #46WHC

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Q-Learning is *probably* not the secret to unlocking AGI. But, combining synthetic data generation (RLAIF, self-instruct, etc.) and data efficient reinforcement learning algorithms is likely the key to advancing the current paradigm of AI research…

TL;DR: Finetuning with reinforcement learning is the secret to training high-performing LLMs like ChatGPT/GPT-4. But, RL is data inefficient by nature, and using humans to manually annotate dataset for finetuning with RL is super expensive. With this in mind, advancing AI research (at least in the current paradigm) will heavily rely on two fundamental goals:

1. Making RL perform better with less data.
2. Synthetically generating as much high-quality data for RL as possibly using LLMs and smaller sets of manually annotated data.

Typical RL setup. Usually, we use RL to learn a policy that iteratively chooses the best action to take given a current state. Then, we use this policy to continually choose the next state and traverse an underlying environment until we reach a terminal/end state. The purpose of RL is to learn a policy that maximizes the reward we receive from the environment as we sequentially choose and visit each state.

RL for LLMs. Training language models is not our typical problem setup for RL. However, we can easily formulate text generation in the lens of RL. Language models operate by auto regressively outputting each token in their output. So, our state is just the current output from the model. Our policy is the language model, which predicts the most likely next token given current tokens as input. The reward is a human preference, and we train the model to generate text that maximizes this reward.

RL algorithms. Using the setup described above, we can easily apply many different RL algorithms to finetuning LLMs. For example, we can use Q-learning to model next token prediction with a lookup table for predicting the next token on simple vocabularies. However, storing this lookup becomes very memory intensive (and eventually not tractable), so we will want to use Deep Q-Learning to model next token prediction with a neural network instead of a lookup table. Going further, most modern research uses more practical, data efficient RL algorithms like PPO in practice.

Where do we hit a wall? Recent work has shown us that using RL to finetune LLMs (i.e., reinforcement learning from human feedback) is incredibly effective. However, there is one primary problem—RL is data inefficient and requires us to collect a ton of data to achieve good performance. To collect data for RLHF, we have humans manually annotate their preferences (e.g., LLaMA-2 is fine-tuned over 1M human preference annotations). Although this technique works well, it is super expensive and the barrier to entry is incredibly high. As a result, RLHF is only leverage by organizations with massive resources (e.g., OpenAI or Meta), while everyday practitioners rarely leverage these techniques (i.e., most open-source LLMs use SFT and no RLHF).

What’s the solution? Although there may not be a perfect solution, recent research has began to leverage powerful LLMs (e.g., GPT-4) to automate the data collection process for finetuning with RL. This was first explored by Constitutional AI by Anthropic, where LLMs synthetically generated harmfulness data for LLM alignment. Later, Google proposed reinforcement learning for AI feedback (RLAIF) where LLMs were used to automate the entire data collection process for RLHF. Surprisingly, using LLMs to generate synthetic data for finetuning with RL is incredibly effective.

Synthetic data from LLMs. We see across a variety of research papers that using LLMs to generate synthetic data is a massive research frontier. Examples of this include:
- Self-Instruct: LLMs can automatically generate instruction tuning datasets with LLMs (similar approaches are followed by Alpaca, Orca, and many other models).
- LLaMA-2: LLMs are capable of generating their own high-quality data for SFT after a small amount of examples are annotated by humans.
- Constitutional AI: LLMs can use self-critique to generate high-quality datasets for alignment via both RLHF and SFT.
- RLAIF: Instead of using humans to collect feedback, we can completely automate the feedback component of RLHF with LLMs and achieve comparable performance.

My takeaway. I’m not sure what advancements in AI/AGI are ahead of us. But, if we stick to the current paradigm of next token prediction (i.e., pretraining —> SFT —> RLHF) with decoder-only transformers, I’m nearly positive that finetuning with RL combined with synthetic generation of data via powerful LLMs will play a massive role in democratizing/improving LLMs. This approach makes cutting-edge training techniques accessible to everyone, instead of just research groups with large amounts of funding!

[Andrej Karpathy @karpathy]

New YouTube video: 1hr general-audience introduction to Large Language Models
https://t.co/Bl4WNuNyFJ

Based on a 30min talk I gave recently; It tries to be non-technical intro, covers mental models for LLM inference, training, finetuning, the emerging LLM OS and LLM Security.

[ISRO @isro]

Chandrayaan-3 Mission:
'India🇮🇳,
I reached my destination
and you too!'
: Chandrayaan-3

Chandrayaan-3 has successfully
soft-landed on the moon 🌖!.

Congratulations, India🇮🇳!

#Chandrayaan_3
#Ch3

[Bhaksar Boruah @boruahbhaskar]

@ChrisStaud 👋

[Piers Morgan @piersmorgan]

What. A. Player. #mbappe

[Bhaksar Boruah @boruahbhaskar]

Happy for Messi !! Congratulations Argentina !! #ARGFRA

[Bhaksar Boruah @boruahbhaskar]

What a game !! #Mbappe #ARGFRA

[Bhaksar Boruah @boruahbhaskar]

Very sad to hear that Shane Warne is no more. He is a legend and will be missed greatly by all cricket lovers.

[Cricinfo @cricinfo]

There will never be anyone like Shane Warne