Abhinav Choudhary 🍩

posting random shit or academic papers!
Welcome to my online diary,
lowkey and isolated

[Abhinav Choudhary 🍩 @meghhdoot]

@ian_dot_so Can't DM you!

[Elias @Eliasfiz]

Today, we're launching Orpheus
Multilingual, a family of open-source models that makes state-of-the-art TTS accessible to billions of new people! 🌎🌎 (1/5)

[Shubham Saboo @Saboo_Shubham_]

After DeepSeek R1, this new research from China will enable RAG AI Agents to process entire codebases and documentation without context limits

It uses Mixture of Experts with Sparse attention to achieve near infinite context in LLMs

100% Opensource.

[Sakana AI @SakanaAILabs]

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels.

https://t.co/DoloC7BGHq

The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production.

We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient!

We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer.

Paper: https://t.co/Rn9ioA10As
Kernel Archive Webpage: https://t.co/hqXoQqAvdS
HuggingFace Dataset: https://t.co/Lh1xEYxZJv

The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations!

We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization.

We (@RobertTLange, @_Aaditya_Prasad, @Floating_Martin, @maxencefaldor, @yujin_tang, @hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

[Abhinav Choudhary 🍩 @meghhdoot]

@gowthami_s thats how CUDA works too xD

[AK @_akhaliq]

Pre-training Small Base LMs with Fewer Tokens

We study the effectiveness of a simple approach to develop a small base language model (LM) starting from an existing large base LM: first inherit a few transformer blocks from the larger LM, and then train this smaller model on a

[Abhinav Choudhary 🍩 @meghhdoot]

@shaily99 Now the ads have become intelligent

[Abhinav Choudhary 🍩 @meghhdoot]

@gowthami_s @vikhyatk following

[Abhinav Choudhary 🍩 @meghhdoot]

Getting an analog watch has been a great decision to save phone screen time.

[Abhinav Choudhary 🍩 @meghhdoot]

@tisnalii @SparklinGuy 10 lakh ki scooter😂

[Abhinav Choudhary 🍩 @meghhdoot]

@tisnalii Same for internet

[Talia Ringer 🕊🪬 @TaliaRinger]

Israeli leftists: "my parents were murdered yesterday, but still we must push on for a better future, which means full recognition of both peoples who are sharing this land"

American leftists: "let us cheer the liberation fighters for decolonizing the peace music festival"

[Abhinav Choudhary 🍩 @meghhdoot]

meanwhile, you get a malware before you do anything

[mrx red @darkcoders_mrx]

Cobalt Strike 4.9 leaked in wild

[Abhinav Choudhary 🍩 @meghhdoot]

Strange that PyPi doesn't list dependencies like npm

[Abhinav Choudhary 🍩 @meghhdoot]

@kipperrii How did you click a picture like that🤔

[Abhinav Choudhary 🍩 @meghhdoot]

web3d coming soon

[AK @_akhaliq]

Text-to-3D using Gaussian Splatting

paper page: https://t.co/cNvqGxpiFJ

present Gaussian Splatting based text-to-3D generation (GSGEN), a novel approach for generating high-quality 3D objects. Previous methods suffer from inaccurate geometry and limited fidelity due to the absence of 3D prior and proper representation. We leverage 3D Gaussian Splatting, a recent state-of-the-art representation, to address existing shortcomings by exploiting the explicit nature that enables the incorporation of 3D prior. Specifically, our method adopts a progressive optimization strategy, which includes a geometry optimization stage and an appearance refinement stage. In geometry optimization, a coarse representation is established under a 3D geometry prior along with the ordinary 2D SDS loss, ensuring a sensible and 3D-consistent rough shape. Subsequently, the obtained Gaussians undergo an iterative refinement to enrich details. In this stage, we increase the number of Gaussians by compactness-based densification to enhance continuity and improve fidelity. With these designs, our approach can generate 3D content with delicate details and more accurate geometry. Extensive evaluations demonstrate the effectiveness of our method, especially for capturing high-frequency components.

[Abhinav Choudhary 🍩 @meghhdoot]

@Unnati_twts Sight seeing .. cafes or beaches ?

[Abhinav Choudhary 🍩 @meghhdoot]

520 strike rate🗿

[TukTuk Academy @TukTuk_Academy]

-34 Balls Century?
-9 ball fifty?
-300+ runs in T20s?

What the heck is going on? Is this how you play the cricket? We don't consider these Asian games even a sport🤮😑

[Abhinav Choudhary 🍩 @meghhdoot]

@lilianweng when is the next lil'log dropping? @lilianweng

[AK @_akhaliq]

Contrastive Decoding Improves Reasoning in Large Language Models

paper page: https://t.co/IYx3dn1Wb5

demonstrate that Contrastive Decoding -- a simple, computationally light, and training-free text generation method proposed by Li et al 2022 -- achieves large out-of-the-box improvements over greedy decoding on a variety of reasoning tasks. Originally shown to improve the perceived quality of long-form text generation, Contrastive Decoding searches for strings that maximize a weighted difference in likelihood between strong and weak models. We show that Contrastive Decoding leads LLaMA-65B to outperform LLaMA 2, GPT-3.5 and PaLM 2-L on the HellaSwag commonsense reasoning benchmark, and to outperform LLaMA 2, GPT-3.5 and PaLM-540B on the GSM8K math word reasoning benchmark, in addition to improvements on a collection of other tasks. Analysis suggests that Contrastive Decoding improves over existing methods by preventing some abstract reasoning errors, as well as by avoiding simpler modes such as copying sections of the input during chain-of-thought. Overall, Contrastive Decoding outperforms nucleus sampling for long-form generation and greedy decoding for reasoning tasks, making it a powerful general purpose method for generating text from language models.