Sourav Tripathy

Exploring the roots of reality, one word at a time...

[Sourav Tripathy @EssenceThinker]

Reality was never more tenuous, the abyss never more intoxicating.

[Math Lady Hazel 🇦🇷 @mathladyhazel]

The Fibonacci elephant.

[Elliot Arledge @elliotarledge]

Co-Founder of Cerebras explains their WSE simplified design compared to classical GPUs made by NVIDIA.

[Sourav Tripathy @EssenceThinker]

Do yourself a favour if you are curious...just pick up Shannon's original paper A mathematical Theory of communication....It's great to go into thinking mode with this paper...

[Sourav Tripathy @EssenceThinker]

@gabegreenberg Sent ! https://t.co/wI4TgocuR0

[Sourav Tripathy @EssenceThinker]

Link to read -

https://t.co/h7w0aYr6sB

[Sourav Tripathy @EssenceThinker]

For the past few days I have been thinking about distributed inference of llms over open internet...So I wrote about it defining the constraints and how petals paper attempts to solve it...

[Sourav Tripathy @EssenceThinker]

@dwarkesh_sp latest podcast with @reinerpope ....(I believe this is how technical podcasts should be )...weekend going 😊

[Sourav Tripathy @EssenceThinker]

@dwarkesh_sp @reinerpope It's gonna be one hell of a weekend watching this and grasping all the ideas talked....

[Sourav Tripathy @EssenceThinker]

@sarahookr @a16z @adaption_ai https://t.co/SdiwLyibvi

I wrote about continual learning through a chain of thoughts....Give it a read..may be

[Sourav Tripathy @EssenceThinker]

@KeshavRamji Intresting paper.saw it trending in alphaxiv and read it .Model reasoning using abstract tokens and producing the same level result as verbal COT. The only thing is why cannot we use Token1 and Token0 instead of 64 tokens or will that create problem in RL ...

[Nick Levine @status_effects]

New work with @AlecRad and @DavidDuvenaud:

Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text.

Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

[Sourav Tripathy @EssenceThinker]

To make use of my old nvidia geforce 1650 Ti ...Hosted a 500 M model in Fp16... Inference using Vllm....

Try out - https://t.co/LxGNHO8Nux

[Sourav Tripathy @EssenceThinker]

For the past three days I have been thinking about how llms can be inferenced through public internet something like decentralised version.came across very few papers - petal ,planetserve,bloombee,MDI-LLM and DSSD(Distributed split speculative decoding) ...It's an elegant problem

[Sourav Tripathy @EssenceThinker]

Reading..DeepseekV4 Tech report...

[Sourav Tripathy @EssenceThinker]

After 3 crashes realized that one should not re initialise CUDA in a forked subprocess...always spawn ..in Pytorch...🫩

[Sourav Tripathy @EssenceThinker]

@maharshii

[Sourav Tripathy @EssenceThinker]

For reference I was making a deepseek 1.5B through Ollma to solve a sudoko...via just reasoning...

[Sourav Tripathy @EssenceThinker]

If you are working with a small model say 1.5B and limited memory (I was using ollama) and for a multi turn task small output is not enough and increasing length cause OOM error because ollama uses llama.cpp and no pagedattention for efficient use....any thoughts..?

[Sourav Tripathy @EssenceThinker]

@arpit_bhayani I had a problem , where I used a chunk from a pdf to generate multiple statements and then I needed to have the source location exactly..not the whole chunk..I used jaccard similarity to find from where it was taken out..It's working good as of now

[Andrej Karpathy @karpathy]

One common issue with personalization in all LLMs is how distracting memory seems to be for the models. A single question from 2 months ago about some topic can keep coming up as some kind of a deep interest of mine with undue mentions in perpetuity. Some kind of trying too hard.