Stan

Machine learning engineer

[Stan @stanqin]

@wangzhian8848 评论区好像很多人看不懂这种表达手法，可能他们是在国外呆太久了

[Vaishnavi @_vmlops]

EFFICIENT LLM INFERENCE - The interview pocket notes you actually need

https://t.co/zNNU5EBTQv

[Stan @stanqin]

@polentafries @anblk984 有趣的是，https://t.co/fBZdgX5Mix虽然火在国外，但其实是上海公司

[Stan @stanqin]

@andy88988696 是真的。而且当时只有Uber司机这么干。当时uber和滴滴竞争，细节原因忘记了，不一定是你说的挣取消费，就算是也是平台补贴的取消费

[Stan @stanqin]

Code Memphis. Initiate!

[Stan @stanqin]

@Novahyphenz @slipknothooh @pangyusio 有很多回帖不知道按字母序排名的存在也是挺可怜的，受 KPI 制度毒害太深

[François Fleuret @francoisfleuret]

My deep learning course @unige_en is available on-line. 1000+ slides, ~20h of screen-casts. Full of examples in @PyTorch.

https://t.co/6OVyjPdwrC

And my "Little Book of Deep Learning" is available as a phone-formatted pdf (400k downloads!)

https://t.co/qXni5GZOMT

[PyQuant News 🐍 @pyquantnews]

I think I *finally* get it:

[Stan @stanqin]

@minux302 ERROR: unknown command "insatll" 召喚失敗だ

[Stan @stanqin]

#TGA2024 #GOTY goes to Astro Bot

[Stan @stanqin]

Everything is smooth.

[Akshay 🚀 @akshay_pachaar]

Traditional RAG vs. HyDE, visually explained!

One critical problem with the traditional RAG system is that questions are not semantically similar to their answers.

Consider the example where you want to find a sentence similar to "What is ML?" It is likely that "What is AI?" will appear more similar than "Machine learning is fun."

This semantic dissimilarity leads to several irrelevant contexts being retrieved during the retrieval step.

HyDE solves this issue.

The visual below illustrates how this approach differs from traditional RAG.

Here's how it works:

- Use an LLM to generate a hypothetical answer (H) for the query (Q). This answer doesn't have to be entirely correct.

- Embed the answer using a contriever model to obtain E. Bi-encoders trained with contrastive learning are generally used for this purpose.

- Use the embedding E to query the vector database and retrieve relevant context (C).

- Pass the hypothetical answer H, the retrieved context C, and the query Q to the LLM to produce a final answer.

Done!

Now, of course, the hypothetical generated will likely contain hallucinated details.

But this does not severely affect the performance due to the contriever model—one which embeds.

More specifically, this model is trained using contrastive learning and it also functions as a near-lossless compressor whose task is to filter out the hallucinated details of the fake document.

This produces a vector embedding that is expected to be more similar to the embeddings of actual documents than the question is to the real documents.

Several studies have shown that HyDE improves the retrieval performance compared to the traditional embedding model.

But this comes at the cost of increased latency and more LLM usage.

Enjoyed this? You should also my RAG series! From building and optimizing RAG apps to evaluating performance and crafting agentic & multi-modal systems—it's all here.

Link in the next tweet!
_____
Find me → @akshay_pachaar ✔️
For more insights and tutorials on AI and Machine Learning!

[Stan @stanqin]

A friend in #OpenAI said #AGI will be achieved in 2027. Let's wait and see.

[Stan @stanqin]

Indulging in Dover's math books ... simply amazing.

[Stan @stanqin]

Focus.

[Adam Silverman (Hiring!) 🖇️ @adamsilverman]

Here is everything that happened in AI Agents this week 🧵 

(save for later)

[Stan @stanqin]

https://t.co/Vck4H0scLR a must read.

[Stan @stanqin]

Embodied learning

[Alex Reibman 🖇️ @AlexReibman]

OpenAI’s biggest competitor just gave AI the ability to control computers

We gave 250+ hackers 24 hours to see what it’s capable of

Here’s what we saw at the Nexgen Computer Use Agents Hackathon w/ @AnthropicAI + @AgentOpsAI @notablecap at @AGIHouseSF (🧵):

[Daniel Han @danielhanchen]

Vision finetuning is finally in🦥@UnslothAI! It took a while, but Llama 3.2 Vision, Pixtral, Qwen2 VL & all Llava variants now work!

1. QLoRA / LoRA is 1.3x to 2x faster for each
2. 30-70% less VRAM usage
3. 3 examples - Radiography, LaTeX, Q&A

Extra stuff:
1. Pixtral chat template in HF conversion now fixed. Added SDPA to Pixtral. Pixtral finetuning also fits in a free 16GB Colab with Unsloth! Also uploaded HF format for Pixtral Base to Unsloth - https://t.co/cRaIe9FRIo

2. Qwen VL had only partial gradient checkpointing support - adding it reduces VRAM usage from 24GB to ~3GB for Qwen VL 2B.

3. Added selective LoRA - select either the language and/or vision parts to finetune. Also select the MLP/attention layers.

4. Uploaded 4bit bitsandbytes quants for Llama 3.2 11B, 90B, Qwen2 VL 2B, 7B, 72B, Pixtral 12B & Base, Llava 1.5 Vicuna and 1.6 Mistral to https://t.co/gHMS1CeFLF

5. Native long context support for vision finetuning! Unsloth can finetune Qwen 72B in < 48Gb VRAM and Llama 3.2 90B in 1x 80GB card!

6. Added merging to 16bit directly for vision to export to vLLM for inference. Ollama also works for Llama vision!

Update Unsloth via `pip install --upgrade unsloth unsloth_zoo` to get the latest updates!

Github: https://t.co/xhCdD7ha6Q
Radiography Llama 3.2 Colab: https://t.co/7gKo83cFRE
OCR LaTeX Qwen2 VL Colab: https://t.co/hNYRtwoKWg
General Q&A Pixtral Colab: https://t.co/5lYP3eVlrZ
Blog: https://t.co/mThzqdmnVg

[Stan @stanqin]

A master is who knows when to sell but not when to buy