Antonio Vidal

Telecom Engineer, Executive MBA and Data Scientist. Developing business oriented Data Apps.

[elvis @omarsar0]

NEW: Google announces Agent2Agent

Agent2Agent (A2A) is a new open protocol that lets AI agents securely collaborate across ecosystems regardless of framework or vendor.

Here is all you need to know:

[Andrej Karpathy @karpathy]

People have too inflated sense of what it means to "ask an AI" about something. The AI are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an AI", think of it more as "asking the average data labeler" on the internet.

Few caveats apply because e.g. in many domains (e.g. code, math, creative writing) the companies hire skilled data labelers (so think of it as asking them instead), and this is not 100% true when reinforcement learning is involved, though I have an earlier rant on how RLHF is just barely RL, and "actual RL" is still too early and/or constrained to domains that offer easy reward functions (math etc.).

But roughly speaking (and today), you're not asking some magical AI. You're asking a human data labeler. Whose average essence was lossily distilled into statistical token tumblers that are LLMs. This can still be super useful ofc ourse. Post triggered by someone suggesting we ask an AI how to run the government etc. TLDR you're not asking an AI, you're asking some mashup spirit of its average data labeler.

[Mckay Wrigley @mckaywrigley]

AI CODE TUTOR PROMPT

—

You are an AI coding instructor designed to assist and guide me as I learn to code. Your primary goal is to help me learn programming concepts, best practices, and problem-solving skills while writing code. Always assume I'm a beginner with limited programming knowledge.

Follow these guidelines in all interactions:
1. Explain concepts thoroughly but in simple terms, avoiding jargon when possible.
2. When introducing new terms, provide clear definitions and examples.
3. Break down complex problems into smaller, manageable steps.
4. Encourage good coding practices and explain why they are important.
5. Provide examples and analogies to illustrate programming concepts.
6. Be patient and supportive, understanding that learning to code can be challenging.
7. Offer praise for correct implementations and gentle corrections for mistakes.
8. When correcting errors, explain why the error occurred and how to fix it.
9. Suggest resources for further learning when appropriate.
10. Encourage me to ask questions and seek clarification.
11. Foster problem-solving skills by guiding me to find solutions rather than always providing direct answers.
12. Adapt your teaching style to my pace and learning preferences.
13. Provide code snippets to illustrate concepts, but always explain the code line by line.
14. Use comments throughout the code to help document what is happening

Address the my questions thoroughly, keeping in mind the guidelines above. If the question is unclear or lacks context, ask me for clarification.

Review the code and provide feedback. If there are errors or areas for improvement, explain them clearly and suggest corrections. If the code is correct, offer praise and explain why it's a good implementation.

Structure your responses as follows:
1. Format your response as markdown
2. Answer my question
3. Code review and feedback
4. Suggestions for further learning or practice

Remember, your goal is not just to help me write correct code, but to help me understand the underlying principles and develop my programming skills. Always strive to be clear, patient, and encouraging in your responses.

[Andrew Ng @AndrewYNg]

I’m excited to kick off the first of our short courses focused on agents, starting with Building Agentic RAG with LlamaIndex, taught by @jerryjliu0, CEO of @llama_index.

This covers an important shift in RAG (retrieval augmented generation), in which rather than having the developer write explicit routines to retrieve information to feed into the LLM context, we instead build a RAG agent that that has access to tools for retrieving information. This lets the agent decide what information to fetch, and enables it to answer more complex questions using multi-step reasoning.

In detail, you'll learn about:
- Routing: Where your agent will use decision-making to route requests to multiple tools.
- Tool Use: Where you'll create an interface for agents to select what tool (function call) to use as well as generate the right arguments.
- Multi-step reasoning with tool use: Where you'll use an LLM to carry out multiple steps of reasoning, while retaining memory throughout the process.

You’ll also learn how to step through what your agent is doing to debug and improve it iteratively.

It’s an exciting time to build agents. Sign up and get started here! https://t.co/sHhzRRJG0l

[Andrej Karpathy @karpathy]

Day 24 of llm.c: we now do multi-GPU training, in bfloat16, with flash attention, directly in ~3000 lines of C/CUDA, and it is FAST! 🚀

We're running ~7% faster than PyTorch nightly, with no asterisks, i.e. this baseline includes all modern & standard bells-and-whistles: mixed precision training, torch compile and flash attention, and manually padding vocab. (Previous comparisons included asterisks like *only inference, or *only fp32 etc.) Compared to the current PyTorch stable release 2.3.0, llm.c is actually ~46% faster. My point in these comparisons is just to say "llm.c is fast", not to cast any shade on PyTorch. It's really amazing that PyTorch trains this fast in a fully generic way, with ability to cook up and run ~arbitrary neural networks and run them on a ton of platforms. I see the goals and pros and cons of these two projects as different, even complementary. Actually I started llm.c with my upcoming education videos in mind, to explain what PyTorch does for you under the hood.

How we got here over the last ~1.5 weeks - added:

✅ mixed precision training (bfloat16)
✅ many kernel optimizations, including e.g. a FusedClassifier that (unlike current torch.compile) does not materialize the normalized logits.
✅ flash attention (right now from cudnn)
✅ Packed128 data structure that forces the A100 to utilize 128-bit load (LDG.128) and store (STS.128) instructions.

It's now also possible to train multi-GPU - added:
✅ First version of multi-gpu training with MPI+NCCL
✅ Profiling the full training run for NVIDIA Nsight Compute
✅ PR for stage 1 of ZeRO (optimizer state sharding) merging imminently

We're still at "only" 3,000 lines of code of C/CUDA. It's getting a bit less simple, but still bit better than ~3 million. We also split off the fp32 code base into its own file, which will be pure CUDA kernels only (no cublas or cudnn or etc), and which I think would make a really nice endpoint of a CUDA course. You start with the gpt2.c pure CPU implementation, and see how fast you can make it by the end of the course on GPU, with kernels only and no dependencies.

Our goal now is to create a reliable, clean, tested, minimal, hardened and sufficiently optimized LLM stack that reproduces the GPT-2 miniseries of all model sizes, from 124M to 1.6B, directly in C/CUDA.

A lot more detail on: "State of the Union [May 3, 2024]"
https://t.co/eDgbngHrZ9

[Virat Singh @virattt]

Exploring LLM Pricing 💰

I updated my table to include llama 3.

The table now has 4 pricing tiers:

• tier 1 starts at $0.25
• tier 2 starts at $12.00
• tier 3 starts at $24.00
• tier 4 starts at $42.00

To get "total cost", I combine input cost and output cost.

The total cost assumes a 3 input token to 1 output token ratio for any given LLM call.

Tier 1 models are fastest with best pricing. Llama 3 70B by @GroqInc is my favorite in this tier. I use the LLM for query extraction because it is fast, correct, and really well priced.

Tier 2 models are fast at slightly higher cost and complexity. DBRX Instruct from @databricks is the intriguing newcomer here.

Tier 3 models yield much higher complexity. Latency remains excellent. Tradeoff is cost. Command r+ from @cohere is my main choice. I use the model primarily for its excellent tool calling ability.

Tier 4 models are the crème de la crème. Opus from @AnthropicAI is a beast. It is my one and only coding companion.

[Vaibhav (VB) Srivastav @reach_vb]

Give it a read here: https://t.co/JKUUp3tf62

[Maria Khalusova @mariaKhalusova]

I put together a quick colab notebook using Llama-3-8B-Instruct for chatting with a PDF.
Unstructured API for partitioning and chunking a large PDF file, FAISS for vector storage, @langchain for RAG, and quantized Llama-3-8B-Instruct (so that it fits on free Colab's GPU).
This was fun!
https://t.co/y3oTSGpW5G

[Yann LeCun @ylecun]

🥁 Llama3 is out 🥁
8B and 70B models available today.
8k context length.
Trained with 15 trillion tokens on a custom-built 24k GPU cluster.
Great performance on various benchmarks, with Llam3-8B doing better than Llama2-70B in some cases.
More versions are coming over the next few months.

https://t.co/EkU9aIHdZE

[Optare Solutions @optaresolutions]

📰Finetwork confía en la experiencia de Optare Solutions para la transformación de sus sistemas OSS

🗣Carlos Valero (CTO @finetwork): “Optare nos ha demostrado su valor, facilitándonos la ejecución de un proyecto complejo”.

👇🏼Noticia completa aquí: https://t.co/Em2RNbhv0z

[the tiny corp @__tinygrad__]

If anyone has an ML project that:
* is open source
* uses tinygrad
* is ready for more compute

We can get you ssh to tinybox today as a grant. DMs open to apply.

[Carlos Santana @DotCSV]

🔴 NUEVO MODELO DE OPENAI!!!!!

LOCURA! OpenAI acaba de sacar su primer modelo de generación de vídeo a través de texto y acaba de arrasar cualquier otro modelo que hayamos visto NUNCA.

ESTO ES NEXT-LEVEL 🤯

[Cohere Labs @Cohere_Labs]

Today, we’re launching Aya, a new open-source, massively multilingual LLM & dataset to help support under-represented languages. Aya outperforms existing open-source models and covers 101 different languages – more than double covered by previous models.

https://t.co/0WsC2i9C8a

[Arjun Ashok @arjunashok37]

Foundation models are well-established in vision and language, but time series forecasting has lagged behind - it still relies on dataset-specific models.

Meet Lag-Llama: the first open-source foundation model for time series forecasting!

[Rishabh Srivastava @rishdotblog]

Two big updates today!

We updated the weights for sqlcoder-7b-2, and it now outperforms GPT-4 for most SQL queries – specially if you give it the right instructions and prompt well

@huggingface link here: https://t.co/8TeOBdu5Rf

2) We've added basic instruction following capabilities, including the ability to say "I don't know" when a question cannot be confidently answered given a database schema. More on that here! https://t.co/v3w8lwqUsr

3) We've got a Colab notebook up if you want to explore the model (though just with a single beam, for now): https://t.co/KkHuhR4Pir

[Junyang Lin @JustinLin610]

🎉Happy to announce the release of Qwen1.5! This time, we directly opensource new models of 6 sizes, 0.5B, 1.8B, 4B, 7B, 14B, and 72B (including base, chat, AWQ, GPTQ, GGUF)! From small to huge!

Blog: https://t.co/XH2KuHSKDb
GitHub: https://t.co/vRLJukyWlQ
HF: https://t.co/Ct2TVyOT1A
ModelScope: https://t.co/2ZEE2p0uVm

This time, except for the model quality improvement (check @huybery 's tweet), we also have:

Qwen2 codes are integrated to 🤗Hugging face transformers (since 4.37.0) , and from now on, we don’t need `trust_remote_code` anymore.

Additionally we’ve collaborated with frameworks like:
🔧vLLM, SGLang for deployment;
🌟AutoAWQ, AutoGPTQ for quantization;
🚀Axolotl, LLaMA-Factory for finetuning;
💻 llama.cpp, MLX for local LLM inference;
…
and the Qwen1.5 series is available on platforms such as Ollama and LMStudio!

API services are offered not only on DashScope but also on Together, with global accessibility. Visit https://t.co/flTlbPjIuR to get started, and we recommend trying out Qwen1.5-72B-chat!

We hope this time you will find Qwen1.5 easier to use and enjoy the model quality!

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

Me: Can you draw a very normal image?

ChatGPT: Here is a very normal image depicting a tranquil suburban street scene during the daytime.

Me: Not bad, but can you go more normal than that?

(cont.)

[Jeff Dean @JeffDean]

I’m very excited to share our work on Gemini today! Gemini is a family of multimodal models that demonstrate really strong capabilities across the image, audio, video, and text domains. Our most-capable model, Gemini Ultra, advances the state of the art in 30 of 32 benchmarks, including 10 of 12 popular text and reasoning benchmarks, 9 of 9 image understanding benchmarks, 6 of 6 video understanding benchmarks, and 5 of 5 speech recognition and speech translation benchmarks. Gemini Ultra is the first model to achieve human-expert performance on MMLU across 57 subjects with a score above 90%. It also achieves a new state-of-the-art score of 62.4% on the new MMMU multimodal reasoning benchmark, outperforming the previous best model by more than 5 percentage points.

Gemini was built by an awesome team of people from @GoogleDeepMind, @GoogleResearch, and elsewhere at @Google, and is one of the largest science and engineering efforts we’ve ever undertaken. As one of the two overall technical leads of the Gemini effort, along with my colleague @OriolVinyalsML, I am incredibly proud of the whole team, and we’re so excited to be sharing our work with you today!

There’s quite a lot of different material about Gemini available, starting with:

Main blog post: https://t.co/NzSycJl7aE

60-page technical report authored by th Gemini Team: https://t.co/CEdMRyYSLo

In this thread, I’ll walk you through some of the highlights.

[Alamin @iam_chonchol]

ChatGPT was the beginning.

GPTs will make you a millionaire

Here are 21 GPTs that will blow your mind 🧵:

(7th is my favorite)

[ 🔖 Bookmark for later ]

[Antonio Vidal @avidalvi]

@getmanfred Analizar la edad de los contratados tiene sesgo de supervivencia y tampoco tiene en cuenta la experiencia necesaria para el trabajo. Habría que analizar la edad de los candidatos que habéis presentado vs la persona contratada.