Data Science PY

#DataScience,#MachineLearning, #AI, #BigData, #BusinessAnalytics, etc. Community from #Paraguay 🇵🇾
Curated by @rubuntu

[Data Science PY @Data_Science_PY]

🎉 ¡No te pierdas el Asunción AI & Deep Learning Meetup 2025-01 ! 🤖

📅 22/01/2025
⏰ 18:30-20:30
📍 Táva Comedor https://t.co/XkiHCBnw3H

📝 Confirma aquí: https://t.co/mLDdCCSZkF

Auspicia @h2oai

#IA #DeepLearning #DataScience

[Ruben Diaz @rubuntu]

https://t.co/1va7pvRudJ

[Data Science PY @Data_Science_PY]

Asunción Artificial Intelligence and Deep Learning Meetup https://t.co/oiSgkAIjSG #Meetup via @Meetup

[Sebastian Raschka @rasbt]

Phi-3 has "only" been trained on 5x fewer tokens than Llama 3 (3.3 trillion instead of 15 trillion)

Phi-3-mini less has "only" 3.8 billion parameters, less than half the size of Llama 3 8B.

Despite being small enough to be deployed on a phone (according to the technical report), it matches the performance of the much larger Mixtral 8x7B and GPT-3.5. (Phi-3 mini can be quantized to 4-bits, so it only requires ≈ 1.8GB of memory.)

What is the secret sauce? According to the technical report, it's dataset quality over quantity: "heavily filtered web data and synthetic data".

Next to the 4k context-window version, there's also a phi-3-mini-128K model that supports up to 128k tokens.

Fun fact: Phi-3 uses the same tokenizer with a vocabulary size of 32,064 as Llama 2.

[Jeremy Howard @jeremyphoward]

Today at @answerdotai we've got something new for you: FSDP/QDoRA. We've tested it with @AIatMeta Llama3 and the results blow away anything we've seen before.

I believe that this combination is likely to create better task-specific models than anything else at any cost. 🧵

[Data Science PY @Data_Science_PY]

Buenas, si hablan Guarani-Jopara les pido el favor de completar las preguntas que puedan del Google Form para un trabajo Tesis de Maestria en Ciencia de Datos de manera a validar un dataset para un modelo de lenguage Guarani. Favor compartir. https://t.co/CBwvmD3KHf

[Daniel Han @danielhanchen]

Had a look through @Grok's code:
1. Attention is scaled by 30/tanh(x/30) ?!
2. Approx GELU is used like Gemma
3. 4x Layernoms unlike 2x for Llama
4. RMS Layernorm downcasts at the end unlike Llama - same as Gemma
5. RoPE is fully in float32 I think like Gemma
6. Multipliers are 1
7. QKV has bias, O no bias MLP no bias
8. Vocab size is 131072. Gemma 256000.

Sadly, it's way too large for me to accelerate finetuning or running with @UnslothAI. 70b Llama fits in 48GB with Unsloth, but Grok will need way too many GPUs :(

[Yang You @YangYou1991]

Exciting News from Open-Sora! 🚀 They've just made the ENTIRE suite of their video-generation model open source! Dive into the world of cutting-edge AI with access to model weights, comprehensive training source code, and detailed architecture insights. Start building your dream video-generation model today! Check it out 👉 https://t.co/12Fl4ZysIG

[Sebastian Raschka @rasbt]

Grok weights are out under Apache 2.0: https://t.co/9K4IfarqXK

It's more open source than other open weights models, which usual come with usage restrictions.

It's less open source than Pythia, Bloom, and OLMo, which come with training code and reproducible datasets.

[DAIR.AI @dair_ai]

The Top ML Papers of the Week (March 11 - March 17):

- SIMA
- Multimodal LLM Pre-training
- Knowledge Conflicts for LLMs
- Retrieval Augmented Thoughts
- LLMs Predict Neuroscience Results
- LMs Can Teach Themselves to Think Before Speaking
...

[QuantPrep🚀 @quant_prep]

A library of Machine Learning models for Stock price forecasting

A mixture of Deep Learning, Reinforcement Learning and Stacked Models:

- LSTM
- Q-learning agent
- Auto-Encoder + Gradient Boosting

Get it here👇

https://t.co/e9qvy0N8EG

[Bojan Tunguz @tunguz]

I am very pleased to announce that our paper "Deep learning models for predicting RNA degradation via dual crowdsourcing" has been published in Nature Machine Intelligence!

1/7

[Google AI @GoogleAI]

Today on the blog, we’re excited to announce the release of @MLCommons Croissant, a metadata format to make ML datasets more easily discoverable and usable across a wide array of tools and platforms. Learn more and try it today →https://t.co/fdjBDannpx #ml #datasets

[Andrew Ng @AndrewYNg]

New short course: Open Source Models with Hugging Face 🤗, taught by @mariaKhalusova, @_marcsun, and Younes Belkada! @huggingface has been a game changer by letting you quickly grab any of hundreds of thousands of already-trained open source models to assemble into new applications. This course teaches you best practices for building this way, including how to search and choose among models.

You’ll learn to use the Transformers library and walk through multiple models for text, audio, and image processing, including zero-shot image segmentation, zero-shot audio classification, and speech recognition. You'll also learn to use multimodal models for visual question answering, image search, and image captioning. Finally, you’ll learn how to demo what you build locally, on the cloud, or via an API using Gradio and Hugging Face Spaces.

You can sign up here: https://t.co/KavDNQHCCY

[Santiago @svpino]

I recorded a step-by-step tutorial on building a RAG application from scratch.

It's a 1-hour YouTube video where I show you how to use Langchain, Pinecone, and OpenAI.

You'll learn how to build a simple application to answer questions from YouTube videos using an LLM.

But the video is not just about the code (the code is usually the least important thing!) The video is about helping you understand the reason for every component and how to build a chain to solve the problem.

Here is the video: https://t.co/VC1jWTSbur

[Sumanth @Sumanth_077]

Pytorch released GPT-fast!⚡️

This is a simple and efficient implementation of pytorch-native transformer text generation:

Here are some key features:

- Very low latency
- <1000 lines of python
- No dependencies other than PyTorch and sentencepiece
- int8/int4 quantization
- Speculative decoding
- Tensor parallelism
- Supports Nvidia and AMD GPUs

Find link to repository in the following tweet!

[Sumanth @Sumanth_077]

Repo:
https://t.co/lvK9wKinab

[LangChain @LangChain]

Building long context RAG from scratch with RAPTOR + Claude3 (Video)

The rise of long context LLMs + embeddings will change RAG design. Instead of splitting docs + indexing doc chunks, it's feasible to index full docs.

But, there is a challenge to flexibly answer lower-level questions from single docs or higher-level questions that span across many docs.

RAPTOR (@parthsarthi03 et al) is one approach to tackle this by building a tree of doc summaries: docs are clustered and clusters are summarized to capture higher-level information across similar docs.

This is repeated recursively, resulting in a tree of summaries with varying levels of abstraction (docs are leafs, intermediate summaries of related docs in the middle, high-level summaries of docs towards the root).

We tested this on 33 web pages (each ranging 2k - 12k tokens) of LangChain docs using Claude3 model from @AnthropicAI to build the summarization tree.

The pages and tree of summaries are indexed together for RAG with Claude3, enabling QA on lower-lever questions or higher-level concepts (captured in summaries that span related pages).

This idea can scale to large collections of documents or to documents of arbitrary size. Video that walks through code / builds this from scratch is below:

Video:
https://t.co/RUbiu7uIcD

Code:
https://t.co/uHcaP7HdNd

Paper:
https://t.co/7oFEyDdoQE

[LlamaIndex 🦙 @llama_index]

Claude 3 Multimodal Cookbook 🧑‍🍳

Claude is not only good at text, it is very good at visual reasoning. We’ve created a comprehensive guide to using Claude for various multi-modal applications:
✅ Structured Data Extraction
✅ RAG

Claude 3 is able to extract an entire list of stock ticker information from an image - we haven’t seen a lot of other models be able to do this.

Huge thanks to @ravithejads for the work on this.

Check out the cookbook: https://t.co/E2cGP8yVGG

[LangChain @LangChain]

🏓Chain-of-Table

This paper from Google proposes a new framework to do question answering over tabular data

This framework involves a series of prompts, flows, and tool calling... perfect for LangGraph!

s/o @HrubyOnRails for the implementation!

Code: https://t.co/OSsTqVF4Kh