Jorge Díaz-Ramírez

Assistant Professor at the @UTarapaca and Phd student in Engineering Sciences in the Computer Science area from @dccuc and @ialab_uc 🤖

[Mayank Pratap Singh @Mayank_022]

𝐕𝐢𝐬𝐮𝐚𝐥 𝐛𝐥𝐨𝐠 on Vision Transformers is live.
https://t.co/N09njkKTXW

Learn how ViT works from the ground up, and fine-tune one on a real classification dataset.

CNNs process images through small sliding filters. Each filter only sees a tiny local region, and the model has to stack many layers before distant parts of an image can even talk to each other.

Vision Transformers threw that whole approach out.

ViT chops an image into patches, treats each patch like a token, and runs self-attention across the full sequence.
Every patch can attend to every other patch from the very first layer. No stacking required.

That global view from layer one is what made ViT surpass CNNs on large-scale benchmarks.

𝐖𝐡𝐚𝐭 𝐭𝐡𝐞 𝐛𝐥𝐨𝐠 𝐜𝐨𝐯𝐞𝐫𝐬:

- Introduction to Vision Transformers and comparison with CNNs
- Adapting transformers to images: patch embeddings and flattening
- Positional encodings in Vision Transformers
- Encoder-only structure for classification
- Benefits and drawbacks of ViT
- Real-world applications of Vision Transformers
- Hands-on: fine-tuning ViT for image classification

The Image below shows

Self-attention connects every pixel to every other pixel at once. Convolution only sees a small local window. That's why ViT captures things CNNs miss, like the optical illusion painting where distant patches form a hidden face.

The architecture is simple. Split image into patches, flatten them into embeddings (like words in a sentence), run them through a Transformer encoder, and the class token collects info from all patches for the final prediction. Patch in, class out.

Inside attention: each patch (query) compares itself to all other patches (keys), softmax gives attention weights, and the weighted sum of values produces a new representation aware of the full image, visualizes what the CLS token actually attends to through attention heatmaps.

The second half of the blog is hands-on code. I fine-tuned ViT-Base from google (86M params) on the Oxford-IIIT Pet dataset, 37 breeds, ~7,400 images.

𝐁𝐥𝐨𝐠 𝐋𝐢𝐧𝐤
https://t.co/N09njkKTXW

𝐒𝐨𝐦𝐞 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬

Dr @sreedathpanat Videos on ViT
ViT paper dissection
https://t.co/sg3JRvcgNG

Build ViT from Scratch
https://t.co/cnHzEeefDA

Original Paper
https://t.co/QiwrlDRQOc

Next up: demystifying Low-Rank Adaptation (LoRA) in PEFT!

Follow me @Mayank_022 along for more deep learning insights, cool fine-tuning projects, and updates from the upcoming blog posts.

[Jorge Díaz-Ramírez @jdiazram]

@GiroBatol Y que pasó con los catetos ?

[Jorge Díaz-Ramírez @jdiazram]

@Enel_Informa_CL Santa victoria todavía sin luz

[Matthias Niessner @MattNiessner]

Want to work on cutting-edge research in #AI?

We have several 𝐏𝐡𝐃 & 𝐏𝐨𝐬𝐭𝐃𝐨𝐜 𝐨𝐩𝐞𝐧𝐢𝐧𝐠𝐬 in our Visual Computing & AI Lab in Munich!

Topics have a strong focus on GenAI, including 3DGs, NeRFs, Diffusion Models, LLMs, etc.

Share & apply :)
https://t.co/HWQhI69CZ0

[Jorge Díaz-Ramírez @jdiazram]

@Enel_Informa_CL Santa Victoria sin luz 🫠

[NVIDIA AI Developer @NVIDIAAIDev]

✨Just announced: Representation agnostics physics simulation.

➡️ https://t.co/hm6Bjev4pk This new framework is now available in the Kaolin Library.

Simulate:
✅Gaussian Splats
✅Neural Radiance Fields
✅Signed Distance Functions and more…

[SkalskiP @skalskip92]

what papers are absolute must-haves if I want to learn vision language models? here's what I have on my list for now:

1. Efficient Estimation of Word Representations in Vector Space (2013)

2. Attention Is All You Need (2017)

3. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)

4. Language Models are Unsupervised Multitask Learners (2018)

5. T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019)

6. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2019)

7. Language Models are Few-Shot Learners (2020)

8. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020)

9. CLIP: Connecting Text and Images (2021)

10. ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision (2021)

11. ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (2021)

12. BEiT: BERT Pre-Training of Image Transformers (2021)

13. LoRA: Low-Rank Adaptation of Large Language Models (2021)

14. Flamingo: a Visual Language Model for Few-Shot Learning (2022)

15. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (2022)

16. LLaMA: Open and Efficient Foundation Language Models (2023)

[khipu.ai @Khipu_AI]

¡Nos vemos en Santiago!
Khipu 2025 se llevará a cabo en la capital de Chile 🇨🇱

[SkalskiP @skalskip92]

OMG. I was waiting for this to happen. Florence-2 is running in browser.

@xenovacom can I run any fine-tuned model or only pre-trained ones?

[CENIA - Centro Nacional de Inteligencia Artificial @cen_ia]

Esta mañana participamos en el evento "GalaxyS24: El impacto de la inteligencia artificial móvil", organizado por @SamsungChile. Desde @cen_ia se sumaron a la conversación: @fran_lucchini, tech lead de transferencia tecnológica y Rodrigo Durán, director ejecutivo. 📱#GalaxyS24

[Geoffrey Hinton @geoffreyhinton]

New paper:
https://t.co/NJyQRsLZ71
Companies are planning to train models with 100x more computation than today’s state of the art, within 18 months. No one knows how powerful they will be. And there’s essentially no regulation on what they’ll be able to do with these models.

[proper @ProperPrompter]

The best inventions of 2023?

@TIME just released their list highlighting the year's top innovations.

The 14 AI tools that made the list:

[Stereolabs @Stereolabs3D]

Learn how to accelerate Smart Robot Navigation with Stereolabs 3D vision and @NVIDIAEmbedded ISAAC sim and NVIDIA ISAAC ROS: https://t.co/nAT4MyRnZj
https://t.co/ehsC5y5RgU

@OpenRoboticsOrg

#ZED #ZEDX #simulation #ROSCon2023 #roscon #goros #ros2 #foxy #Humble

[Sai Vemprala @saihv]

Introducing GRID: the General Robot Intelligence Development platform, designed for prototyping smart and safe robots rapidly using foundation models, LLMs, and simulation.

Paper: https://t.co/vwX330272W
Try now: https://t.co/rXvuYPBPha
GitHub: https://t.co/00sxgUs08m

🧵👇(1/N)

[CENIA - Centro Nacional de Inteligencia Artificial @cen_ia]

🎧Nuestro director de vinculación con el medio, Rodrigo Durán Rojas, será el invitado al primer capítulo del podcast #FuturIA, para hablar de los impactos de la #InteligenciaArtificial y el Índice Latinoamericano de IA elaborado por @cen_ia 🤩

[CENIA - Centro Nacional de Inteligencia Artificial @cen_ia]

#CeniaEnLaPrensa 🗞️Felices de ser parte de este proyecto junto a la @UTarapaca, el cual beneficiará a la región de Arica y Parinacota permitiendo instalar una gran capacidad de cómputo y almacenar grandes bases de datos a través energías limpias.

[Julio Hurtado @JuliousHurtado]

Great opportunity to meet new people, interact with the community and listen to amazing talks!

[Xavi @xamat]

A week ago, Meta presented Chain of Verification (https://t.co/u6zK5BqDaK), that addresses and mitigates hallucination. It is a specific implementation of Reflexion, similar to DRA. I have added it to my hallucination guide https://t.co/kw6mO4XJ4d

[DAIR.AI @dair_ai]

Top ML Papers of the Week (August 21 - August 27):

- Code Llama
- Prompt2Model
- Use of LLMs for Illicit Purposes
- Survey on Instruction Tuning for LLMs
- A Survey on LLM-based Autonomous Agents
- Language to Rewards for Robotic Skill Synthesis
...

[CENIA - Centro Nacional de Inteligencia Artificial @cen_ia]

#IndiceLatamIA 🌎 Es un honor contar hoy con la presencia de Eve Andersson, senior director de Google Alphabet, quien nos presenta la charla magistral "Using AI to build inclusive technology". Sigue en vivo la transmisión de la ceremonia de lanzamiento 👉https://t.co/Eae6heDUGF