MrCruzPlus

Tech Consultant

[MrCruzPlus @MrCruzPlus]

@CopyRebeldia Bueno, que se queden con el pentágono, y que se inicie un cierre de cuentas de ChatGPT... Iniciare con la mia.

[Miguel Ángel Durán @midudev]

Este repositorio tiene una colección de Agentes de IA, LLMs y MCPs tremenda. Muchos además se pueden usar en local, sin pagar suscripciones.

→ https://t.co/Hbyoy64g9c

[Miguel Ángel Durán @midudev]

¡Crea tu propio modelo de IA! Este repositorio contiene el código para desarrollar, pre-entrenar y ajustar un LLM como GPT desde cero:

https://t.co/wZxr7eheaH

[Victoria Slocum @victorialslocum]

𝗠𝗖𝗣 vs 𝗔𝗴𝗲𝗻𝘁 𝗦𝗸𝗶𝗹𝗹𝘀

(Is one actually better than the other?)

If you've been following the vibe coding movement, you've probably experienced your coding agent not being able to code well with specialized tools or infrastructure. Legacy syntax, wrong parameters, inefficient patterns, leading to tons of debugging that often feels like you're going in circles.

Two solutions have emerged: 𝗠𝗖𝗣 (𝗠𝗼𝗱𝗲𝗹 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗼𝘁𝗼𝗰𝗼𝗹) and 𝗔𝗴𝗲𝗻𝘁 𝗦𝗸𝗶𝗹𝗹𝘀. But which one should you use?

𝗠𝗖𝗣 is pretty much just a standardized API gateway. Your agent makes a deterministic API call with fixed input/output schemas and gets a deterministic response back. It's clean and predictable.

𝗦𝗸𝗶𝗹𝗹𝘀 are different. When an agent uses a skill, it's interpreting natural language instructions about 𝘩𝘰𝘸 to accomplish something. The agent decides which skill to use, when, 𝘢𝘯𝘥 𝘩𝘰𝘸 to execute it. Less "call this function" and more "here's how to think about this problem."

So they 𝗯𝗼𝘁𝗵 have a place in agentic coding. Skills are great for behavioral guidance, MCP works better for the direct infrastructure calls.

We just released our Agent Skills repository to help coding agents create end-to-end AI apps and write better @weaviate_io code. It includes:

1️⃣ 𝗪𝗲𝗮𝘃𝗶𝗮𝘁𝗲 𝗦𝗸𝗶𝗹𝗹𝘀 (/skills/weaviate): Focused scripts for schema inspection, data ingestion, Query Agent integration, and search operations
2️⃣ 𝗖𝗼𝗼𝗸𝗯𝗼𝗼𝗸𝘀 (/skills/weaviate-cookbooks): End-to-end project blueprints for building complete applications (Query Agent chatbots, multivector PDF RAG, various RAG patterns, DSPy agents)

Check it out here: https://t.co/BDSX2b6IOR

[AlgoMaster.io @algomaster_io]

12 must-know data structures for coding interviews:

1. 𝐀𝐫𝐫𝐚𝐲𝐬
2. 𝐌𝐚𝐭𝐫𝐢𝐱 (2𝐃 𝐀𝐫𝐫𝐚𝐲)
3. 𝐋𝐢𝐧𝐤𝐞𝐝 𝐋𝐢𝐬𝐭
4. 𝐒𝐭𝐚𝐜𝐤
5. 𝐐𝐮𝐞𝐮𝐞
6. 𝐇𝐚𝐬𝐡𝐌𝐚𝐩
7. 𝐓𝐫𝐞𝐞
8. 𝐁𝐢𝐧𝐚𝐫𝐲 𝐒𝐞𝐚𝐫𝐜𝐡 𝐓𝐫𝐞𝐞 (𝐁𝐒𝐓)
9. 𝐇𝐞𝐚𝐩 (𝐏𝐫𝐢𝐨𝐫𝐢𝐭𝐲 𝐐𝐮𝐞𝐮𝐞)
10. 𝐓𝐫𝐢𝐞
11. 𝐆𝐫𝐚𝐩𝐡
12. 𝐔𝐧𝐢𝐨𝐧-𝐅𝐢𝐧𝐝 (𝐃𝐢𝐬𝐣𝐨𝐢𝐧𝐭 𝐒𝐞𝐭)

♻️ Repost to help others in your network

[Real Python @realpython]

Python Supply Chain Security Made Easy #python https://t.co/Yyt4YxZQVK

[Python Software Foundation @ThePSF]

🔒 Python security just got stronger! The PSRT now has official governance (PEP 811) and a clear process for growing the team. Interested in joining? Learn more in this blog post from PSF Security Developer in Residence Seth Larson : https://t.co/xSVriZm56E

#python

[Dhanian 🗯️ @e_opore]

Backend Development Roadmap

PHASE 1: PROGRAMMING FUNDAMENTALS
├── Core Programming
│ ├── Data structures (arrays, lists, maps/sets)
│ ├── Algorithms (searching, sorting, complexity)
│ ├── Object-oriented programming
│ └── Functional programming concepts
├── Language Proficiency
│ ├── Choose primary language:
│ │ ├── Java/Spring Boot
│ │ ├── Python/Django/FastAPI
│ │ ├── JavaScript/Node.js/TypeScript
│ │ ├── Go/Golang
│ │ └── C#/.NET Core
│ └── Language-specific patterns & best practices
├── Development Tools
│ ├── Git & GitHub/GitLab
│ ├── IDE (VS Code, IntelliJ, PyCharm)
│ ├── Package managers (npm, pip, Maven)
│ └── Basic command line usage

PHASE 2: WEB FUNDAMENTALS
├── HTTP Protocol
│ ├── Request/Response cycle
│ ├── Methods (GET, POST, PUT, DELETE)
│ ├── Status codes (200, 201, 400, 404, 500)
│ ├── Headers and cookies
│ └── RESTful principles
├── APIs
│ ├── REST API design
│ ├── JSON/XML handling
│ ├── API documentation (OpenAPI/Swagger)
│ ├── API versioning strategies
│ └── Rate limiting and throttling
├── Authentication & Authorization
│ ├── Sessions vs tokens
│ ├── JWT (JSON Web Tokens)
│ ├── OAuth 2.0 / OpenID Connect
│ └── Role-based access control (RBAC)

PHASE 3: DATABASES
├── SQL Databases
│ ├── PostgreSQL / MySQL
│ ├── Database design & normalization
│ ├── Indexes and query optimization
│ ├── ACID properties
│ └── ORMs (Hibernate, SQLAlchemy, Prisma)
├── NoSQL Databases
│ ├── Document DBs (MongoDB)
│ ├── Key-Value stores (Redis)
│ ├── Wide-column stores (Cassandra)
│ └── Graph databases (Neo4j) - optional
├── Database Advanced
│ ├── Transactions and locking
│ ├── Connection pooling
│ ├── Database migrations
│ └── Read replicas and sharding

PHASE 4: BACKEND FRAMEWORKS
├── Framework Mastery (Choose one primary)
│ ├── Spring Boot (Java)
│ │ ├── Dependency injection
│ │ ├── Spring Data JPA
│ │ ├── Spring Security
│ │ └── Spring Cloud (microservices)
│ ├── Django/FastAPI (Python)
│ │ ├── Django ORM/DRF
│ │ ├── FastAPI async
│ │ ├── Pydantic models
│ │ └── Celery for tasks
│ ├── Express/Nest.js (Node.js)
│ │ ├── Middleware patterns
│ │ ├── Dependency injection
│ │ ├── Nest.js modules
│ │ └── TypeORM/Prisma
│ └── .NET Core (C#)
│ ├── Entity Framework
│ ├── https://t.co/0vjTXidSga Core MVC/Web API
│ └── Dependency injection
├── Advanced Concepts
│ ├── Dependency injection
│ ├── Middleware/interceptors
│ ├── Validation and serialization
│ └── Background jobs/queues

PHASE 5: SYSTEM DESIGN
├── Design Principles
│ ├── SOLID principles
│ ├── Design patterns (Factory, Singleton, Observer)
│ ├── Microservices vs Monolith
│ └── Domain-Driven Design (DDD) basics
├── Scalability Patterns
│ ├── Load balancing
│ ├── Caching strategies (Redis, Memcached)
│ ├── CDN integration
│ └── Database scaling (read replicas, sharding)
├── Communication Patterns
│ ├── Synchronous (REST, gRPC)
│ ├── Asynchronous (message queues)
│ ├── Event-driven architecture
│ └── WebSockets for real-time

PHASE 6: DEVOPS & DEPLOYMENT
├── Containerization
│ ├── Docker fundamentals
│ ├── Docker Compose for local dev
│ ├── Dockerfile optimization
│ └── Multi-stage builds
├── Cloud Platforms (Choose one)
│ ├── AWS (EC2, RDS, S3, Lambda, ECS/EKS)
│ ├── Azure (App Service, SQL Database, Functions, AKS)
│ ├── Google Cloud (Compute Engine, Cloud SQL, GKE)
│ └── Serverless concepts
├── CI/CD Pipeline
│ ├── GitHub Actions / GitLab CI / Jenkins
│ ├── Automated testing in pipeline
│ ├── Deployment strategies (blue-green, canary)
│ └── Infrastructure as Code (Terraform, CloudFormation)

PHASE 7: MICROSERVICES & DISTRIBUTED SYSTEMS
├── Microservices Architecture
│ ├── Service decomposition
│ ├── Inter-service communication
│ ├── API Gateway pattern
│ └── Service discovery
├── Message Brokers
│ ├── RabbitMQ / Apache Kafka
│ ├── Pub/Sub patterns
│ ├── Event sourcing
│ └── Message ordering and delivery guarantees
├── Resilience & Observability
│ ├── Circuit breakers (Hystrix, Resilience4j)
│ ├── Distributed tracing (Jaeger, Zipkin)
│ ├── Centralized logging (ELK stack)
│ └── Metrics collection (Prometheus, Grafana)

PHASE 8: ADVANCED TOPICS
├── Performance Optimization
│ ├── Profiling tools
│ ├── Database query optimization
│ ├── Connection pooling
│ └── Garbage collection tuning (language-specific)
├── Security
│ ├── OWASP Top 10
│ ├── Input validation and sanitization
│ ├── SQL injection prevention
│ ├── Encryption (SSL/TLS, data at rest)
│ └── Security headers
├── Testing Strategies
│ ├── Unit testing (JUnit, pytest, Jest)
│ ├── Integration testing
│ ├── End-to-end testing
│ ├── Test doubles (mocks, stubs)
│ └── Test-driven development (TDD)

PHASE 9: SPECIALIZED DOMAINS (CHOOSE 1-2)
├── Real-time Systems
│ ├── WebSocket implementation
│ ├── https://t.co/q2Ckfdr3oa (Node.js)
│ ├── SignalR (.NET)
│ └── Real-time database (Firebase)
├── Search & Analytics
│ ├── Elasticsearch / OpenSearch
│ ├── Full-text search implementation
│ ├── Log aggregation
│ └── Analytics pipelines
├── Machine Learning Integration
│ ├── Model serving (TensorFlow Serving, TorchServe)
│ ├── Feature stores
│ └── ML pipelines
├── FinTech / High-Frequency Systems
│ ├── Low-latency optimization
│ ├── Financial data processing
│ └── High-volume transaction systems

LEARNING RESOURCES
├── Documentation
│ ├── Official framework docs
│ ├── MDN Web Docs
│ └── Cloud provider documentation
├── Practice Platforms
│ ├── LeetCode (algorithms)
│ ├── HackerRank
│ ├── System design practice (https://t.co/NyYdPlvsK7)
│ └── Build your own projects
├── Books
│ ├── Grab The Backend Engineering Handbook:https://t.co/t9mqUuRbjx

PROJECTS TO BUILD
1. REST API with CRUD operations (Blog API, Todo API)
2. E-commerce backend (products, cart, orders, payments)
3. Real-time chat application
4. Microservices-based system (3-4 services)
5. Social media backend with feeds, notifications, messaging
6. Analytics pipeline with data processing

CAREER PATHS
├── Junior Backend Developer (0-2 years)
│ ├── Focus: Framework proficiency, basic DB skills
├── Mid-Level Backend Developer (2-5 years)
│ ├── Focus: System design, distributed systems basics
├── Senior Backend Developer (5+ years)
│ ├── Focus: Architecture decisions, mentoring, cross-team collaboration
├── Specialization Paths
│ ├── Cloud/DevOps Engineer
│ ├── Data Engineer
│ ├── Platform Engineer
│ └── Security Engineer

Estimated Time: 1-2 years for solid foundation, 3-5 years for senior level
Key Principle: Build, deploy, monitor, iterate. Real-world projects > tutorials.

Grab The Backend Engineering Handbook:https://t.co/t9mqUuRbjx

Progression Strategy:
1. Months 1-3: Language + Framework + Basic API
2. Months 4-6: Database + Authentication + Testing
3. Months 7-12: Docker + Cloud + Basic System Design
4. Year 2: Microservices + Advanced Topics
5. Year 3+: Specialization + Architecture

Backend Mindset: Always consider scalability, security, and maintainability from day one. Every line of code should answer "what happens when this gets 1000x more traffic/users/data?"

Get The Backend Engineering Handbook now and begin your journey : https://t.co/t9mqUuRbjx

[Adnan Sharif Lihaz @lihazadn_Ai]

🧵 ChatGPT, Grok, Gemini, Claude & Perplexity — which AI should you actually use?

What each one does best, where it falls short, and how to use it smartly.

No buzzwords. Just real-world clarity.

📌Save this post & dive into the thread 👇

[Dhanian 🗯️ @e_opore]

Docker & Containerization Roadmap

PHASE 1: FUNDAMENTALS & BASICS
├── Container Concepts
│ ├── What are containers vs VMs
│ ├── Containerization benefits
│ ├── Docker architecture
│ └── Container orchestration overview
├── Docker Installation
│ ├── Docker Desktop (Windows/Mac)
│ ├── Docker Engine (Linux)
│ └── Verify installation: `docker --version`
├── Basic Commands
│ ├── `docker run hello-world`
│ ├── `docker ps`, `docker images`
│ ├── `docker pull`, `docker rmi`
│ └── Container lifecycle commands

PHASE 2: DOCKER IMAGES
├── Dockerfile Fundamentals
│ ├── Base images (FROM)
│ ├── Copy files (COPY/ADD)
│ ├── Run commands (RUN)
│ ├── Expose ports (EXPOSE)
│ └── Default command (CMD vs ENTRYPOINT)
├── Building Images
│ ├── `docker build -t myapp:tag .`
│ ├── Multi-stage builds
│ └── Image layers and caching
├── Docker Hub & Registries
│ ├── Push/pull to Docker Hub
│ ├── Private registries
│ └── Image tagging best practices

PHASE 3: CONTAINER MANAGEMENT
├── Container Operations
│ ├── Interactive vs detached mode
│ ├── Port mapping (`-p 8080:80`)
│ ├── Volume mounts (`-v /host:/container`)
│ └── Environment variables (`-e`)
├── Docker Networking
│ ├── Default networks (bridge, host, none)
│ ├── Create custom networks
│ ├── Container-to-container communication
│ └── DNS resolution between containers
├── Docker Storage
│ ├── Volumes (`docker volume create`)
│ ├── Bind mounts
│ └── tmpfs mounts

PHASE 4: DOCKER COMPOSE
├── Compose Basics
│ ├── `docker-compose.yml` structure
│ ├── Services, networks, volumes
│ └── `docker-compose up/down`
├── Multi-Container Applications
│ ├── Web app + database setup
│ ├── Environment variables in Compose
│ └── Dependency management (depends_on)
├── Production Considerations
│ ├── Compose file versions
│ ├── Resource limits
│ └── Health checks

PHASE 5: PRODUCTION & BEST PRACTICES
├── Security
│ ├── Non-root users in containers
│ ├── Image scanning (Trivy, Grype)
│ ├── Secrets management
│ └── Reducing attack surface
├── Optimization
│ ├── Small base images (Alpine)
│ ├── Layer minimization
│ ├── `.dockerignore` file
│ └── Build context optimization
├── CI/CD Integration
│ ├── Docker in GitHub Actions
│ ├── Docker in GitLab CI
│ └── Build automation pipelines

PHASE 6: CONTAINER ORCHESTRATION (KUBERNETES)
├── Kubernetes Fundamentals
│ ├── Pods, Deployments, Services
│ ├── `kubectl` basic commands
│ ├── Minikube / Kind setup
│ └── YAML manifest files
├── Docker & Kubernetes
│ ├── Building images for Kubernetes
│ ├── Private registry integration
│ └── ConfigMaps and Secrets
├── Production Kubernetes
│ ├── Helm charts
│ ├── Ingress controllers
│ └── Monitoring (Prometheus, Grafana)

PHASE 7: ADVANCED TOPICS
├── Docker Swarm (Alternative to K8s)
│ ├── Swarm mode basics
│ ├── Services and stacks
│ └── Overlay networks
├── Cloud Container Services
│ ├── AWS ECS/EKS
│ ├── Azure AKS
│ ├── Google GKE
│ └── Managed container registries
├── Monitoring & Logging
│ ├── Docker logs management
│ ├── Centralized logging (ELK, Loki)
│ └── Monitoring (cAdvisor, Portainer)

PHASE 8: SPECIALIZED SCENARIOS
├── Development Workflows
│ ├── Development containers (Dev Containers)
│ ├── Live reload in containers
│ └── Debugging containers
├── Legacy Application Containerization
│ ├── Monolith to container migration
│ ├── Database in containers (stateful apps)
│ └── Backup strategies
├── Edge Cases
│ ├── Docker on ARM/Raspberry Pi
│ ├── GPU acceleration (NVIDIA Docker)
│ └── Windows containers

LEARNING RESOURCES
├── Official Documentation
│ ├── https://t.co/xNt68NA32C
│ └── https://t.co/yPtRlY2MoJ
├── Practice Platforms
│ ├── Play with Docker (https://t.co/C3Bkr2h6zD)
│ ├── Katacoda (free courses)
│ └── Docker's official tutorial
├── Certifications (Optional)
│ ├── Docker Certified Associate (DCA)
│ └── Certified Kubernetes Administrator (CKA)

PROJECTS TO BUILD
1. Simple static website container
2. Multi-container app (Node.js + Redis + PostgreSQL)
3. CI/CD pipeline with Docker
4. Deploy to Kubernetes cluster
5. Full-stack microservices application

Estimated Time: 3-6 months for comprehensive understanding
Prerequisites: Basic Linux/CLI, Git fundamentals

Progression Tips:
1. Start with Phase 1-2, build simple containers
2. Move to Phase 3-4 for multi-container apps
3. Skip to Phase 6 early for Kubernetes basics
4. Return to Phase 5 for production optimizations
5. Practice daily with real projects

Key Philosophy; Learn by doing - break things, fix them, understand why they broke.

Grab this Docker Handbook; https://t.co/sC9bTrAJWt

[Python Programming @PythonPr]

16 Must Know API Concepts

[Akshay 🚀 @akshay_pachaar]

DevOps vs. MLOps vs. LLMOps:

Many teams are trying to apply DevOps practices to LLM apps.

But DevOps, MLOps, and LLMOps solve fundamentally different problems.

Here's why this matters: 88% of ML initiatives struggled to reach production using traditional DevOps practices.

And LLMs introduce challenges that even MLOps wasn't designed for.

Let's break it down:

→ DevOps is software-centric.

You write code, test it, and deploy it. The feedback loop is straightforward: Does the code work or not?

The primary artifact is code. Testing is deterministic. The tooling is mature after 15+ years.

→ MLOps is (model + data) centric.

Here, you're dealing with data drift, model decay, and continuous retraining.

The code might be fine, but the model's performance can degrade over time because the world changes.

A fraud detection model might work perfectly at launch, then fail within weeks as fraudsters adapt.

The primary artifact expands to code + data + models. You need to version all three. This is why MLflow, DVC, and feature stores became essential.

→ LLMOps is foundation-model-centric.

Here, you're typically not training models from scratch. Instead, you select a foundation model and optimize through three parallel paths:

- Prompt Engineering
- Context/RAG Setup
- Fine-Tuning

Unlike DevOps and MLOps, these paths run in parallel, not sequentially.

But here's what really separates LLMOps: The monitoring is completely different.

In MLOps, you track data drift, model decay, and accuracy.

In LLMOps, you're watching for:

- Hallucination detection
- Bias and toxicity
- Token usage and cost
- Human feedback loops

This is because LLM outputs are non-deterministic. You can't just check if the output is "correct." You need to ensure it's safe, grounded, and cost-effective.

63% of production AI systems experience dangerous hallucinations within their first 90 days.

The cost model also flips.

MLOps costs are training-heavy (GPU hours during development).

LLMOps costs are inference-heavy (every request consumes tokens).

This is why prompt efficiency, caching, and model routing matter so much in LLMOps.

The evaluation loop in LLMOps feeds back into all three optimization paths simultaneously. Failed evals might mean you need better prompts, richer context, OR fine-tuning.

So it's not a linear pipeline anymore.

One more thing: prompt versioning and RAG pipelines are now first-class citizens in LLMOps, just like data versioning became essential in MLOps.

And the ops layer you choose should match the system you're building.

👉 Over to you: What does your LLM monitoring stack look like right now?
_____
Find me → @akshay_pachaar
For more insights and tutorials on AI and Machine Learning!

[TIC Educativas💻📱 @TicEducativas]

¿Sabías que cada IA tiene un "superpoder"? 🧠

✨ No es solo qué pides, sino cómo lo haces.

​De ChatGPT como formador a Gemini como director de proyectos, la clave está en el rol que le asignas. Optimiza tu tiempo y el de tus alumnos con estas rutas de prompting. 👇
​#EdTech #IA #InnovaciónEducativa #TIC #ChatGPT #Gemini
Créditos: Inteligencia Artificial México

[Python Developer @PythonDvz]

Generative AI Project Structure

[Python Programming @PythonPr]

SQL to Python

[MrCruzPlus @MrCruzPlus]

He is so clear, very strongly😅, and that’s what is wasting the old-school American style of interventionist discourse 😂 https://t.co/jiFePfOwNJ

[MJ Kayum @mjkayum]

Nano Banana + N8N = AI Creatives Factory

This AI system creates scroll-stopping visuals at scale using Google's newest image model.

No designers. No agencies. No $50K creative budgets.

Just endless professional-grade ads that look like top brands made them.

Here's how it works:
→ Upload your product catalog to Airtable
→ N8N automation scrapes product details and images
→ Nano Banana creates multiple creative angles for each product
→ System generates different backgrounds, styles, and compositions automatically
→ All variations get organized in Airtable with performance tracking ready

Each visual pennies to generate.
You own 100% of the assets forever.
Runs 24/7 without touching it.

While competitors spend hours in Photoshop or pay agencies thousands per month, you'll be cranking out unlimited variations automatically.

Perfect brand consistency across every creative.

Built 100% in N8N.

Want the complete workflow?
Comment "NANO" + RT + Like

I'll DM you the entire N8N template + Airtable setup
(Must be following so I can DM)

Skip this and keep paying designers $200 per ad variation.

[Ai With Piyas @piyascode9]

Secret AIs That Nobody Knows (Part 2)

1. https://t.co/sbDCxWuam4 – Workflow automation bot
2. https://t.co/M94jpvoUd7 – AI content writer
3. https://t.co/zJxTBxwHqn – Clean, modern slide creation
4. https://t.co/jNN2yZ9hW6 – Remove unwanted objects
5. https://t.co/5XUyF7zA3I – Homework help & solutions
6. https://t.co/uZ8HqR3GJD – Data analysis workspace
7. https://t.co/jZt8m9mBph – Video summarization
8. https://t.co/GYDNdKDpxI – Natural-sounding AI voices
9. https://t.co/IpGSNztTCO – Automatic short-form clips
10. Sloyd AI – 3D asset creator
11. https://t.co/IrKVTS2Gg1 – Code explanation tool
12. Jurny AI – AI travel planner

[ Bookmark for Later 🔖 ]

[Akshay 🚀 @akshay_pachaar]

I have been fine-tuning LLMs for over 2 years now!

Here are the top 5 LLM fine-tuning techniques, explained with visuals:

First of all, what's so different about LLM finetuning?

Traditional fine‑tuning is impractical for LLMs (billions of params; 100s GB).

Since this kind of compute isn't accessible to everyone, parameter-efficient finetuning (PEFT) came into existence.

Before we go into details of each technique, here's some background that will help you better understand these techniques:

LLM weights are matrices of numbers adjusted during finetuning.

Most PEFT techniques involve finding a lower-rank adaptation of these matrices, a smaller-dimensional matrix that can still represent the information stored in the original.

Now with a basic understanding of the rank of a matrix, we're in a good position to understand the different finetuning techniques.

(refer to the image below for a visual explanation of each technique)

1) LoRA

- Add two low-rank trainable matrices, A and B, alongside weight matrices.
- Instead of fine-tuning W, adjust the updates in these low-rank matrices.

Even for the largest of LLMs, LoRA matrices take up a few MBs of memory.

2) LoRA-FA

While LoRA significantly decreases the total trainable parameters, it requires substantial activation memory to update the low-rank weights.

LoRA-FA (FA stands for Frozen-A) freezes matrix A and only updates matrix B.

3) VeRA

- In LoRA, low-rank matrices A and B are unique for each layer.
- In VeRA, A and B are frozen, random, and shared across all layers.
- Instead, it learns layer-specific scaling VECTORS (b and d) instead.

4) Delta-LoRA

- It tunes the matrix W as well, but not in the traditional way.
- Here, the difference (or delta) between the product of matrices A and B in two consecutive training steps is added to W.

5) LoRA+

- In LoRA, both matrices A and B are updated with the same learning rate.
- Authors of LoRA+ found that setting a higher learning rate for matrix B results in better convergence.
____
Find me → @akshay_pachaar
Every day, I share tutorials and insights on ML, LLMs, and AI Engineering.

[Matt Dancho (Business Science) @mdancho84]

🚨NEW: Awesome Generative AI Data Scientist GitHub Repo

LangGraph Ecosystem:

1. Prebuilt Agents
2. AI Data Science Agents
3. LangMem
4. LangGraph Supervisor
5. Open Deep Research
6. LangGraph Reflection
7. LangGraph Big Tool
8. LangGraph CodeAct
9. LangGraph Swarm
10. LangGraph MCP Adapters