In the post-AlphaFold era, drug discovery is on a tear
—Genome-wide virtual screening @ScienceMagazine
https://t.co/GPyjSK6PcM
—Transforming the industry in many ways @TheEconomist https://t.co/ppAMkbFqMp
Another banger post from Anthropic!
It's all about improving your agents via evals.
Here are my quick takeaways from the blog:
The capabilities that make agents useful (autonomy, intelligence, flexibility) are the same ones that make them hard to evaluate. You can't just run unit tests and expect your agentic app to work.
This guide breaks down the practical framework Anthropic devs use for agent evals.
They mentioned three types of graders, each with trade-offs:
- Code-based graders are fast, cheap, and reproducible, but brittle to valid variations.
- Model-based graders handle nuance and open-ended tasks, but are non-deterministic and require human calibration.
- Human graders are gold-standard quality, but expensive and slow.
They also talk about two categories of evals that serve different purposes.
1) Capability evals ask "what can this agent do well?" and start at low pass rates.
2) Regression evals ask "can it still handle previous tasks?" and should stay near 100%. Tasks graduating from capability to regression represent real progress.
For non-determinism, two metrics matter. pass@k measures the probability of at least one success in k attempts. pass^k measures the probability that all k trials succeed. These diverge dramatically, at k=10, pass@k can approach 100% while pass^k falls to near zero.
A really good tip in the blogs is to start with 20-50 simple tasks from real failures rather than waiting for perfection. Convert manual checks you already perform into test cases. Grade outputs, not paths taken. Include partial credit for complex tasks.
Common pitfalls include rigid grading that penalizes equivalent but differently-formatted answers, ambiguous task specifications, and stochastic tasks impossible to reproduce.
I highly recommend this read.
Blog: https://t.co/ZyH2mwKVqp
Learn to build effective AI agents in our academy: https://t.co/JBU5beIoD0
Stop wasting hours trying to learn AI. 📘📚
I have already done it for you.
With one list. Zero confusion. And no fluff
📹 Videos:
1. LLM Introduction: https://t.co/OLkDaY7H7g
2. LLMs from Scratch: https://t.co/2N91lxtSh1
3. Agentic AI Overview (Stanford): https://t.co/4xX3JiNUlm
4. Building and Evaluating Agents: https://t.co/paoZnUdySm
5. Building Effective Agents: https://t.co/TICgQVUFCk
6. Building Agents with MCP: https://t.co/EPKu3G7Fms
7. Building an Agent from Scratch: https://t.co/U3XSK0Qx85
8. Philo Agents: https://t.co/wGNERv8TpH
🗂️ Repos
1. GenAI Agents: https://t.co/mmvbJpOSlG
2. Microsoft's AI Agents for Beginners: https://t.co/zQLhaT3Wpl
3. Prompt Engineering Guide: https://t.co/wYFPPKX4H1
4. Hands-On Large Language Models: https://t.co/92ZZJh6MXG
5. AI Agents for Beginners: https://t.co/zQLhaT3Wpl
6. GenAI Agentshttps://lnkd.in/dEt72MEy
7. Made with ML: https://t.co/gvzfW6aKnf
8. Hands-On AI Engineering:https://t.co/DHTvNbpMVk
9. Awesome Generative AI Guide: https://t.co/jfjbl5CQvA
10. Designing Machine Learning Systems: https://t.co/KIV2X9uKQs
11. Machine Learning for Beginners from Microsoft: https://t.co/Gdl4oZxQEY
12. LLM Course: https://t.co/59QVZqGED0
🗺️ Guides
1. Google's Agent Whitepaper: https://t.co/Psy7vqkVR2
2. Google's Agent Companion: https://t.co/sdw5mZhsOM
3. Building Effective Agents by Anthropic: https://t.co/euyR1t8Dg0.
4. Claude Code Best Agentic Coding practices: https://t.co/psPsyWWMwH
5. OpenAI's Practical Guide to Building Agents: https://t.co/67B9XaLolZ
📚Books:
1. Understanding Deep Learning: https://t.co/j5xz5XfkFL
2. Building an LLM from Scratch: https://t.co/TX8BJPpxX0
3. The LLM Engineering Handbook: https://t.co/rSWCiuvZYW
4. AI Agents: The Definitive Guide - Nicole Koenigstein: https://t.co/m3Tq3vE6Go
5. Building Applications with AI Agents - Michael Albada: https://t.co/QM4mYlYdWh
6. AI Agents with MCP - Kyle Stratis: https://t.co/sIqEe430D6
7. AI Engineering: https://t.co/hs4bN4xFAV
📜 Papers
1. ReAct: https://t.co/k3tFy7pbvR
2. Generative Agents: https://t.co/yQp1Tjl94i.
3. Toolformer: https://t.co/RKBauj9GVX
4. Chain-of-Thought Prompting: https://t.co/fVqbr68wvt.
🧑🏫 Courses:
1. HuggingFace's Agent Course: https://t.co/OYgR6C8HNd
2. MCP with Anthropic: https://t.co/n5nnFc9DUC
3. Building Vector Databases with Pinecone: https://t.co/MLhJnfbdGG
4. Vector Databases from Embeddings to Apps: https://t.co/31zEvnwtfZ
5. Agent Memory: https://t.co/ldYc6RVmMj
Repost for your network ♻️
LLM agents break down on long tasks.
This is where context engineering really matters.
Agents can reason and use tools, but extended operations cause unbounded context growth and accumulated errors.
Common fixes like context compression or retrieval-augmented prompting force trade-offs between information fidelity and reasoning stability.
This new research introduces InfiAgent, a framework that keeps the agent's reasoning context strictly bounded regardless of how long the task runs.
The idea is externalize persistent state into a file-centric abstraction. Instead of cramming everything into context, the agent maintains a workspace of files that persist across steps. At each decision point, it reconstructs context from a workspace state snapshot plus a fixed window of recent actions.
This decouples task duration from context size. Whether the task takes 10 steps or 1000, the reasoning context stays the same length.
This is nice because the approach requires no task-specific fine-tuning. The agent operates the same way regardless of the domain.
Experiments on DeepResearch and an 80-paper literature review task show that InfiAgent with a 20B open-source model is competitive with larger proprietary systems. It maintains substantially higher long-horizon coverage than context-centric baselines.
The 80-paper literature review is particularly telling. That's exactly the kind of extended task where traditional agents accumulate errors and lose track of what they've done. InfiAgent's file-based state externalization prevents this degradation.
Paper: https://t.co/U0rGJf457x
Learn to build effective AI agents in our academy: https://t.co/JBU5beHQNs
Of >105,000 participants with 30-year follow-up, only 9.3% achieved healthy aging (age 70, w/o any chronic diseases). Their diet was significantly associated with this outcome🧵 @NatureMedicine
2024 FDA approvals https://t.co/mEpJ4Oc4f4
The FDA approved 50 new therapeutics in 2024, with green lights for a novel schizophrenia drug, a first MASH medicine and much more - find out about them in our annual round-up
The latest large language of life #AI model (LLLM) predicts the impact of coding and non-coding DNA variants with scores for all ~9 billion possible single nucelotide variants, from whole genomes across multiple species https://t.co/dco0qCpDYb @NatureBiotech@yun_s_song@gsbenegas@cralbors@chengzhong_ye@UCBerkeley