8/8 Evaluating agents
Demystifying evals for AI agents by Anthropic. Tasks, trials, graders, and transcripts, and how to measure reliability over several runs.
https://t.co/fEo60AEhVS
8 free resources to learn LLM agents, one for each step from agent foundations to evaluating agents.
All eight come from Chapter 19 of LLM Roadmap, now out in its 3rd edition. One resource per post below.
7/8 Human-in-the-loop
Interrupts, LangChain docs. How to pause an agent, save its state, wait for a person to approve or edit, and resume.
https://t.co/cFdkQfhEEW
🟢 New Free Workshop Announcement | Building a Unified Data Layer for AI Agents by Farhan Syah, founder of NodeDB
Vector search helps AI agents find relevant information, but building a useful agent often requires more than similarity search. Agents also need access to structured records, relationships between entities, conversation history, and persistent state across sessions.
Supporting these needs can mean connecting several databases and writing code to keep them synchronized. As the application grows, managing that data infrastructure becomes a significant engineering challenge.
Join Farhan Syah, founder of NodeDB, for a free technical workshop hosted by To Data & Beyond, exploring how a unified data layer can support the retrieval, memory, and context requirements of AI agents.
📅 Date & Time: Saturday, 31st Oct, 16:00-17:00 UTC
📍 Registration link in the comments
Yesterday we hosted @Andrey__HQ, co-founder of AgentWeb, for an expert talk on how AI agents can act on sites like LinkedIn, Amazon, and DoorDash without driving a browser.
The full 45-minute recording, with the live demos and the Q&A, is now on the To Data & Beyond YouTube channel.
Andrey explained why screenshot-based browser agents are slow and expensive, then showed AgentWeb working at the network-request level. In the live demo, Claude collected 661 people and company pages that engaged with a founder's last six LinkedIn posts into one spreadsheet and sent her a connection request, using 23 AgentWeb calls for about $1.15.
He then connected AgentWeb to Claude as a custom connector live and walked through the session handling, CAPTCHAs, and residential IPs behind it.
If you are building agents that need live web data, watch the LinkedIn demo and compare its cost and run time with your current browser-agent setup.
Also, make sure to check out our upcoming free workshop with Farhan Syah, in which he explains "Building a Unified Data Layer for AI Agents"
Link in the comments!
🔵 New Video Course Announcement | Context Engineering for AI Agents
We spent the last two months building, running, and measuring AI agents. This course covers how to design, build, and evaluate what the model sees at each step of an agent’s run, implemented with the Claude Agent SDK.
The course has eight video lessons and comes with a 165-page book, an 186-slide deck, two code repositories, a hands-on assignment, and a certificate of completion.
Link in the comments!
🔵 New Blog: A Practical Roadmap to Forward Deployed Engineering in AI by @YoussefHosni951
Forward Deployed Engineering is becoming a more visible career path in enterprise AI, with companies such as OpenAI, Anthropic, Palantir, and Cursor hiring for the role.
As businesses move from AI experiments to production deployments, they need engineers who can understand customer problems, integrate models with existing systems, and make those solutions reliable enough for everyday use.
This blog explains what the role involves and how to start building the skills it requires. It covers twelve areas through a practical roadmap, with suggested resources and a running project that connects customer discovery, implementation, evaluation, and operational handoff.
Read it for free (link in the comments!)