Data Platform at Doordash. Previously at Uber and Netflix Cloud Platform. Interested in Data scale challenges and managing engineering excellence. Hiring now!
Excited to announce that @PierreB80788038, creator of ZeroFS is joining @tensorlake to help us build an elastic file storage on top of object store for sandboxes!
This would enable sandboxes to have a shared file system between them.
It was great hanging out with all the meetup attendees and doing a demo of Multi Agent Observability and Governance. Good talks from Google Gemini and Modal as well. Read more at https://t.co/rdcPvPucqR
The NYC AI engineering community showed up last night for our AI engineering meetup at Google's office, featuring cutting-edge demos and production agent stories. Thank you to everyone who attended or expressed interest ๐ช
The energy in the room proved the demand for real conversations about what it takes to ship agents at scale. Thank you again to our speakers:
๐ฃ๏ธ โLukas Geiger - Cloud Customer Engineer @ Google ๐ฃ๏ธ Christopher Page - Applied AI @ Google
๐ฃ๏ธ @jvmncs - FDE @modal
๐ฃ๏ธ @stonse - Head of Engineering @ Galileo
And a special thanks to @Google NYC for hosting us, and Kashaf Mazhar + our team members, @bigal123 and Jim LeVan, for organizing the event. Stay tuned for more community events soon ๐
Every agent your team ships has its own hardcoded guardrails, its own bespoke logic, its own failure modes. That's not governance. These brittle controls soon become a liability.
Galileo is proud to announce the open-source launch of Agent Control ๐
Agent Control is the open-source control plane that solves for the open, centralized governance needs for all your AI Agents.
๐ฌ "We've had a front-row seat to agent development at Fortune 500 and digital-native companies. They have been struggling to hard-code safety rules and controls into each agent which makes them brittle. With Agent Control, developers can now create policies in one place and then use those to enforce guardrails everywhere." โ @YashSheth46, Co-founder & CTO, Galileo
Agent Control integrates seamlessly with all your agents using the @ control hook or just by leveraging our native integrations with some of the leading agent frameworks.
No redeployment. No code changes. No vendor lock-in.
๐ฌ โCentralized management of policies can help organizations to manage AI agent behaviors. A unified control plane and centralized governance of agents can help organizations efficiently deploy AI agents at scale. Organizations that embrace eval engineering as a core competency will shorten the time to value for their AI investments. By taking a lifecycle approach, organizations can achieve a continuous improvement loop for AI systems.โ โ Tim Law, @IDC Research Director, AI and Automation
Agent Control is already backed by partners including @awscloud, @Cisco AI Defense, @crewAIInc, @glean, @ServiceNow, and @rubrikInc, and it works with the guardrail providers you already use, from our Luna models to NVIDIA NeMo or AWS Bedrock.
The repo is live, built in the open with contributions from some of the largest AI infrastructure companies in the world, try it out today: https://t.co/Abib0Txnon
Watch Yash walk through how it works in the video below, and check the comments for links to our launch webinar, announcement blog, and full press release. ๐
@rungalileo As @YashSheth46 explains, a control plane for Agent governance is a need that will become increasingly crucial.
Proud of the teamโs work on this and hopefully will be well received by the OSS community
Love this framing!
When it comes to building Agentic Systems, making it Eval driven surely pays off strong dividends in ensuring you have built it right and can continue to monitor your investment using solid metrics and alerts.
Iย have been developing Agentic Systems for the past few years and the same patterns keep emerging. ๐
๐๐๐ฎ๐น๐๐ฎ๐๐ถ๐ผ๐ป ๐๐ฟ๐ถ๐๐ฒ๐ป ๐๐ฒ๐๐ฒ๐น๐ผ๐ฝ๐บ๐ฒ๐ป๐ is the most reliable way to be successful in building your ๐๐ด๐ฒ๐ป๐๐ถ๐ฐ ๐ฆ๐๐๐๐ฒ๐บ๐ and continue improving them - here is my template.
Letโs zoom in:
๐ญ. Define a problem you want to solve: is GenAI even needed?
๐ฎ. Build a Prototype: figure out if the solution is feasible.
๐ฏ. Define Performance Metrics: you must have output metrics defined for how you will measure success of your application.
๐ฐ. Define Evals: split the above into smaller input metrics that can move the key metrics forward. Decompose them into tasks that could be automated and move the given input metrics. Define Evals for each. Store the Evals in your Observability Platform.
โน๏ธ Steps ๐ญ. - ๐ฐ. are where AI Product Managers can help, but can also be handled by AI Engineers.
๐ฑ. Build a PoC: it can be simple (excel sheet) or more complex (user facing UI). Regardless of what it is, expose it to the users for feedback as soon as possible.
๐ฒ. Instrument your application: gather traces and human feedback and store it in an Observability Platform next to previously stored Evals.
๐ณ. Run Evals on traced data: traces contain inputs and outputs of your application, run evals on top of them.
๐ด. Analyse Failing Evals and negative user feedback: this data is gold as it specifically pinpoints where the Agentic System needs improvement.
๐ต. Use data from the previous step to improve your application - prompt engineer, improve AI system topology, finetune models etc. Make sure that the changes move Evals into the right direction.
๐ญ๐ฌ. Build and expose the improved application to the users.
๐ญ๐ญ. Monitor the application in production: this comes out of the box - you have implemented evaluations and traces for development purposes, they can be reused for monitoring. Configure specific alerting thresholds and enjoy the peace of mind.
โ ๐๐ผ๐ป๐๐ถ๐ป๐๐ผ๐๐ ๐๐ฒ๐๐ฒ๐น๐ผ๐ฝ๐บ๐ฒ๐ป๐ ๐ผ๐ณ ๐๐ผ๐๐ฟ ๐ฎ๐ฝ๐ฝ๐น๐ถ๐ฐ๐ฎ๐๐ถ๐ผ๐ป:
โก๏ธ Run steps ๐ฒ. - ๐ญ๐ฌ. to continuously improve and evolve your application.
โก๏ธ As you build up in complexity, new requirements can be added to the same application, this includes running steps ๐ญ. - ๐ฑ. and attaching the new logic as routes to your Agentic System.
โก๏ธ You start off with a simple Chatbot and add a route that can classify user intent to take action (e.g. add items to a shopping cart).
Join me in a free webinar this Friday to learn how LLMOps patterns fit into this picture: https://t.co/gNy4ijenih
What is your experience in evolving Agentic Systems? Let me know in the comments ๐
Iย have been developing Agentic Systems for the past few years and the same patterns keep emerging. ๐
๐๐๐ฎ๐น๐๐ฎ๐๐ถ๐ผ๐ป ๐๐ฟ๐ถ๐๐ฒ๐ป ๐๐ฒ๐๐ฒ๐น๐ผ๐ฝ๐บ๐ฒ๐ป๐ is the most reliable way to be successful in building your ๐๐ด๐ฒ๐ป๐๐ถ๐ฐ ๐ฆ๐๐๐๐ฒ๐บ๐ and continue improving them - here is my template.
Letโs zoom in:
๐ญ. Define a problem you want to solve: is GenAI even needed?
๐ฎ. Build a Prototype: figure out if the solution is feasible.
๐ฏ. Define Performance Metrics: you must have output metrics defined for how you will measure success of your application.
๐ฐ. Define Evals: split the above into smaller input metrics that can move the key metrics forward. Decompose them into tasks that could be automated and move the given input metrics. Define Evals for each. Store the Evals in your Observability Platform.
โน๏ธ Steps ๐ญ. - ๐ฐ. are where AI Product Managers can help, but can also be handled by AI Engineers.
๐ฑ. Build a PoC: it can be simple (excel sheet) or more complex (user facing UI). Regardless of what it is, expose it to the users for feedback as soon as possible.
๐ฒ. Instrument your application: gather traces and human feedback and store it in an Observability Platform next to previously stored Evals.
๐ณ. Run Evals on traced data: traces contain inputs and outputs of your application, run evals on top of them.
๐ด. Analyse Failing Evals and negative user feedback: this data is gold as it specifically pinpoints where the Agentic System needs improvement.
๐ต. Use data from the previous step to improve your application - prompt engineer, improve AI system topology, finetune models etc. Make sure that the changes move Evals into the right direction.
๐ญ๐ฌ. Build and expose the improved application to the users.
๐ญ๐ญ. Monitor the application in production: this comes out of the box - you have implemented evaluations and traces for development purposes, they can be reused for monitoring. Configure specific alerting thresholds and enjoy the peace of mind.
โ ๐๐ผ๐ป๐๐ถ๐ป๐๐ผ๐๐ ๐๐ฒ๐๐ฒ๐น๐ผ๐ฝ๐บ๐ฒ๐ป๐ ๐ผ๐ณ ๐๐ผ๐๐ฟ ๐ฎ๐ฝ๐ฝ๐น๐ถ๐ฐ๐ฎ๐๐ถ๐ผ๐ป:
โก๏ธ Run steps ๐ฒ. - ๐ญ๐ฌ. to continuously improve and evolve your application.
โก๏ธ As you build up in complexity, new requirements can be added to the same application, this includes running steps ๐ญ. - ๐ฑ. and attaching the new logic as routes to your Agentic System.
โก๏ธ You start off with a simple Chatbot and add a route that can classify user intent to take action (e.g. add items to a shopping cart).
Join me in a free webinar this Friday to learn how LLMOps patterns fit into this picture: https://t.co/gNy4ijenih
What is your experience in evolving Agentic Systems? Let me know in the comments ๐
If youโre a distributed systems engineer excited about database-adjacent systems, container infrastructure and cluster schedulers, weโre hiring at @tensorlake.
Our team has built systems at HashiCorp, AWS, Netlify, and SurrealDB.
Build foundational AI infrastructure with people who care about correctness and scale.
Our platform lets developers build agents and workflows on a durable, sandboxed, serverless runtime, handling fan-out, retries, and correctness under failure.
Below: Claude Agent SDK powered agentic web crawler running on Tensorlake!
Hey SWEs, PMs and aspirational AI builders, here is a free course/materials to up your game or learn what Evals/Observability are and why they are an important part of your skillset as you embark on your journey of building AI based applications!
https://t.co/aIXZobD9wZ
Over the past few years, DoorDash Labs has been building one of the most sophisticated autonomy stacks designed for the real-world challenges of local delivery.
Today, we're introducing Dot, the first commercial autonomous delivery robot to travel on bike lanes, roads, and sidewalks. At one-tenth the size of a car reaching speeds of up to 20 mph, Dot is purpose-built for local commerce.
Together with our new Autonomous Delivery Platform, Dot is the next step toward a more efficient, sustainable, and accessible delivery ecosystem for everyone.
Build Your Own "Git" In C From Scratch
-really great playlist, use this to get most out of this.
-it's great if someone wants to understand how really "Git" works behind the scenes.
-it would give you solid foundation for "Low level system programming".
-don't watch completely, it's quite long, just do what you need.
-learn as you require concepts that you need.
10/ When reviewing your products and roadmap, ask:
- What are the key feedback loops?
- Where are the delays in the system?
- What are the hidden incentives?
- Which metrics reinforce current behavior?
- What mental models drive decisions?
Follow @nurijanian for more product thinking from a technical perspective.
I share frameworks and practical guides in https://t.co/ngCnvp77SD if you want to dive deeper into product thinking.
Hash Table is the most important data structure out there and apart from knowing how to use it well, it is even more interesting to know how it is built and the internal details.
Some time back, I was on a spree to dissect the inner workings of HashTables to its core including the math behind it to build a detailed and practical understanding. I compiled all my learnings in an 11-video series
give it a watch - https://t.co/7HvgfE6yMW
I went through 2 books, 4 papers, 13 articles, and 3 codebases to understand them end to end and if you are a curious engineer and love diving deep, you'll love it. The topics I have covered in the series are
- internal structure of a HashTable
- collision resolution through chaining and open addressing
- collision resolution through linear and quadratic probing
- what determines the performance of the hash table
- why are HashTables always doubled in size when they grow
- how to implement HashSets and HashMaps
โก I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
https://t.co/cxLmaeew2f
#AsliEngineering #Engineering #HashTableInternals
Over the past month or so, both me and @shirshanka been inundated with curious questions on #datalakehouse and #data#catalogs.
We thought it ll be fun to exchange ideas and think out loud across both these areas together.
So tomorrow, Thursday, July 11th - 9am PST. Its on. Weโre going to try something different.
Tune in to the Linkedin Live event
https://t.co/rsPWLJKayJ