Daniel Sef

Experienced Software Engineer, design and develop data architectures, streaming systems and Digital Products #java #springboot #kafka #cloudnative

[Daniel Sef @danielsef]

@TommiPedruzzi AI

[Daniel Sef @danielsef]

@AlfieJCarter CLAUDE

[Daniel Sef @danielsef]

https://t.co/mOYiPPRQTT

[ℏεsam @Hesamation]

https://t.co/LsZLoCMqTN

[Daniel Sef @danielsef]

Use this SaaS Template for advanced applications https://t.co/lecGD6q5WR via @YouTube

[Daniel Sef @danielsef]

He Built The Revenue Engines for Google, Facebook & Square https://t.co/KlxqTHkT9n via @YouTube

[Tech with Mak @techNmak]

The full journey of modern AI engineering, from start to finish.

[Andrej Karpathy @karpathy]

+1 for "context engineering" over "prompt engineering".

People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step. Science because doing this right involves task descriptions and explanations, few shot examples, RAG, related (possibly multimodal) data, tools, state and history, compacting... Too little or of the wrong form and the LLM doesn't have the right context for optimal performance. Too much or too irrelevant and the LLM costs might go up and performance might come down. Doing this well is highly non-trivial. And art because of the guiding intuition around LLM psychology of people spirits.

On top of context engineering itself, an LLM app has to:
- break up problems just right into control flows
- pack the context windows just right
- dispatch calls to LLMs of the right kind and capability
- handle generation-verification UIUX flows
- a lot more - guardrails, security, evals, parallelism, prefetching, ...

So context engineering is just one small piece of an emerging thick layer of non-trivial software that coordinates individual LLM calls (and a lot more) into full LLM apps. The term "ChatGPT wrapper" is tired and really, really wrong.

[Daniel Sef @danielsef]

@phuctm97 @phuctm97, what tech stack do you recommend for a highly interactive or data-intensive application? Thanks!

[Andrew Ng @AndrewYNg]

I think AI agentic workflows will drive massive AI progress this year — perhaps even more than the next generation of foundation models. This is an important trend, and I urge everyone who works in AI to pay attention to it.

Today, we mostly use LLMs in zero-shot mode, prompting a model to generate final output token by token without revising its work. This is akin to asking someone to compose an essay from start to finish, typing straight through with no backspacing allowed, and expecting a high-quality result. Despite the difficulty, LLMs do amazingly well at this task!

With an agentic workflow, however, we can ask the LLM to iterate over a document many times. For example, it might take a sequence of steps such as:
- Plan an outline.
- Decide what, if any, web searches are needed to gather more information.
- Write a first draft.
- Read over the first draft to spot unjustified arguments or extraneous information.
- Revise the draft taking into account any weaknesses spotted.
- And so on.

This iterative process is critical for most human writers to write good text. With AI, such an iterative workflow yields much better results than writing in a single pass.

Devin’s splashy demo recently received a lot of social media buzz. My team has been closely following the evolution of AI that writes code. We analyzed results from a number of research teams, focusing on an algorithm’s ability to do well on the widely used HumanEval coding benchmark. You can see our findings in the diagram below.

GPT-3.5 (zero shot) was 48.1% correct. GPT-4 (zero shot) does better at 67.0%. However, the improvement from GPT-3.5 to GPT-4 is dwarfed by incorporating an iterative agent workflow. Indeed, wrapped in an agent loop, GPT-3.5 achieves up to 95.1%.

Open source agent tools and the academic literature on agents are proliferating, making this an exciting time but also a confusing one. To help put this work into perspective, I’d like to share a framework for categorizing design patterns for building agents. My team AI Fund is successfully using these patterns in many applications, and I hope you find them useful.

- Reflection: The LLM examines its own work to come up with ways to improve it.
- Tool use: The LLM is given tools such as web search, code execution, or any other function to help it gather information, take action, or process data.
- Planning: The LLM comes up with, and executes, a multistep plan to achieve a goal (for example, writing an outline for an essay, then doing online research, then writing a draft, and so on).
- Multi-agent collaboration: More than one AI agent work together, splitting up tasks and discussing and debating ideas, to come up with better solutions than a single agent would.

I’ll elaborate on these design patterns and offer suggested readings for each next week.

[Original text: https://t.co/y4McIAjD2m]

[Rohan Paul @rohanpaul_ai]

Satya Nadela explains the AI Agentic Future.

The business logic is all going to these Agents.

-----

Video from Bg2 Pod Youtube Channel (link in comment)

[Daniel Sef @danielsef]

@lax97981 Apps

[Alfaiz Ali @AlfaizAliX]

10 hidden gem websites to create killer videos that nobody is using yet: 🧵🎥

[The SamurAI @samuraipreneur]

Here's Mila.

She isn't real, but makes income on auto pilot.

I'll show you how to set it all up in 2 minutes:

✦ https://t.co/DSePgevfFi

[Ryan Peterman @ryanlpeterman]

Writing is one of the most important skills for software engineers. A lot of it is repetitive in structure (e.g. design docs, post mortems, etc).

Here are my top 6 software engineering templates that I wish I had sooner:

[Daniel Sef @danielsef]

@AndrewBolis Guide

[LangChain @LangChain]

RAG From Scratch: Feedback + self-reflection

Our RAG From Scratch video series walks through impt RAG concepts in short / focused videos w/ code.

This is the final part in our series, focusing on self-reflection + feedback to improve RAG systems.

🔧Problem: RAG systems can suffer from low quality retrieval (e.g., if a user question is out of the domain for the index) and / or hallucinations in generation. A naive retrieve-generate pipeline has no ability to detect or self-correct from these kids of errors.

💡Idea: The concept of "flow engineering" has been recently introduced by @itamar_mar the context of code generation: iteratively build an answer to a code question w/ unit tests to check and self-correct errors.
@HamelHusain has a great blog post that mentions the benefit of unit-testing in inference loop. Several works have applied this RAG, such as Self-RAG (@AkariAsai et al) and Corrective-RAG (@Jiachen_Gu + colleages). In both cases, checks for document relevance, hallucinations, and / or answer quality are performed in the RAG answer flow. We've implemented these ideas using LangGraph to orchestrate the checks and feedback. We've also shown that LangGraph allows both to run reliable w/ smaller OSS models.

📽️ Video:
https://t.co/bfBD0NmeEf

💻 Code:
CRAG:
https://t.co/FG0gJ9K0Xw
Self-RAG:
https://t.co/3ThXzscI7g
Both with @MistralAI-7b + @ollama:
https://t.co/VSdkUzUcJe
https://t.co/OK7uq0tm1s

🧠References:
1/ Self-RAG
https://t.co/hL6zKlqMQp
2/ C-RAG:
https://t.co/WsqjxBodT5
3/ Flow-engineering:
https://t.co/CGsXVoyIE7
4/ Blog on evals, covering unit tests:
https://t.co/k9zy27F3D9

[Bindu Reddy @bindureddy]

An LLM Ops and AI agents Platform can help you crank out multiple bots to perform different tasks of varying complexity.

Some of our customers have a list of 700 agents they want to build!!

You will need support for various things, including:

- Dynamic RAG, information retrieval, and vector stores
- LLM orchestration, monitoring, and debugging
- Code execution and data wrangling at scale
- Connectors to a bunch of different systems

Check out Abacus AI - AI agents have become our fastest-growing vertical....

[Matthew Berman @MatthewBerman]

DBRX by @databricks ...it's REALLY good!!

The New MoE 132b parameter model is open-source and costs $10 m to train. Thank you, Databricks, for your contribution to OS.

Check out the full explanation and testing:

🎥👇