Here’s my latest market overview of the best-selling mechanical keyboard switches (May 2025). Based on data from 17 vendors and manufacturers, representing around 1 million switches sold last month:
https://t.co/hRMJjnbORn
This poop-looking robot could one day perform tasks inside the human body.
The robot is made of magnetic slime that can be controlled by external magnets to move, rotate, and form shapes.
#gigadgets#medical#robootics#slime#research
As we wrap up 2024, this year's final edition of The Batch summarizes some of the top stories of the year:
- Agents ascendant: The growth of agentic workflows
- Prices tumble: Rapidly falling LLM token prices
- Generative video takes off
- Smaller is beautiful: Small language models
- Alternatives to acquisitions
See the full articles here: https://t.co/cQ1eqbeK4a
OpenAI just announced API access to o1 (advanced reasoning model) yesterday. I'm delighted to announce today a new short course, Reasoning with o1, built with @OpenAI, and taught by @colintjarvis, Head of AI Solutions at OpenAI, to show you how to use this effectively!
Unlike previous language models which generate output directly, o1 “thinks before it responds,” and generates many reasoning tokens before returning a more thoughtful and accurate response. It is great at complex reasoning -- including planning for agentic workflows, coding, and domain-specific reasoning in STEM fields like law. But how you should use it is quite different from other LLMs.
I think o1 will be a game changer for many AI applications; and in this course, you'll learn how to use it effectively.
In detail, you’ll:
- Learn to recognize what tasks o1 is suited for, and when to use a smaller model, or combine o1 with a smaller model
- Understand the new principles of prompting reasoning models: Be simple and direct; no explicit chain-of-thought required; use structure; show rather than tell
- Implement multi-step orchestration in which o1 plans, and hands tasks over to gpt-4o-mini to execute specific steps; this illustrates a design pattern to optimize intelligence (accuracy) and cost
- Use o1 for a coding task to build a new application, edit existing code, and test performance by running a coding competition between o1-mini and GPT 4o
- Use o1 for image understanding and learn how it performs better with a "hierarchy of reasoning," in which it incurs the latency and cost upfront, preprocessing the image and indexing it with rich details so it can be used for Q&A later
- Learn a technique called meta-prompting, in which you use o1 to improve your prompts. Using a customer support evaluation set, you'll iteratively use o1 to modify a prompt to improve performance
You'll also learn about how OpenAI used reinforcement learning to produce a model that uses "test-time compute" to improve performance.
I think you'll find this course enjoyable and valuable.
Please sign up for it here: https://t.co/0XIGzinyrx
Something fun: AI Avatar of me built, by https://t.co/zpIxRSuky4 and @realavatarai.
Video has details. This is a work in progress, but please come chat with me in avatar form, and let me know what you think!
https://t.co/vMO2CM0xfb
Thank you Jeff Daniel @consciouspilot and team for working with us on this!
New short course: Practical Multi AI Agents and Advanced Use Cases with crewAI. Learn to build and deploy advanced agent-based systems in real applications in this course, created with @crewAIInc and taught by its founder, @joaomdmoura! (Disclosure: I've made a small seed investment in CrewAI.)
In this course, you’ll learn how to create advanced agent-based apps that use external tools, do performance testing, can be trained with human feedback, and perform multiple tasks with different large language models.
You will build several practical agentic apps that provide real business value, such as an automated project planning system, lead scoring and engagement pipeline, customer support data analysis, and a robust content creation system.
In detail, you will learn how to:
- Create these multi-agent systems with the building blocks of tasks, agents, and crews, along with the different things that make them work, such as caching, memory, and guardrails.
- Integrate your multi-agent application with internal and external systems.
- Connect multiple agents in complex setups, including parallel, sequential, and hybrid configurations, and create flows involving multiple agentic applications working together.
- Test your agentic workflow and train it using human feedback to optimize its performance for better and more consistent results.
- Work with multiple LLMs in your multi-agent system, using the appropriate model sizes and providers to fit each agent’s specific task.
- Start a project from scratch in your environment and prepare it for deployment.
You’ll also learn from an interview between João and Jacob Wilson, the Commercial GenAI Principal at PwC , in which they discuss deploying agentic workflows in real industry use cases.
By the end of this course, you will be equipped to start building custom multi-agentic systems for your work.
Please sign up here! https://t.co/JkD52B3ONA
⭐️Cool project built on LangChain: Jenova
An all-in-one AI assistant that intelligently selects the best models and tools for your tasks. Powered by GPT-4o, Claude 3.5, and Gemini 1.5
Excited to see systems become multi-model!
https://t.co/1HOdHvyhsn
Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with @LaminiAI and @Meta, and taught by Lamini’s CEO @realSharonZhou, and Meta’s Senior Director of Partner Engineering, @asangani7. (Disclosure: I am an investor in Lamini.)
The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations:
- Create an evaluation dataset to measure model accuracy
- Add prompt engineering and self-reflection
- Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM
Using the Llama 3-8B parameter model, you will:
- Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates
- Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning
- See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further
I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models.
Please sign up here: https://t.co/FITZFVlzNk
@catalinmpit nice list. I believe practical use cases can be implemented for each of the features listed. I love Postgres, but no "one size fits all". Just taking Kafka as an example, i would like to see comparison based on bechmarks and metrics that compares to Kafka. Scaling is not easy
New AI Agentic course! Learn to use LangGraph to build single and multi-agent LLM applications in AI Agents in LangGraph. This short course, taught by LangChain @langchain founder Harrison Chase @hwchase17 and @tavilyai founder @weiss_rotem, shows how to integrate agentic search to enhance an agent's knowledge with query-focused answers in predictable formats. Also learn to implement agentic memory to save state for reasoning and debugging, and see how human-in-the-loop input can guide agents at key junctures.
You'll build an agent from scratch, then reconstruct it with LangGraph to thoroughly understand the framework. Finally, you'll build a sophisticated essay-writing agent that incorporates all the learnings from the course.
Sign up here! https://t.co/ZDpjLmdyDL
I took Ethereum quizzes on https://t.co/02Tbb2j0aa and overall scored 54 out of 54! Try it yourself at https://t.co/xSSNaPv17T #ethereumquiz#ethereum#quiz
Tool use, in which an LLM is given functions it can request to call for gathering information, taking action, or manipulating data, is a key design pattern of AI agentic workflows. You may be familiar with LLM-based systems that can perform a web search or execute code. Some of the large, consumer-facing LLMs already incorporate these features. But tool use goes well beyond these examples.
If you prompt an online LLM-based chat system, “What is the best coffee maker according to reviewers?”, it might decide to carry out a web search and download one or more web pages to gain context. Early on, LLM developers realized that relying only on a pre-trained transformer to generate output tokens is limiting, and that giving an LLM a tool for web search lets it do much more. With such a tool, an LLM is either fine-tuned or prompted (perhaps with few-shot prompting) to generate a special string like {tool: web-search, query: "coffee maker reviews"} to request calling a search engine. (The exact format of the string depends on the implementation.) A post-processing step then looks for strings like these, calls the web search function with the relevant parameters when it finds one, and passes the result back to the LLM as additional input context for further processing.
Similarly, if you ask, “If I invest $100 at compound 7% interest for 12 years, what do I have at the end?”, rather than trying to generate the answer directly using a transformer network — which is unlikely to result in the right answer — the LLM might use a code execution tool to run a Python command to compute 100 * (1+0.07)**12 to get the right answer. The LLM might generate a string like this: {tool: python-interpreter, code: "100 * (1+0.07)**12"}.
But tool use in agentic workflows now goes much further. Developers are using functions to search different sources (web, Wikipedia, arXiv, etc.), to interface with productivity tools (send email, read/write calendar entries, etc.), generate or interpret images, and much more. We can prompt an LLM using context that gives detailed descriptions of many functions. These descriptions might include a text description of what the function does plus details of what arguments the function expects. And we’d expect the LLM to automatically choose the right function to call to do a job.
Further, systems are being built in which the LLM has access to hundreds of tools. In such settings, there might be too many functions at your disposal to put all of them into the LLM context, so you might use heuristics to pick the most relevant subset to include in the LLM context at the current step of processing. This technique, which is described in the Gorilla paper cited below, is reminiscent of how, if there is too much text to include as context, retrieval augmented generation (RAG) systems offer heuristics for picking a subset of the text to include.
Early in the history of LLMs, before widespread availability of large multimodal models (LMMs) like LLaVa, GPT-4V, and Gemini, LLMs could not process images directly, so a lot of work on tool use was carried out by the computer vision community. At that time, the only way for an LLM-based system to manipulate an image was by calling a function to, say, carry out object recognition or some other function on it. Since then, practices for tool use have exploded. GPT-4’s function calling capability, released in the middle of last year, was a significant step toward general-purpose tool use. Since then, more and more LLMs are being developed to similarly be facile with tool use.
If you’re interested in learning more about tool use, I recommend:
- Gorilla: Large Language Model Connected with Massive APIs, Patil et al. (2023)
- MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action, Yang et al. (2023)
- Efficient Tool Use with Chain-of-Abstraction Reasoning, Gao et al. (2024)
Both Tool Use and Reflection, which I posted about last week, are design patterns that I can get to work fairly reliably on my applications — both are capabilities well worth learning about. In the future, I’ll describe the Planning and Multi-agent collaboration design patterns. They allow AI agents to do much more but are less mature, less predictable — albeit very exciting — technologies.
[Original text: https://t.co/gHCOYSsKQO ]
Leadership.
It looks like this.
How Singapore prepares it citizens for the age AI.
While some western countries sell yesterday’s careers to prop up institutions that refuse to grow and adapt with life long financial burdens.
@MKBHD@LinusEkenstam@omooretweets YouTuber MrBeast called out a deepfake ad on X that was falsely using his image to promote a scam iPhone giveaway.
MrBeast said the deepfakes are a "serious problem" as quality improves with advancing tech.
https://t.co/6GX58pCoVK
https://t.co/4s1FbGtOan