Bala Gorre

🚀 IT Strategy & Data Architecture Expert | 🌐 Cloud Native Transformation Leader @ Fannie Mae | 📊 AI/ML Enthusiast | Reston, VA | 🤝 Let's Connect!

[Bala Gorre @balu_gorre]

May we always be like marching ants carrying one grain of sugar at a time–falling again and again but never giving up until we reach the top

[Zunaira Ai @ZunairaAi]

I hope you've found this thread helpful.

Follow me @ZunairaAi for more.

Like/Repost the quote thread if you can:

[Aurimas Griciūnas @Aurimas_Gr]

Weekend project: my 𝗢𝗽𝗲𝗻 𝗦𝗼𝘂𝗿𝗰𝗲 implementation of 𝗗𝗲𝗲𝗽 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗔𝗴𝗲𝗻𝘁 from scratch 👇

Few weeks ago I released an episode of my Newsletter and an update to the 𝗔𝗜 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀 𝗛𝗮𝗻𝗱𝗯𝗼𝗼𝗸 GitHub repository.

There I implemented a Deep Research Agent from scratch without using any LLM Orchestration frameworks (using DeepSeek-R1 for some planning tasks).

In the project we implement the following Agentic topology:

𝟭. A user provides a query or topic to be researched.
𝟮. A LLM creates an outline of the final report that it will be aiming for. It will be instructed to produce not more than a certain number of paragraphs.
𝟯. Each of the paragraph description is fed into a research process separately to produce a comprehensive set of information to be used in report construction. Detailed description of the research process will be outlined in the next section.
𝟰. All of the information will be fed into a summarisation step that will construct the final report including conclusion.
𝟱. The report will then be delivered to the user in MarkDown form.

Each of the research steps are following given flow:

𝟭. Once we have the outline of each paragraph, it will be passed to a LLM to construct Web Search queries in an attempt to best enrich the information needed.
𝟮. The LLM will output the search query and the reasoning behind it.
𝟯. We will execute Web search against the query and retrieve top relevant results.
𝟰. The results will be passed to the Reflection step where a LLM will reason about any missed nuances to try and come up with a search query that would enrich the initial results.
𝟱. This process will be repeated for n times in an attempt to get the best set of information possible.

Detailed walkthrough Blog Post: https://t.co/Xl36Qu3zPv
GitHub with implementation code and Notebooks to follow: https://t.co/qHfhwRb669

Happy Building!

Be sure to leave a like or star if you find the content useful!

#LLM #AI #MachineLearning

[Sumanth @Sumanth_077]

This repo covers everything you need to know about MLOps!

It covers every stage of the pipeline, including model building, monitoring, configurations, testing, packaging, deployment and cicd.

100% Open Source

[Hamel Husain @HamelHusain]

Lots of great talks at AI Summit - these are my favorites (links with timestamps)

1. Jane Street - impressive internal dev tools with AI https://t.co/p0ASEbXl1I

2. windsurf - The demo has convinced me to give it a try and is a serious cursor competitor https://t.co/0DWB3vmgYI

3. Fine-tuning (OpenPipe + Method) - when you have evals, fine-tuning can be a superpower https://t.co/TP0YfexCob

[Aurimas Griciūnas @Aurimas_Gr]

What is 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 and why might it be important for your AI applications?

The idea of Contextual Retrieval was suggested by the Anthropic team late last year. It aims to improve accuracy and relevance of data that is retrieved in Retrieval Augmented Generation based AI systems.

There have been various methods suggested for improving the process before:

➡️ Overlapping chunking.
➡️ Hierarchical chunking.
➡️ …

I personally tried at least a few with different levels of success (they do help).

However, I love the intuitiveness and simplicity of Contextual Retrieval. And it does provide better results.

𝘗𝘳𝘦𝘱𝘳𝘰𝘤𝘦𝘴𝘴𝘪𝘯𝘨:

𝟭. Split each of your documents into chunks via chosen chunking strategy.
𝟮. For each chunk separately, add it to a prompt together with the whole document.
𝟯. Include instructions to situate the chunk in the document and generate short context for it. Pass the prompt to a chosen LLM.
𝟰. Combine the context that was generated in the previous step and the chunk that the context was generated for.
𝟱. Pass the data through a TF-IDF embedder.
𝟲. Pass the data through a LLM based embedding model.
𝟳. Store the data generated in steps 5. and 6. in databases that support efficient search.

𝘙𝘦𝘵𝘳𝘪𝘦𝘷𝘢𝘭:

𝟴. Use user query for relevant context retrieval. ANN search for semantic and TF-IDF index for exact search.
𝟵. Use Rank Fusion techniques to combine and deduplicate the retrieved results and produce top N elements.
𝟭𝟬. Rerank the previous results and narrow down to top K elements.
𝟭𝟭. Pass the result of step 10. to a LLM together with the user query to produce the final answer.

❗️ Step 3. might sound extremely costly and it is, but with Prompt Caching, the costs can be significantly reduced.
✅ Prompt caching can be implemented in both proprietary and open source model cases (remember Cache Augmented Generation?).

Have you tried using Contextual Retrieval in your applications already? Let me know about your accuracy gains in the comment section 👇

#AI #LLM #RAG

[Bala Gorre @balu_gorre]

@maxperzon yes

[Bala Gorre @balu_gorre]

@liamottley_ GPT

[Bala Gorre @balu_gorre]

@atulkumarzz Data

[Bala Gorre @balu_gorre]

@alifcoder send

[Maria Khalusova @mariaKhalusova]

Next week I want to toy with LLM agents, specifically in context of building better RAG.
Give me your best tutorials/tools in the comments.

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Prompt engineering is one of the most rapidly-evolving research topics in AI, but we can (roughly) group recent research on this topic into four categories…

(1) Reasoning: Simple prompting techniques are effective for many problems, but more sophisticated strategies are required to solve multi-step reasoning problems.
- [1] uses zero-shot CoT prompting to automatically generate problem-solving rationales to use for standard CoT prompting.
- [2] selects CoT exemplars based on their complexity (exemplars that have the maximum number of reasoning steps are selected first).
- [3] improves CoT prompting by asking the LLM to progressively refine the generated rationale.
- [4] decomposes complex tasks into several sub-tasks that can be solved via independent prompts and later aggregated into a final answer.

(2) Tool Usage: LLMs are powerful, but they have notable limitations. We can solve many of these limitations by teaching the LLM how to leverage external, specialized tools.
- [5, 6] finetune a language model to teach it how to leverage a fixed, simple set of text-based APIs when answering questions.
- [7] uses a central LLM-based controller to generate a program—written in natural language—that composes several tools to solve a complex reasoning task.
- [8] uses a retrieval-based finetuning technique to teach an LLM to adaptively make calls to APIs based on their documentation when solving a problem.
- [9] uses an LLM as a central controller for leveraging a variety of tools in the form of deep learning model APIs.
- [10, 11] integrates code-capable LLMs with a sandboxed Python environment to execute programs when solving problems.

(3) Context Window: Given the emphasis of recent LLMs on long contexts for RAG / few-shot learning, the properties of context windows and in-context learning have been studied in depth.
- [12] shows that including irrelevant context in the LLM’s prompt can drastically deteriorate performance.
- [13] finds that LLMs pay the most attention to information at the beginning/end of the prompt, while information placed in the middle of a long context is forgotten.
- [14] proposes a theoretically-grounded strategy for optimally selecting few-shot exemplars.

(4) Better Writing: One of the most popular use-cases of LLMs is for improving human writing, and prompt engineering can be used to make more effective writing tools with LLMs.
- [15] improves the writing abilities of an LLM by first generating an outline and then filling in each component of the outline one-by-one.
- [16] uses a smaller LLM to generate a “directional stimulus” (i.e., a textual hint) that can be used as extra context to improve an LLM’s writing ability on a given task.
- [17] improves the quality of LLM-generated summaries by iteratively prompting the LLM to increase the information density of the summary.

-----
Bibliography
[1] Automatic chain of thought prompting in large language models.
[2] Complexity-based prompting for multi-step reasoning.
[3] Progressive-hint prompting improves reasoning in large language models.
[4] Decomposed prompting: A modular approach for solving complex tasks.
[5] Toolformer: Language models can teach themselves to use tools.
[6] Gpt4tools: Teaching large language model to use tools via self-instruction.
[7] Chameleon: Plug-and-play compositional reasoning with large language models.
[8] Gorilla: Large language model connected with massive apis.
[9] Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface.
[10] PAL: Program-aided Language Models.
[11] Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.
[12] Large language models can be easily distracted by irrelevant context.
[13] Lost in the middle: How language models use long contexts.
[14] Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning.
[15] Skeleton-of-thought: Large language models can do parallel decoding.
[16] Guiding large language models via directional stimulus prompting.
[17] From sparse to dense: GPT-4 summarization with chain of density prompting.

[Jerry Liu @jerryjliu0]

Inspired by @AndrewYNg’s threads on agents in the past few weeks, I’m excited to share my talk from the @weights_biases conference that outlines how to build a general context-augmented research assistant 🧑‍🔬

Naive RAG is mostly good for simple questions in a single-shot setting. But building a research agent enables the following:
✅ Multi-hop QA
✅ Query understanding and task planning
✅ Interfaces to interact with a broad range of external tools
✅ Reflection to iteratively correct behavior
✅ Memory for personalization

We start with the agent ingredients and then compose full agents. Full guides to @llama_index resources linked below 👇

Check it out: https://t.co/gkRaqTkU6P

[Andrew Bolis @AndrewBolis]

I don't understand why people are not using ChatGPT for job search.

Copy and paste these 7 ChatGPT prompts to land your job in 28 days:

[Josep Ferrer @iamjosepferrer]

Struggling with Machine Learning algorithms? 🤖

Then you better stay with me! 🤓

Today I am starting with a new ML model
... so it is the turn of the Support Vector Machine! 👇🏻

[Jason Helmes @anymanfitness]

Just finished this book - Bad Therapy by @AbigailShrier

This is one of the most eye-opening books I've ever read. It's a must read for any parent, any teacher, and should be required reading for any school administrator as well.

The book dives into trying to figure out why kids are having so many mental health problems, when there are so many resources devoted to improving mental health outcomes.

Anxiety, depression, suicide, etc are all higher than they've ever been with kids, even though their lives are arguably better than ever before. It just doesn't make sense.

A few key takeaways from the book:

A constant attention on how kids are "feeling" or "thinking" is causing negative outcomes.

Constantly ruminating on your emotions and how you feel negatively impacts your mental health. If all you do is focus on your emotions, you are destined to be anxious or depressed.

We incessantly ask kids how they're feeling, if they're happy, how their mental health is, etc, and this is creating kids who think they're fragile instead of resilient.

Trying to solve every problem for kids has caused a generation who can't do anything for themselves.

We (Gen X) were told to "suck it up" or "you'll live" or "rub some dirt on it" all the time. Many of us came to the conclusion this is "bad parenting" because our feelings were neglected, and we vowed not to do this to our own children.

Because of that, kids immediately over-dramatize everything that happens to them, making mountains out of molehills, and thinking the world must revolve around their emotions and feelings.

You develop confidence and strong mental health by doing things, not by thinking or via therapy.

You can't think your way out of anxiety. You don't gain confidence by analysis of your thoughts or mental health issues.

You gain confidence and eliminate anxiety by doing gradually more difficult tasks, excelling at them, and realizing you are a competent, capable person.

The non-stop attention therapy gives to these small, common emotions we all feel blows them out of proportion to their seriousness (not talking about genuine disorders here, just normal anxieties that millions of people go to therapy to try to avoid).

One of the best ways to decrease your happiness is to chase it.

Our society constantly tells kids they should be "happy" and asks them if they are.

Happiness isn't a state you should be in 24/7. That's not realistic. Joy and bliss aren't permanent states - they are fleeting.

Contentment, stillness, and being even-keeled are much better goals to aim for mentally.

The happiest, most well adjusted kids come from families with loving parents that have strict rules for the household.

This one really set off the confirmation bias in me... I feel really blessed we have 2 well adjusted middle school kids who do great in school, are very respectful and well mannered, and we barely even need to parent them.

But for years, we were very strict with them. Bedtimes, family rules, how we do things, etc. The in-laws and lots of friends thought we were totalitarian.

In reality, we just had high standards. And it's really paying off right now. I found it really interesting that strict rules equals happy kids. Makes sense, though, as kids need to know what their boundaries are.

Constantly surveying school-age kids about their mental health causes more issues than it solves.

Mental health resources is big money. Districts need to validate all the resources allocated towards mental health, and they often do that via surveys.

Asking kids non-stop questions like:

- Have you thought about self harm?
- Have you thought about suicide?
- Have you been so anxious you can't get out of bed?

Etc, etc puts into their heads the idea that themselves, or many of their peers are broken and cannot function properly in the real world.

It normalizes situations that would be incredibly rare at any other time in history.

There's a lot of other takeaways, too, but I'll stop there.

It's a fantastic book. Go pick it up and read it. This isn't an affiliate thing or a promotion thing at all. I just really enjoyed it, and it will further shape the way I parent moving forward.

[EDDIE CHENG @ecomEddie]

If you can’t motivate yourself, you’ll never achieve anything significant.

How to force your brain to crave doing hard things:

[Bala Gorre @balu_gorre]

@alifcoder send

[Michael Pyrcz🌻 @GeostatsGuy]

As a professor, I work very hard to make my #DataScience educational content accessible and actionable!

Here’s my #DataAnalytics & #geostatistics course, all the lectures, with linked #Python well-documented demonstration workflows and interactive dashboards!

My goal is to ease your learning journey as we build powerful skills together @ https://t.co/40r6WTUpwC ∀!

[Dan Kornas @DanKornas]

👉 Text Summarization

A curated list of resources dedicated to text summarization.

🔗 https://t.co/vfex7YrZBw

[Rishabh @Rixhabh__]

I've watched 200 YouTube videos to learn Python.

96% of them were a complete waste of time.

But these 10 channels taught me more than all my teachers combined: ↓