Murali Subbarao

@msubbarao

Saratoga, CA, USA

Joined November 2008

47 Following

86 Followers

73 Posts

Murali Subbarao @msubbarao

7 months ago

What’s at Knowledge 2026 that you can’t find anywhere else? Hands-on learning, interactive demos, and real examples of how your peers are putting AI to work—and getting ahead. Register early for $400 off. (Hurry! Offer ends 12/31) https://t.co/SCs1DE48gY

msubbarao's tweet photo. What’s at Knowledge 2026 that you can’t find anywhere else? Hands-on learning, interactive demos, and real examples of how your peers are putting AI to work—and getting ahead. Register early for $400 off. (Hurry! Offer ends 12/31) https://t.co/SCs1DE48gY https://t.co/wM955DJ3G9

0

0

0

0

21

Murali Subbarao @msubbarao

about 1 year ago

@ServiceNow is one of @LinkedIn's Top Companies in the U.S.! Proud to be part of a team that makes the world work better for everyone as we grow our careers, build new skills, and thrive in a culture that puts people first. Join us: https://t.co/wC4Y7qFQWp #LinkedInTopCompanies

msubbarao's tweet photo. @ServiceNow is one of @LinkedIn's Top Companies in the U.S.!

Proud to be part of a team that makes the world work better for everyone as we grow our careers, build new skills, and thrive in a culture that puts people first. Join us: https://t.co/wC4Y7qFQWp

#LinkedInTopCompanies https://t.co/QYmJHDYWAt

0

1

0

0

19

msubbarao retweeted

about 3 years ago

The new 100k token model from @AnthropicAI is awesome: dump in giant docs/books into the prompt, do LLM tasks! 📚🛠️ It got me thinking about the relationship to fine-tuning and in-context learning - is it better than both, worse than both, or used in niche cases? 🧵 In the extreme, if context windows are infinite, then putting all the data into the prompt seems similar to “throw-away” fine-tuning 🤔 Pros ✅: - Similar to fine-tuning, you get benefits of explicitly giving this black-box model access to all your knowledge (just in the inputs instead of in the weights) - Less hand-engineered than retrieval-augmented generation (RAG) - You can more easily feed in new data than actually fine-tuning (which seems hard to use) Cons ❌: - You have to feed in this data for every inference call - As a result, marginal cost/latency go way up. 💵 Retrieval augmented generation is more limited in functionality (because inherently requires some hand-engineering and data pipelining), but on the other hand can reduce cost/latency. So is this approach of feeding everything into the input prompt a happy middle or worst of both worlds? Of course, going back to the context window of 100k: 100k tokens is a lot, but if you have gigabytes or terabytes of data, 100k tokens can’t fit everything (it can’t actually fit UBER SEC filings). So either way you will need to do some retrieval from your data, in the absence of fine-tuning. And then the question becomes whether you'll always want to maximize the context window, or you're ok with retrieving smaller chunks. Thoughts? Added some diagrams below to help clarify my thinking 🖼️

jerryjliu0's tweet photo. The new 100k token model from @AnthropicAI is awesome: dump in giant docs/books into the prompt, do LLM tasks! 📚🛠️

It got me thinking about the relationship to fine-tuning and in-context learning - is it better than both, worse than both, or used in niche cases? 🧵

In the extreme, if context windows are infinite, then putting all the data into the prompt seems similar to “throw-away” fine-tuning 🤔

Pros ✅:
- Similar to fine-tuning, you get benefits of explicitly giving this black-box model access to all your knowledge (just in the inputs instead of in the weights)
- Less hand-engineered than retrieval-augmented generation (RAG)
- You can more easily feed in new data than actually fine-tuning (which seems hard to use)

Cons ❌:
- You have to feed in this data for every inference call
- As a result, marginal cost/latency go way up. 💵

Retrieval augmented generation is more limited in functionality (because inherently requires some hand-engineering and data pipelining), but on the other hand can reduce cost/latency. So is this approach of feeding everything into the input prompt a happy middle or worst of both worlds?

Of course, going back to the context window of 100k: 100k tokens is a lot, but if you have gigabytes or terabytes of data, 100k tokens can’t fit everything (it can’t actually fit UBER SEC filings). So either way you will need to do some retrieval from your data, in the absence of fine-tuning. And then the question becomes whether you'll always want to maximize the context window, or you're ok with retrieving smaller chunks.

Thoughts? Added some diagrams below to help clarify my thinking 🖼️

jerryjliu0's tweet photo. The new 100k token model from @AnthropicAI is awesome: dump in giant docs/books into the prompt, do LLM tasks! 📚🛠️

It got me thinking about the relationship to fine-tuning and in-context learning - is it better than both, worse than both, or used in niche cases? 🧵

In the extreme, if context windows are infinite, then putting all the data into the prompt seems similar to “throw-away” fine-tuning 🤔

Pros ✅:
- Similar to fine-tuning, you get benefits of explicitly giving this black-box model access to all your knowledge (just in the inputs instead of in the weights)
- Less hand-engineered than retrieval-augmented generation (RAG)
- You can more easily feed in new data than actually fine-tuning (which seems hard to use)

Cons ❌:
- You have to feed in this data for every inference call
- As a result, marginal cost/latency go way up. 💵

Retrieval augmented generation is more limited in functionality (because inherently requires some hand-engineering and data pipelining), but on the other hand can reduce cost/latency. So is this approach of feeding everything into the input prompt a happy middle or worst of both worlds?

Of course, going back to the context window of 100k: 100k tokens is a lot, but if you have gigabytes or terabytes of data, 100k tokens can’t fit everything (it can’t actually fit UBER SEC filings). So either way you will need to do some retrieval from your data, in the absence of fine-tuning. And then the question becomes whether you'll always want to maximize the context window, or you're ok with retrieving smaller chunks.

Thoughts? Added some diagrams below to help clarify my thinking 🖼️

jerryjliu0's tweet photo. The new 100k token model from @AnthropicAI is awesome: dump in giant docs/books into the prompt, do LLM tasks! 📚🛠️

It got me thinking about the relationship to fine-tuning and in-context learning - is it better than both, worse than both, or used in niche cases? 🧵

In the extreme, if context windows are infinite, then putting all the data into the prompt seems similar to “throw-away” fine-tuning 🤔

Pros ✅:
- Similar to fine-tuning, you get benefits of explicitly giving this black-box model access to all your knowledge (just in the inputs instead of in the weights)
- Less hand-engineered than retrieval-augmented generation (RAG)
- You can more easily feed in new data than actually fine-tuning (which seems hard to use)

Cons ❌:
- You have to feed in this data for every inference call
- As a result, marginal cost/latency go way up. 💵

Retrieval augmented generation is more limited in functionality (because inherently requires some hand-engineering and data pipelining), but on the other hand can reduce cost/latency. So is this approach of feeding everything into the input prompt a happy middle or worst of both worlds?

Of course, going back to the context window of 100k: 100k tokens is a lot, but if you have gigabytes or terabytes of data, 100k tokens can’t fit everything (it can’t actually fit UBER SEC filings). So either way you will need to do some retrieval from your data, in the absence of fine-tuning. And then the question becomes whether you'll always want to maximize the context window, or you're ok with retrieving smaller chunks.

Thoughts? Added some diagrams below to help clarify my thinking 🖼️

26

466

84

391

255K

Murali Subbarao @msubbarao

about 3 years ago

Exciting news! Join me this Saturday for the "Inspiring and Mentoring Women in STEM" event hosted by the Joy Thomas Foundation. Featuring keynote speaker Vidita Vaidya, a renowned neuroscientist and professor, and Tanishka Kabra, the recipient of the 2nd…https://t.co/d8thCKhAY3

0

1

0

0

40

Who to follow

Velera is the nation’s premier payments CUSO and an integrated fintech solutions provider, serving more than 4,000 financial institutions across North America.

Suraj Deshmukh | सुरज देशमुख

Confidential Containers @Microsoft | ex-@kinvolkio ex-@RedHat | bibliophile | He/Him | Opinions are my own. 🟥🟩 🟦🟨

I love credit unions. Credit union consultant & speaker. When I'm not helping CUs grow or speaking at conferences you can find me vagabonding around the world.

Murali Subbarao @msubbarao

over 4 years ago

Lightstep Incident Response: Context and automation you need to respond to incidents fast https://t.co/cR4f7LlUY2 by @kcthota

0

0

0

0

0

Murali Subbarao @msubbarao

about 5 years ago

Knowledge 2021 will feature more than 500 digital sessions across 12 channels. Familiarize yourself with our platform before you go. #Know21 https://t.co/SroLGERKJ5

0

1

0

0

0

Murali Subbarao @msubbarao

about 7 years ago

ServiceNow is looking for: Staff Engineer - NLU Platform https://t.co/3RUV3WfmnR #job

0

1

0

0

0

Murali Subbarao @msubbarao

about 8 years ago

Excited to officially announce that Parlo has been acquired by ServiceNow! Parlo's #enterprise #NLU engine will enhance ServiceNow’s native #AI capabilities across its Now Platform and products, making getting work done as easy as…https://t.co/R9OsNuo2zE https://t.co/bO9Uu93seA

0

4

0

0

0

Murali Subbarao @msubbarao

over 8 years ago

Proud of the Parlo team for being featured in VentureBeat's 2017 Intelligent Assistance & Bot Landscape report! #AI…https://t.co/SUD3mTSPdU

0

0

0

0

0

msubbarao retweeted

Parlo @GetParlo

about 9 years ago · San Francisco

We are now on @ProductHunt, please check us out! https://t.co/yFkcdQ8hF7 #AI #Chatbots #bots #NLP #NLU #producthunt #voicebots #witai #apiai

0

11

6

0

0

Murali Subbarao @msubbarao

over 9 years ago

@CheggHelp Thank you!

1

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@CheggHelp please cancel and refund the charge

1

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@CheggHelp charge must be for a free offer after text book buy. After 30d there was an auto charge. Want service cancelled charge reversed

0

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@CheggHelp or it could be [email protected]

2

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@CheggHelp I don't have an order number. Just the charge on my CC bill. My email is [email protected]

1

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@CheggHelp verifying a 14.95 transaction on my card

1

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@wingstop #order

1

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

“Tommy Hilfiger Bot: Artificial Intelligence Gone Wrong” by @GetXpressBuy https://t.co/MNayzFi42d

0

0

0

0

0

Murali Subbarao @msubbarao

over 9 years ago

@benjkeys Loved your article on 'Killer Feature'. @GetXpressBuy has a chatbot development platform. Would love to explore working together.

0

0

0

0

0

Murali Subbarao @msubbarao

almost 10 years ago

Brands - Are you listening? #chatbot #messenger #commerce https://t.co/XoiMLP70yW

Parlo @GetParlo

almost 10 years ago

Checkout our blog: Your Consumers Want To Chat, But Are You Listening? https://t.co/9lGnCumjkQ #ChatBots

GetParlo's tweet photo. Checkout our blog: Your Consumers Want To Chat, But Are You Listening? https://t.co/9lGnCumjkQ #ChatBots https://t.co/9KusaQ9wTp

0

4

3

0

0

1

1

0

0

0

Last Seen Users on Sotwe

Trends for you

Most Popular Users