James Kaplan

Partner @McKinsey, CTO McKinsey Technology, own views

[James Kaplan @jmk37]

Finally got around to reading what @GoDaddy said about #prompt engineering.

https://t.co/SnQUqxaf13

1. Sometimes one prompt isn’t enough — makes sense to break down prompts instead of one giant prompt to rule them all. Also, would it make sense to present the user a old fashioned list of options get a read on which prompt to route him/her to?

2. Be careful with structured outputs. The old editor in me might ask “where should you not be careful?” That aside, like the idea of lowering the prompt temperature for structured outputs and especially identifying (and testing for) common failure modes.

3. Prompts aren’t portable across models — I’m sure this is right. I would love for someone (maybe us) to analyze how outputs vary across models for different types of prompts.

4. AI guardrails are essential. Given the probabilistic nature of #GenAI output, interrogate the results to determine when to escalate to a number. Makes sense — but how do you determine the full set of guardrails you need? What bad results do you want to look for?

5. Models can be slow and unreliable. 1% of chat interactions fail at the GenAI provider, and they time out after 30 seconds. Makes sense to move to more asynchronous responses.

6. Memory management is hard. They seem to be considering using stacks to implement memory — provide working memory to delegate prompts and reap the results when the discussion moves back to the controller.

7. Adaptive model selection is the future (even though they haven’t implemented it you). How do you reconcile adaptive model selection with the observation that prompts don’t work across models? (Which I suppose means you select the model at “design” time rather than “run” time?)

8. Use #RAG effectively. Suggested a couple of patterns for RAG, which I am not sure I agree with. I would argue that GenAI in the enterprise will involve more RAG than non-RAG, so there will be many, many RAG patters. I wonder if GoDaddy has a bit of a narrow view as they mostly seemed to talking about a customer support use case.

9. Tune your data for RAG. Argues that you can remove extraneous language to improve query performance. Do you really need to do this. Couldn’t you do RAG at design time, shove the results into a database and query that at run time?

10. Test! Test! Test! Points out that test will be more time consuming that build in #LLM integration. I am sure this is true — again data about how much you give in increased testing in return for reduce build effort would be interesting.

[James Kaplan @jmk37]

Good podcast from @prateekj at @MoxxieVentures on Infinite #MachineLearning podcast with @timt , now @MenloVentures and former cto at @splunk on #AI #Database platforms

- we should remember that the bus is the bottleneck in database performance
- he’s very intrigued by agentic workflows
- still developers don’t take LLM trust and safety seriously enough
- DBMS are complicated to develop as operating systems
- users don’t care about real-time analytics and five minutes is good enough (probably not true in capital markets!)

[James Kaplan @jmk37]

@philvenables In manhattan yes. Not terrible in the sticks I think.

[James Kaplan @jmk37]

@JJ_Kasper @mattyglesias Well when you are 107 years old…

[James Kaplan @jmk37]

@philvenables There was a great book about this.

[James Kaplan @jmk37]

@MayaKaczorowski Paper notepad in secure location

[James Kaplan @jmk37]

@sam_d_1995 @heyotetsuo What if you live on first ave?

[James Kaplan @jmk37]

Thanks to each of the several dozen #CTO, #CIO, #infrastructure and #cloud leaders who attended the second session of the #Cloud Leader Forum that McKinsey & Company / McKinsey Digital held yesterday in #nyc.

Some themes that I heard:
- Increasing attent…https://t.co/sUuWxX7nDS

[James Kaplan @jmk37]

Thanks to each of the several dozen #CTO, #CIO, #infrastructure and #cloud leaders who attended the second session of the #Cloud Leader Forum that McKinsey & Company / McKinsey Digital held yesterday in #nyc.

Some themes that I heard:
- Increasing attent…https://t.co/3JFZScEzc5

[James Kaplan @jmk37]

Given the accelerated pace of #tech change, the current models of harnessing #innovation are not sufficient. My McKinsey & Company / McKinsey Digital colleague @SteveVanKuiken's recent Harvard Business Review article discusses how…https://t.co/TNohID1kF6 https://t.co/reJFX2vzY7

[James Kaplan @jmk37]

My McKinsey & Company / McKinsey Digital colleague Charlie Lewis and I speak with Lena Smart, Felix C. and Amy Berman of MongoDB about building a #cybersecurity #culture. https://t.co/7NfvxjUKIQ

[James Kaplan @jmk37]

Five learnings on #cloud strategy based on detailed profiles of more than 50 #enterprisetech cloud programs -- new article from my McKinsey & Company / McKinsey Digital colleagues @guberguber, Kaavini Takkar, Brendan Campbell and V…https://t.co/0rgSB55ln6 https://t.co/jgN6YDI0Qg

[James Kaplan @jmk37]

Across industries and regions, the evidence is clear: #tech is increasingly a key part of every organization’s business strategy. Yet #boardofdirectors often struggle with the best way to engage in technology, and there continues…https://t.co/yWqVDPiBJZ https://t.co/Ue4XbBcusU

[James Kaplan @jmk37]

I wonder if there is some parallel between adoption of #cloudcomputing and adoption of enterprise #VOIP (with #cloud having 10x or more the complexity)

- VOIP was a clearly superior technical model
- Transition costs were sobering for many companies
- Bu…https://t.co/DMgfMp7Uhq

[James Kaplan @jmk37]

The results of our survey of more than 1,300 business leaders and 3,000 consumers globally suggest that establishing #trust in products and experiences that leverage #AI, #digital technologies, and #data not only meets consumer ex…https://t.co/a8xk59JeFJ https://t.co/0KTa7Jlao6

[James Kaplan @jmk37]

What do #cxo leaders at #insurance carriers need to know about #cloudcomputing?

My McKinsey & Company / McKinsey Digital colleagues @SteveVanKuiken, Sanjay Kaniyar, Binu Sudhakaran, Mathew Lee and Ani Majumder suggest the followin…https://t.co/1pFJo9RNDQ https://t.co/fTtrf1XROX

[James Kaplan @jmk37]

#banks are using more analytical solutions and tools to analyze and mitigate the threat of #cyberattacks . As the use of cyber models grows, so too do the risks related to the design and use of those models. This is why the focus on #modelriskmanagement f…https://t.co/2RKgbUwr8b

[James Kaplan @jmk37]

Here’s how to build a comprehensive, measurable, and objective end-to-end risk appetite framework as a foundation for managing #techrisk and #cyber #risk.

From my McKinsey & Company McKinsey Digital colleagues Lucy Shenton, Daniel…https://t.co/Fti517naxn https://t.co/JRxnGGTAKv

[James Kaplan @jmk37]

Watching my colleagues Jeffrey Caso and Luisa Armstrong present on #cloudcomputing #cybersecurity trends at Amazon Web Services (AWS) #reinforce. https://t.co/b7XAIKEgRm