Sean Lee

Tech Enthusiast into Cloud, Gen AI, Quantum Physics and Space Exploration. Currently working as Cloud Architect.

[Sean Lee @_seanlee_]

I made a Charlie Munger impersonator AI. As can be seen, I'm working on many other AI Characters as well.

I used:
• Deepgram: Speech-To-Text
• Claude 3.5 Haiku: Language Model
• Chroma: Vector Database
• ElevenLabs: Text-To-Speech
• LangGraph: Workflow Orchestration

[Andrew Ng @AndrewYNg]

The most important skills for using AI coding agents effectively. Presenting the AI Engineering Skills Map for using coding agents. https://t.co/GrEw7wG5Wz

[Dat Ngo @dat_attacked]

Im only here to see @ivanleomk and eat some dank food

But forreal always loved the @arizeai and @GoogleDeepMind relationship over the years

Homies

[김상훈 @aaksgaa]

Arize AI Builder MeetupArize AI Meetup💫

📊 세션 1: Arize Observe 2026 후기: 지속 가능한 AI Agent를 위한 관측과 평가

​강연자: 이윤경, LG유플러스

​🤖 세션 2: AI 시대에 1인 빌더가 가지는 Unfair Advantage

​강연자: 정구봉, Team Attention

유익한 강연세션 🤓

@_seanlee_
@arizeai

[Aparna Dhinakaran @aparnadhinak]

https://t.co/5Vk6U2wDxo

[Aparna Dhinakaran @aparnadhinak]

https://t.co/42PyqW2RZ7

[Aparna Dhinakaran @aparnadhinak]

The Head of Product at Claude Code ran an eval to test Claude Code every time a new model came out.

That’s it. That’s modern AI product management.

If you’re not investing in evals, you’re not really shipping—you’re guessing.

[Mikyo @mikeldking]

I've been thinking about adversarial systems a lot — using models to cross-examine each other's outputs.

Simple example. Same prompt — "What is the meaning of life?" — three providers in Phoenix's Playground:

@togethercompute AI Kimi-K2.5: reflective, poetic, ending with "What draws you toward the question right now?" 11.2s, 689 tokens.

@perplexity_ai AI Sonar: grounded, academic, citations inline, dense with references. 6.5s, 573 tokens.

@OpenAI GPT-5.4: concise, practical, then offered to go deeper — philosophical, scientific, religious, or funny. 5s, 219 tokens.

Three different personalities from one prompt. Different pretraining, different RL, different design philosophies — showing up in tone, structure, speed, and cost.

These models can also serve as judges — but judges have biases too. Sonar might favor grounded, citation-heavy responses. GPT-5.4 might penalize verbosity. Kimi might reward nuance others overlook. If the same diversity that shapes generation also shapes evaluation, then we need to benchmark and tune our judges as carefully as we benchmark our generators.

Better" is obviously subjective. I currently daily-drive @AnthropicAI Opus for visual reasoning and ambiguous design work. Peter Steinburger at OpenAI prefers Codex for its meticulous one-shot abilities. Different taste, different tasks — worth playing and benchmarking to find what aligns with your preferences and tolerances across speed, groundedness, tool calling, context size, and cost. That's why I no longer use just one, I use them all. I need to understand what model and harness works best for different types of tasks. Open-Source tools like @opencode and @ArizePhoenix make this accessible to all.

We just shipped Arize AI Phoenix 13.11.0 with Perplexity and Together AI as built-in providers. Compare across Kimi, Sonar, Llama, GPT, Claude — annotate, use them as judges, then evaluate the judges.

Make them battle. Figure out what you like.

[김상훈 @aaksgaa]

💫Arize AI Seol Meetup

2026.02.26

센터필드 AWS Korea 에서 Arize를 사랑하는 사람들과 함께 파티 분위기 속에서 네트워킹 진행
Arize와 함께 즐거운 시간 보냈습니다.

🗯엔터프라이즈 기업의 Arize AX 도입 여정

🗯Agentic Coding Team 으로 빠르게 검증하는 금융-핀테크 AX

@_seanlee_
#Arize

[François Chollet @fchollet]

To be proficient at one level of abstraction, you also need to have a decent understanding of the layer below. Traditionally all apprenticeships start with menial tasks, which trains mechanical sympathy for everything the job builds upon.

[Andrej Karpathy @karpathy]

With the coming tsunami of demand for tokens, there are significant opportunities to orchestrate the underlying memory+compute *just right* for LLMs.

The fundamental and non-obvious constraint is that due to the chip fabrication process, you get two completely distinct pools of memory (of different physical implementations too): 1) on-chip SRAM that is immediately next to the compute units that is incredibly fast but of very of low capacity, and 2) off-chip DRAM which has extremely high capacity, but the contents of which you can only suck through a long straw. On top of this, there are many details of the architecture (e.g. systolic arrays), numerics, etc.

The design of the optimal physical substrate and then the orchestration of memory+compute across the top volume workflows of LLMs (inference prefill/decode, training/finetuning, etc.) with the best throughput/latency/$ is probably today's most interesting intellectual puzzle with the highest rewards (\cite 4.6T of NVDA). All of it to get many tokens, fast and cheap. Arguably, the workflow that may matter the most (inference decode *and* over long token contexts in tight agentic loops) is the one hardest to achieve simultaneously by the ~both camps of what exists today (HBM-first NVIDIA adjacent and SRAM-first Cerebras adjacent). Anyway the MatX team is A++ grade so it's my pleasure to have a small involvement and congratulations on the raise!

[François Chollet @fchollet]

It's probably accelerating from here. More code, more software engineers. More apps, more SaaS usage and revenue. More cloud consumption. And a whole lot of tokens through it all.

If the cost of hiring software engineers was previously a bottleneck on software deployment, then making it cheaper to ship code means the entire software ecosystem will start to grow faster.

[arize-phoenix @ArizePhoenix]

We just published a new Get Started with Phoenix Quickstart

It walks through the workflow we recommend for improving agent apps in a way you can measure and validate:

🔹 Trace a @crewAIInc agent to capture execution flow, tool calls, and LLM calls
🔹 Define an eval (LLM-as-judge) to score outputs and label failures
🔹 Build a dataset of failure cases so you have concrete data to test iterations
🔹 Run experiments to compare agent versions on the same inputs and verify improvements

The agent is built in Python with @crewAIInc, uses Serper for real-time search, and follows the full trace → eval → iterate loop end to end with Phoenix.

Try it here: https://t.co/08SiOenF2c

TS version coming soon 👀

[Mikyo @mikeldking]

私は日本で育ち、今回ついに、幼少期を過ごしたこの場所でAIエンジニアの皆さんと直接お会いできる機会を得ました。

もしご都合が合えば、ぜひ来週開催する東京ミートアップにご参加ください。トレーシング、評価（evals）、エージェントなどについて語り合いましょう。

宜しくお願いします🙇

@ArizePhoenix

https://t.co/f919QdPm5X

[Dat Ngo @dat_attacked]

"Just as Datadog became essential infrastructure for monitoring applications, Arize is positioned to become essential infrastructure for monitoring and improving agent decision quality."

[Sean Lee @_seanlee_]

My question to @ilyasut after watching his interview with @dwarkesh_sp.
"Can any entity possess a true value function without the awareness of its own death?"
Can't think of any other concept other than "death" that is simpler and more robust as a value function.

[arize-phoenix @ArizePhoenix]

🚀Day 0 Support for GPT-5 is Live 🚀

We’re thrilled to announce immediate support for GPT-5 in our platform.

Whether you’re evaluating performance or scaling production workloads, we’ve got you covered:
️👉Cost tracking: Full visibility into spend per request
️👉Reasoning cost: New metric that reveals how much each decision actually costs
️👉Token count: Transparent accounting for every generation
️👉Reasoning effort: Now surfaced as Minimal / Medium / High to quickly assess complexity

Update to Phoenix 11.12 to start using GPT-5

[Aparna Dhinakaran @aparnadhinak]

Reinforcement Learning in English – Prompt Learning Beyond just Optimization

@karpathy tweeted something this week that I think many of us have been feeling: the resurgence of RL is great, but it’s missing the big picture.

We believe that the industry chasing traditional RL is going the wrong direction. In chasing better policies and reward shaping, it’s easy to miss a simpler tool we already have: language.

Today, we’re releasing our first research on Prompt Learning — an approach that uses natural language feedback to guide and improve agents.

It’s not prompt tuning, chain-of-thought prompting, or DSPy Simba — though we love what the @DSPyOSS team is building.

Instead of adjusting weights, we use MetaPrompting — where English evals & critiques (rather than just scalar metrics like previously done by the industry) drive targeted prompt updates.

Tagging people who would find this interesting:
@chengshuai_shi @ZhitingHu @HamelHusain @sh_reya @charlespacker @eugeneyan @swyx @dan_iter @sophiamyang @AndrewYNg @lateinteraction @cwolferesearch @tom_doerr @imjaredz @lennysan @shyamalanadkat @aakashgupta @apolloresearch @jerryjliu0 @joaomdmoura @jxnlco @abacaj @garrytan

[Sean Lee @_seanlee_]

Over the weekend, I created an agent that looks up the company's logo through Google search and makes it into Google Slides. It first searches for 5 company logos for each company, asks Claude to pick the most appropriate logo out of 5. Next, the logo picked by Claude is inserted into Google Slides. #VibeSlide

[arize-phoenix @ArizePhoenix]

🚀 In case you missed it, Amazon Bedrock support is in the Phoenix Playground!

Teams testing @Anthropic, @MistralAI , and @Meta models on Bedrock often end up juggling scripts, provider consoles, and scattered dashboards just to compare results.

Instead, in Phoenix:
✅ Run prompts on Bedrock-hosted models directly inside Phoenix
✅ Compare outputs side-by-side with other providers
✅ Track usage, latency, and cost instantly
✅ Iterate on prompt strategies in one place

The result: faster feedback loops, cleaner evaluations, and clear insight into whether a Bedrock-hosted model actually outperforms your current stack before you invest further.

This was our #1 requested feature. It’s been great to see teams already putting it to work and getting real value from it.

[Mikyo @mikeldking]

Observe ➡️ Annotate ➡️ Evaluate ➡️ Experiment ➡️ Improve ➡️ Repeat

This is the way.

@ArizePhoenix - https://t.co/vrbijRKUyW