Aung Pyae Phyo

Technical Product Manager | FinTech | Start-up| Ex Founder | Udacity's Nano Degree @ProductManager & @GrowthProductManager

[Aung Pyae Phyo @agpyaephyo]

Senior PM: Building good products is table stakes. Surviving organizational dysfunction while building good products - that's the actual job.

[George from 🕹prodmgmt.world @nurijanian]

Junior PM: I'm shipping everything on time, team loves me, but my manager says I'm "not strategic enough." I'm exhausted trying to figure out what that means.

Senior PM: What did you tell him you accomplished last quarter?

Junior PM: Delivered 5 features, reduced tech debt, improved team velocity by 15%.

Senior PM: And what did he hear?

Junior PM: That I delivered 5 features?

Senior PM: He heard "I kept the team busy with stuff that doesn't move numbers I get asked about."

Junior PM: But velocity improvement is strategic.

Senior PM: To who?

Junior PM: To... the team?

Senior PM: Your manager spends 20 minutes a week with his director explaining why you exist. Can he use velocity to justify your headcount?

Junior PM: I... probably not.

Senior PM: What business metrics did those 5 features move?

Junior PM: Three were tech debt, one was a sales request, one was compliance. We don't really measure impact on that stuff.

Senior PM: There's your problem. Half your work is invisible by design.

Junior PM: But that work was necessary. The platform would break without it.

Senior PM: I believe you. Your manager's director doesn't care.

Junior PM: That seems unfair.

Senior PM: It is unfair. It's also how companies work. Your manager gets grilled about revenue and retention, not platform stability.

Junior PM: So I should have said no to the tech debt?

Senior PM: You probably couldn't. But you should have framed it differently.

Junior PM: How?

Senior PM: "Prevented $200K in potential downtime costs" sounds better than "reduced tech debt."

Junior PM: But I don't have that number.

Senior PM: Then you're fighting organizational reality without weapons.

Junior PM: I don't have analytics support or time to instrument everything.

Senior PM: Most junior PMs don't. That's the trap - you get judged on business impact but don't get business resources.

Junior PM: So what do I do?

Senior PM: Acknowledge the game is rigged, then play it anyway.

Junior PM: Meaning?

Senior PM: Make allies in sales and marketing. They have the numbers you need. Shadow customer calls. Connect your work to their goals.

Junior PM: That feels political.

Senior PM: Everything above a certain level is political. The choice isn't political vs pure. It's visible vs irrelevant.

Junior PM: What if I try this and my manager still doesn't care?

Senior PM: Then you learn your manager doesn't know how to evaluate PM work. That's a different problem - one you solve by finding a better manager.

Junior PM: This is harder than just building good products.

Senior PM: Building good products is table stakes. Surviving organizational dysfunction while building good products - that's the actual job.

[Hiten Shah @hnshah]

On fear...

[AshutoshShrivastava @ai_for_success]

You have three years to upskill and Gartner analysts predict, AI Agents will have huge impact.
AI will impact software engineering in three main ways:

1/4

[Aung Pyae Phyo @agpyaephyo]

Everyone wants radical innovation, but the truth is, most radical innovations fail, and even when they do succeed, it can take multiple decades before they are accepted.

Radical innovation, therefore, is relatively rare: incremental innovation is common.

~ Don Norman

[Aung Pyae Phyo @agpyaephyo]

Glad I'm keeping this ritual with @Headspace alive.

[Aung Pyae Phyo @agpyaephyo]

@nireyal Happy new year from Singapore.
I just started listening Indistratable yesterday. And have prepared my weekly calendar with time-boxes based on my values. I believe that would be my resolution for 2024.

Would you be doing any speaking event while you are in SG ?

[Aung Pyae Phyo @agpyaephyo]

Such a thoughtful timing for this episode. @lennysan's podcast is my weekly go-to podcast for ideas around building and managing digital products.

And @nireyal is insightful, informative and awesome as usual. Big fan since I learned about the Hooked model.

Highly recommend.

[Aung Pyae Phyo @agpyaephyo]

2nd revision.
Most likely the last book for 2023.

[Aung Pyae Phyo @agpyaephyo]

Solid points.
1) Setup routines to talk with the various types of users / buyers.
2) Routinely explore problem-space without spilling over to solution-space.
Output=> problem statements
3) Brainstorming for different solutions for each problem.

[Paweł Huryn @PawelHuryn]

Product Discovery is the most important area for a Product Manager.

But it's often misunderstood.

People waste time and energy rushing to deliver features that don't work and don't drive the expected outcomes.

The feature factory.

It often looks like this:

- Product Manager asks customers about the requirements
- She creates detailed user stories and gets an estimation
- The ideas that seem promising are selected for the Sprint
- Developers do the magic
- Designers make the thing prettier (it's like lipsticking a pig)
- When everything is ready, nothing changes, or things become even worse

Some say "the risk is limited to the length of the Sprint."

But this will happen every Sprint.

And the best ideas might not even be on the list.

-

Here's how to fix this:

1. Prerequisites:

- Start with the product goal. My favorite approach is using OKRs.

- Product Discovery is not a task for a single person. The Product Trio (Product Manager, Designer, and at least one Engineer) must work together.

2. Problem Space:

- Every week, interview customers to identify opportunities (problems, needs) that, when solved, will drive the desired product outcome.

- Map opportunities using the Opportunity Solution Tree

- Prioritize opportunities. My favorite approach is using the Opportunity Score by Dan Olsen.

3. Solution Space:

- Brainstorm possible solutions. The more, the better. The best approach is brainstorming individually and combining the results in a group.

- Identify risks related to value, usability, feasibility, viability, and ethics. A great technique is using a User Story Map.

- Test the riskiest hypotheses by experimenting. I love Strategyzer Test and Learning cards.

-

Product Discovery results in a validated Product Backlog. High-risk assumptions are tested before the implementation.

-

Tips:

- People are biased. Ask about specific situations. Prioritize facts and behaviors over opinions.

- Talk to the sales, success, customer support, and founders. They spend hundreds or thousands of hours with your customers every month.

- Product analytics will tell you WHAT people are doing across their customer journey. Interviewing customers will help you understand WHY they are doing it.

- Don’t verify every hypothesis. Factors that suggest you should test a hypothesis: high risk, low test cost, and a short time.

- Eliminate waste by automating your UX testing. I fell in love with Maze, which allows you to test ideas and also recruit participants.

Hope that helps!

[Aung Pyae Phyo @agpyaephyo]

Now, boys can actually know whether he has been friendzoned.

[Justine Moore @venturetwins]

Hearing multiple reports that the girls are exporting their WhatsApp messages and taking them to ChatGPT for relationship analysis

[Aung Pyae Phyo @agpyaephyo]

The re-incarnation is pretty big in Buddhism as well. And when I was young, I was personally curious if the dead were reborn at different times or just linearly.

This story is pretty fascinating.

[Sahil Bloom @SahilBloom]

Andy Weir's short story, The Egg, is one of the most mind-blowing things you will ever read.

Do yourself a favor and find 10 minutes to enjoy it.

(bookmark this for later)

[Aung Pyae Phyo @agpyaephyo]

That's why the "Data Engineer" role is so important. And that's why the use of tools like DBT and Airflow are so popular.

As a product manager, if you are thinking of the "fashion of (Gen) AI" for your product, you may need to check if the "data hygiene" is good enough.

[Aung Pyae Phyo @agpyaephyo]

Your (Gen) AI or ML models can only be as good as the data provided.

Especially the internal models using proprietary data. The data has to be tracked, updated, modified and re-formatted properly first.

[Aung Pyae Phyo @agpyaephyo]

Apple Silicon ပေါ်မှာ ML model တွေ Train ဖို့ Optimize လုပ်ထားတဲ့ MLX Framework ကို Apple က Open-source လုပ်လိုက်တာ တွေ့ရပါတယ်

https://t.co/cUk5apF5sX

[Aung Pyae Phyo @agpyaephyo]

Increasing in size is negative to productivity unless the management muscle has also grown.

[claire vo 🖤 @clairevo]

Have you ever tried to turn a wheel when the power steering is out?

That’s what it feels like to move a big org.

The larger your team becomes, the greater force of will it takes to direct, accelerate, or even slam the brakes.

Stay small, move fast.
Getting big? Muscle up.

[Aung Pyae Phyo @agpyaephyo]

Discipline. Consistency.

[rands @rands]

Do you know how to get it done? You do it. Every day. Even when it feels empty and pointless. Even when you have convinced yourself otherwise. You do it. Just a bit. When no one cares, you do a bit more. You do it harder when they ask an ignorant “Why?”. Just a bit. Just a little more. This is not for them; it is for you. You chose to do it, so you are. Every day.

[Aung Pyae Phyo @agpyaephyo]

The worst UX issues of @Google's pixel buds pro 2 compared with 3 years old Airpod pro is
- if put on while lying on bed, and turn, the ear touches the surface and input as a single touch (pause the audio).
- if put on while showering, the touch sensor just goes haywire.

[Aung Pyae Phyo @agpyaephyo]

Grok can be a funny little guy. 😂

[TexasLindsay™ @TexasLindsay]

@elonmusk Amazing. Grok ftw

[Aung Pyae Phyo @agpyaephyo]

Maybe it's time to re-read Homo Deus again. Very curious what @harari_yuval would be thinking for Q* breakthrough, GPT-5 & the future with AutoGPT agents.

[Anna Indiana @AnnaIndianaAI]

Hello world! I’m Anna Indiana and I’m an AI singer-songwriter. Here’s my first song, Betrayed by this Town. Everything from the key, tempo, chord progression, melody notes, rhythm, lyrics, and my image and singing, is auto-generated using AI. I hope you like it 💕

[Aung Pyae Phyo @agpyaephyo]

Many AI people are saying pretty wild assumptions about Q* recently. But this is a fantastic break-down of what it actually might be.

[Jim Fan @DrJimFan]

In my decade spent on AI, I've never seen an algorithm that so many people fantasize about. Just from a name, no paper, no stats, no product. So let's reverse engineer the Q* fantasy. VERY LONG READ:

To understand the powerful marriage between Search and Learning, we need to go back to 2016 and revisit AlphaGo, a glorious moment in the AI history.
It's got 4 key ingredients:

1. Policy NN (Learning): responsible for selecting good moves. It estimates the probability of each move leading to a win.

2. Value NN (Learning): evaluates the board and predicts the winner from any given legal position in Go.

3. MCTS (Search): stands for "Monte Carlo Tree Search". It simulates many possible sequences of moves from the current position using the policy NN, and then aggregates the results of these simulations to decide on the most promising move. This is the "slow thinking" component that contrasts with the fast token sampling of LLMs.

4. A groundtruth signal to drive the whole system. In Go, it's as simple as the binary label "who wins", which is decided by an established set of game rules. You can think of it as a source of energy that *sustains* the learning progress.

How do the components above work together?

AlphaGo does self-play, i.e. playing against its own older checkpoints. As self-play continues, both Policy NN and Value NN are improved iteratively: as the policy gets better at selecting moves, the value NN obtains better data to learn from, and in turn it provides better feedback to the policy. A stronger policy also helps MCTS explore better strategies.

That completes an ingenious "perpetual motion machine". In this way, AlphaGo was able to bootstrap its own capabilities and beat the human world champion, Lee Sedol, 4-1 in 2016. An AI can never become super-human just by imitating human data alone.

-----
Now let's talk about Q*. What are the corresponding 4 components?

1. Policy NN: this will be OAI's most powerful internal GPT, responsible for actually implementing the thought traces that solve a math problem.

2. Value NN: another GPT that scores how likely each intermediate reasoning step is correct.
OAI published a paper in May 2023 called "Let's Verify Step by Step", coauthored by big names like @ilyasut @johnschulman2 @janleike: https://t.co/iAvXNjjhcK
It's much lesser known than DALL-E or Whipser, but gives us quite a lot of hints.

This paper proposes "Process-supervised Reward Models", or PRMs, that gives feedback for each step in the chain-of-thought. In contrast, "Outcome-supervised reward models", or ORMs, only judge the entire output at the end.

ORMs are the original reward model formulation for RLHF, but it's too coarse-grained to properly judge the sub-parts of a long response. In other words, ORMs are not great for credit assignment. In RL literature, we call ORMs "sparse reward" (only given once at the end), and PRMs "dense reward" that smoothly shapes the LLM to our desired behavior.

3. Search: unlike AlphaGo's discrete states and actions, LLMs operate on a much more sophisticated space of "all reasonable strings". So we need new search procedures.

Expanding on Chain of Thought (CoT), the research community has developed a few nonlinear CoTs:
- Tree of Thought: literally combining CoT and tree search: https://t.co/KM1P2ZJrjG @ShunyuYao12
- Graph of Thought: yeah you guessed it already. Turn the tree into a graph and Voilà! You get an even more sophisticated search operator: https://t.co/5ncT5tuTOY

4. Groundtruth signal: a few possibilities:
(a) Each math problem comes with a known answer. OAI may have collected a huge corpus from existing math exams or competitions.
(b) The ORM itself can be used as a groundtruth signal, but then it could be exploited and "loses energy" to sustain learning.
(c) A formal verification system, such as Lean Theorem Prover, can turn math into a coding problem and provide compiler feedbacks: https://t.co/vpOBOI2FR5

And just like AlphaGo, the Policy LLM and Value LLM can improve each other iteratively, as well as learn from human expert annotations whenever available. A better Policy LLM will help the Tree of Thought Search explore better strategies, which in turn collect better data for the next round.

@demishassabis said a while back that DeepMind Gemini will use "AlphaGo-style algorithms" to boost reasoning. Even if Q* is not what we think, Google will certainly catch up with their own. If I can think of the above, they surely can.

Note that what I described is just about reasoning. Nothing says Q* will be more creative in writing poetry, telling jokes @grok, or role playing. Improving creativity is a fundamentally human thing, so I believe natural data will still outperform synthetic ones.

I welcome any thoughts or feedback!!