Liwei Mao

building Blu so kids can accelerate in math. ex @CashApp, @button

[Liwei Mao @liweimao]

We actually ask pre-trieval question for each skill in a lesson to allow the student to skip the instruction for that specific skill

Format is a mix of mcq, free response, drag and drop, so there is a chance of guessing / some noise. But I'd say it's largely averaged out due to the sheer size of the data. We find kids who get the pre-trieval right overwhelmingly passes the lesson

[Liwei Mao @liweimao]

We looked at 1.1M answers from 2,700 kids. The biggest surprise: the worked example is doing most of the heavy lifting.

Strong initial instruction matters the most for the 1/3 who need it. https://t.co/9HFGf22b8d

[Liwei Mao @liweimao]

we did a lot of backtesting and found interestingly that that kids who missed twice in a row still go on to pass the lesson more than half the time, so it's not a good predictor.

We briefly had the AI tutor kick in earlier and the results were actually slightly worse. Some of the kids who figured out what they were doing wrong were impatient and annoyed by the AI tutor and would spam thru it and fail as a result.

3 consecutive wrong on the other hand, was highly predictive of failure. They had less than a 10% chance of failing. So presenting the AI tutor at the moment they really need it had better results

[Liwei Mao @liweimao]

That’s a very common misconception we have as adults.

We assume you can just present advanced problems to a kid and they’ll figure it out. But think back to 3rd grade. Did you invent column addition just by being shown the problem? What about the area of a circle?

To derive the underlying structure, you already need a fairly advanced schema of how math works. And to build that schema, especially for a kindergartener, you have to show and tell them about each foundational block, then have them solve and solidify that knowledge.

[Liwei Mao @liweimao]

Lots of GT kids are like this!

They like to figure out the solution themselves rather than read the worked examples.

I wonder if because she spends a long time deriving the initial problems herself, the later practice gets boring -- not as much dopamine as the first one.

One thing you can try is not letting her skip topics, but having her solve fewer practice problems under each one. That's what we do in our app and what Englemann recommends, you move kids forwards as soon as they show they can do it independently (for example, 3 consecutive correct answers if a clean streak). But you can tune and personalize it for her. She may need fewer or more. Exposure to different problem variants matters a lot more than doing the exact same variant multiple times once she's grasped it.

If it takes her painstakingly long not just on the first question, but on subsequent questions, that might be a fluency issue. If so, prioritizing drilling & additional practice on fluency procedures can make her faster by a lot!

[Liwei Mao @liweimao]

@YitzyMytelka for sure, recognizing the underlying structure is what lets kids solve new problems.

Carefully guiding them there is a big part of sequencing, which we believe matters most. Our data held that fixed while examining the other components.

[Liwei Mao @liweimao]

What matters most in a math lesson?

1.1M answers from ~2,700 kids, 331 lessons:

• With great sequencing, 2/3 of kids don't need the tutorial
• Strong instruction is critical for the other 1/3
• Worked examples do most of the heavy lifting
• AI tutor takes kids about to fail from 19% to 43%

Full write-up: https://t.co/e5vmZ4RDRh

[Liwei Mao @liweimao]

We looked at 1.1M answers from 2,700 kids. The biggest surprise: the worked example is doing most of the heavy lifting.

Strong initial instruction matters the most for the 1/3 who need it. https://t.co/9HFGf22b8d

[Liwei Mao @liweimao]

thanks!

1) Interleaving is sooo important for testing whether or not the student truly grasped the underlying concept

2) When it comes to sequencing & initial instruction, teaching related things together (like multiplication and division word problems) makes a huge difference for preventing misrules later. We don't quite mix worked examples -- we explicitly show contrasting pairs so kids can see when it’s a multiplication vs division story

[Liwei Mao @liweimao]

@supermathlily 💯 we are frequently finding things that surprise us when we dig deep into the data

[Liwei Mao @liweimao]

@ben_m_somers ty

[Liwei Mao @liweimao]

@amaankahmad Thanks

[Liwei Mao @liweimao]

@ganz We love data! Will be sharing more data breakdowns including some experiment results soon

[Liwei Mao @liweimao]

@gtmom 💯 she's sharp as a whip

[Liwei Mao @liweimao]

had a 45 min chat yesterday with Marcy Stein. Truly legendary.

She recommended this 1966 Engelmann video with pre-K kids. Watching them go from simple addition through basic algebra in one session challenged every assumption I had about Direct Instruction.

[Liwei Mao @liweimao]

@yiran__c thx for catching! fixed

[Liwei Mao @liweimao]

https://t.co/slu9dz3pCF

[Liwei Mao @liweimao]

@mpershan Idk if every life skill session is every parent or kids tea, but maybe it doesn't have to be.

I will say the younger kids are shocking adept at using ai, which is why it's banned during academics time

[Liwei Mao @liweimao]

@mpershan Unless you are min/maxing your kid on academics for research later or whatever, it feels like a more reasonable model than having every kid spend a fixed number of hours on academics a day