Charles C. Lee

also go by Charlie. staff eng @ Shopify working on all things magic ✨. Would love to hear about what you’re building!

[Evadne W. @evadne]

Recursive Language Models with Elixir. Of course. The solution was in front of us all this time.

[will brown @willcb]

if you’re building a RAG system in 2025 just build a good search engine backend + let the model query it

[Charles C. Lee @ReallyChar]

All I can say is we've had a blast working on this iteration - 🚀

[tobi lutke @tobi]

Major sidekick update rolling out right now. Excited for this.

[Greg Kamradt @GregKamradt]

How structured outputs work under the hood (via breakout at OpenAI DevDay)

Guess why the first structured output request is slow, but the 2nd+ is fast?

Engineering:
* Unconstrained token decoding isn't good. The model could pick any token.
* Limiting which tokens can be produced next is what works. They use "token masking" for this. Constrains the token which are picked at the end of sampling
* After generating probabilities with a forward pass, ignore (or mask) tokens you know aren't applicable.
* Mask tokens need updating at every step of the inference. Needs to be really fast. Mask determination happens on the CPU. Needs it to be as quick, as quick as time between tokens. <10ms
* They build up an index with the JSON schema you apply to speed up mask making
* Index Making: Grammar > Parser > Iterate over all tokens and parse states > Index. Index once, cache later. First query to structured outputs go slow, then the 2nd+ query is quick.
* Recursive schema makes this hard. But they believe they will be a big deal so they are moving towards it.
* CFG: Context Free Grammar approach. Regular expression + stack. Their attempt at indexing

Research
* Retraining and fine tuning gets you to 100% schema adherence

API Design
* Additional properties: What do you do with properties that were note defined in the schema? Decided to disallow them
* Design principle, be explicit instead of implicit. Requiring developers to set properties. It's harder to use, but communication is more clear
* Design principle: All properties are design by default. It makes optional params nullable
* Decided to generate fields in the same order they were defined in the schema, this helps with requiring chain of thought for structured outputs

[Unsloth AI @UnslothAI]

We made a step-by-step tutorial on how to finetune Llama-3 with Google Colab &amp; deploy it to @Ollama

Tutorial: https://t.co/YtxXzlFOSp

Colab notebook: https://t.co/FnmHK9sysS

Blog post &amp; video coming soon. 🦥

[Harrison Chase @hwchase17]

❓What is an agent?

I get asked this question a lot, so I wrote a little blog on this topic and other things:
- What is an agent?
- What does it mean to be agentic?
- Why is “agentic” a helpful concept?
- Agentic is new

Check it out here: https://t.co/KCwYmzSd5Z

[Morten Just @mortenjust]

This is fast. Chrome running Gemini locally on my laptop. 2 lines of code.

[☼░▒▅ @thatboyweeks]

@mortenjust Query expansion like this could be promising

[Ethan Mollick @emollick]

This is an interesting overview of what it is like to actually build production software with today's LLMs. It also shows how weird LLMs are to work with from a software perspective &amp; how much we have to learn.

It is why I stress co-intelligence for now. https://t.co/DIQrGqIa5P

[Charles C. Lee @ReallyChar]

@CharlesSchwab loving the updates to the web platform! When will existing users be able to use ToS with their Schwab assets?

[Charles C. Lee @ReallyChar]

@Apple the iCloud web beta is fantastic. Hoping this comment serves as one more data point to keep whatever investment in this type of work going. 🙏

[Charles C. Lee @ReallyChar]

@overstimulat3d @rails @stimulusreflex @flydotio @realkeydb +1 to a breakdown. Also curious if you had any thoughts around hotwire vs stimulusreflex (ie. dev ux).

[Charles C. Lee @ReallyChar]

@LtSquigs Gotta get one of those mobos that have visual error codes on board.

[Charles C. Lee @ReallyChar]

@wongmjane Really hoping this is an instance of "it's gonna get worse before it gets better".

[Charles C. Lee @ReallyChar]

@mariaersh @ian_eck @jtmorehead_ @joulee Agreed with the hashtag based organization. Trade offs, oh trade offs 🙃

[Charles C. Lee @ReallyChar]

@mariaersh @ian_eck @jtmorehead_ @joulee I love notion but not having an offline mode like https://t.co/6H9YA41Sxy or Evernote makes it feel slow since it seems to be fetching all the pages on load.

[Charles C. Lee @ReallyChar]

@centralcomputer No need to be sorry, stock is limited. Glad the shops were taking safety precautions. Better luck next time. 🤞

[Charles C. Lee @ReallyChar]

@AMDRyzen absolutely killing it with the new line up.

[Charles C. Lee @ReallyChar]

@AndrewYNg DLSS 2.0 is insane for future engines in gaming &amp; media - https://t.co/gp1b8ouZIw