Robert Kirk

[Robert Kirk @_robertkirk_]

@notkevinzhang Building a pcb component selection engine

[Robert Kirk @_robertkirk_]

Good example of the difference between Sol and Fable:

I was optimizing a system by evaluating with a benchmark.

Sol made good progress towards increasing the score.

Fable realized sol had been optimizing for the bench and created an opaque eval so the improvements generalize.

[Robert Kirk @_robertkirk_]

@ex0t1clol @OpenAIDevs Flex inference takes ~5-10 mins per turn or so. So if modeled after that it would be dramatically slower but still useful for specific things.

[Robert Kirk @_robertkirk_]

@thsottiaux Would it be possible to add webhook support to codex similar to scheduled actions right now? Can basically do it already by polling but webhooks would be better.

[Robert Kirk @_robertkirk_]

@pa1ar Does anyone know how well dynamic reasoning effort works on luna max? Is "max" just the highest it will go, or a recommendation, or some combination?

[Robert Kirk @_robertkirk_]

@adonis_singh Is Whispr flow significantly better than the codex universal voice-to-text feature?

[Robert Kirk @_robertkirk_]

@0xSero You need to consider cost not token count

[Robert Kirk @_robertkirk_]

@Cloudflare @CloudflareDev I can't access my support cases, it redirects me to this. Also haven't gotten any support in 3 weeks.

[Robert Kirk @_robertkirk_]

@MikushRab Seems like the issue is image input / spatial reasoning. Would be interested to see this on gemini

[Robert Kirk @_robertkirk_]

@paulg @Austen Do you think weaker models will eventually get good enough for most use cases and become broadly commoditized? Right now the major labs’ biggest moat is just having the best model

[Robert Kirk @_robertkirk_]

@gdb Can we get this natively in codex?

[Robert Kirk @_robertkirk_]

In computer architecture work is done optimizing for specific benchmarks because they’re what reviewers use. The problem is 99% of real users do nothing like the benchmark workloads.

[dax @thdxr]

imagine a benchmark of two editors

one opens a file 10x faster! wow it must be better. but oh wait they just disabled syntax highlighting

real products have to do things that make it worse on benchmarks but better in practice

[Robert Kirk @_robertkirk_]

@steeliewheel @alex_prompter Definitely problems, though many of these are somewhat independent from AI right?

[Robert Kirk @_robertkirk_]

@theo Maybe the prompts should be the unit of division?

[Robert Kirk @_robertkirk_]

@mcuban Perhaps LLMs for enterprise AI need access to previous answers to (cosine) similar questions so they can be more consistent.

[Robert Kirk @_robertkirk_]

@pmarca “Make your answers as long and detailed as you possibly can” is surprising. I find it’s easy to ask for more but you often want a quick overview first.

[Robert Kirk @_robertkirk_]

@OpenAIDevs You should add latest model names to the system prompt. It’s really annoying when it keeps saying a new model doesn’t exist

[Robert Kirk @_robertkirk_]

@cormachayden_ My poor mac battery

[Robert Kirk @_robertkirk_]

@sama Cheap models with super fast ttft and good instruction following unlock a lot of applications that are too slow right now