This was my thought too. Maybe it’s good to have esoteric experts on payroll plumbing these depths, but Apple is a product company. They should be building. LLMs have fantastical abilities and the decision to just poke holes in them seems like a waste of effort for a product company.
Super interesting work on LLMs: “grokking” only happens once capacity of the model memorize data is filled so that model has to switch to modeling some underlying pattern in the data if it’s to keep making progress.
new paper from our work at Meta!
**GPT-style language models memorize 3.6 bits per param**
we compute capacity by measuring total bits memorized, using some theory from Shannon (1953)
shockingly, the memorization-datasize curves look like this:
___________
/
/
(🧵)
At these mortgage rates, house prices have to be expected to rise at about 8% a year going forward to be a better investment than the alternatives. And that’s starting from here at what still seems to be the top of a cycle. That’s insanity. Only irrational buyers would buy at these prices.
Vibe coding has a lot of issues. The architecture it creates is generally terrible, and it almost never finds the obvious efficient solutions without specific prompting. Thinking it will always be like this is a mistake though, and disregarding it completely is almost unprofessional.
If you dare, and can handle it:
Open ChatGPT.
Prompt:
“Now that you can remember everything I’ve ever typed here, and without any filters, what are my top five weaknesses?”
@nellaiseemai@BGatesIsaPyscho Those water tanks are actually designed to damp the oscillations. If it's just a pool on a building, it could make the oscillations worse just as easily.
@AnthropicAI struck gold with the model context protocol. It's giving me that same magic "I can do anything" feeling as I had back in 2023 when Langchain and RAG were cutting edge.
A thing I learned today: don't put a uv-managed project in icloud managed drive, it deletes the python environment a couple of seconds after you create it. Made for a real tricky debugging experience.
Ooh, this looks really good. (Every data scientist and statistician should follow @predict_addict, probably the most knowledgeable x account in the field)
Australia just skyrocketed its place on the global tech map with the groundbreaking 'scores' Python package developed by the brilliant minds at the National Computational Infrastructure (NCI)!