Brian Foote

Software Ethologist

[Brian Foote @bigballofmud]

@IntuitMachine Some are discovering that some of the quality and clarity that alluded most traditional teams may not be beyond the reach of djinn driven development…

[Brian Foote @bigballofmud]

@IntuitMachine I'm going with "certainly present, but fleeting in the day to day details"...

[Brian Foote @bigballofmud]

@EMostaque Whereas with code, they are trained on repositories with version histories, where trajectories of these refinements are visible…

[Brian Foote @bigballofmud]

@LarryKellogg Dunno about that. Recognition for achievements in onanism are already fairly commonplace...

[Brian Foote @bigballofmud]

Sometimes the Scandinavians gets it right. John Hopfield receiving the Noble Prize in Physics is their most satisfying call since ... Bob Dylan...

[Brian Foote @bigballofmud]

@WiringTheBrain Whereas I don't see it as a useful distinction...

[Brian Foote @bigballofmud]

@IntuitMachine Echoes of the steroid era in baseball. Who's juicing and who isn't. But there is no drug test for GPT. I used to ask: "Were Einstein to be shown to have used cognitive performance enhancers, would we demand he give back his Nobel Prize"...

[Brian Foote @bigballofmud]

Echoes of the steroid era in baseball. Who's juicing and who isn't. But there is no drug test for GPT. I used to ask: "Were Einstein to be shown to have used cognitive performance enhancers, would we demand he give back his Nobel Prize"...

[Carlos E. Perez @IntuitMachine]

Let's be serious now, perhaps AI shouldn't be used for writing but instead for reading only.

[Brian Foote @bigballofmud]

The problem with pre-training is, you are always the last to know...

[Brian Foote @bigballofmud]

As of my last training cut-off in September 2021, the term "bot 'splaining" is not a standard or commonly accepted term in technology or linguistics. The term seems to be a play on the term "mansplaining".

[Brian Foote @bigballofmud]

@jasongorman That a square cm worth of synapses might store and compute with 10,000 lifetimes worth of reading ... suggests for one thing that our learning speed rather than capacity might be where the bottleneck is...

[Brian Foote @bigballofmud]

A superb take on the "nature got there first" perspective on neural nets from Terry Sejnowski: This remarkable tidbit: "LLMs have hundreds of billions of weights, about the same as the number of synapses under a square centimeter of cortex." https://t.co/Z2qtjBZtWe

[Brian Foote @bigballofmud]

Confabulation at the capacity and context margins is a inherent consequence of how neural networks work. RHLPing one to give it a super-ego that strives to somehow eschew uncertainty won't solve this fundamental problem. An NN is neither a database nor an expert system.

[Brian Foote @bigballofmud]

It's every bit as much Captain Kirk as it is Spock...

[Carlos E. Perez @IntuitMachine]

Let's get something straight for people who still can't grok it. GPT-4 is an intuition machine that has an intuition on all kinds of things such as language, programming languages and following instructions. But, like any intuition, it's not always correct!

[Brian Foote @bigballofmud]

I think we've all had our John Henry vs. the Steam Drill moments with these beasts. Your typical thought-leader might be thinking, gee, 1000x as many artificial thoughts to lead. You're thinking, what, give poker another look?

[Kent Beck 🌻 @KentBeck]

I’ve been reluctant to try ChatGPT. Today I got over that reluctance. Now I understand why I was reluctant.

The value of 90% of my skills just dropped to $0. The leverage for the remaining 10% went up 1000x. I need to recalibrate.

[Brian Foote @bigballofmud]

@KentBeck I think we've all had our John Henry vs. the Steam Drill moments with these beasts. Your typical thought-leader might be thinking, gee, 1000x as many artificial thoughts to lead. You're thinking, what, give poker another look?

[Brian Foote @bigballofmud]

@cromwellian @perrymetzger @anderssandberg @robinhanson @regardthefrost @ProbablyTooMuch Ah, undergrad Psychology flashbacks: Tolman's philosopher rats vs. Clark Hull's Big-D(rive)...

[Brian Foote @bigballofmud]

@marypcbuk I asked you-know-who how much paper you'd need for that: The height of a stack of paper containing all the information GPT-3 was trained on would be approximately 2,250,000 meters, or around 1,400 miles.

[Brian Foote @bigballofmud]

If you talk to an LLM like a big box website chat box, it will dumb things down for you. If you up your game rhetorically, you'll steer it towards the loftier upper-middlebrow manifolds of the plankton rich parts of the memepool...

[Yohan J. John @DrYohanJohn]

"What appears to be intelligence in LLMs may in fact be a mirror that reflects the intelligence of the interviewer, a remarkable twist that could be considered a reverse Turing test."

Terrence Sejnowski

https://t.co/IlslOTppMf

[Brian Foote @bigballofmud]

@IntuitMachine In part, the hubris that comes from the certainty that they are smarter than a mere neural network trained on 10,000 times more prose than they could read in a lifetime...