Having lived in both nyc and sf, I think that your friend’s observations are true (it’s less of an issue in NYC, imo). I suspect this is due to optimization culture, where tech bros pathologically reverse engineer and “optimize” their lives by reasoning logically about social dynamics.
ANTHROPIC FACES CALL TO DELAY MASSIVE IPO OVER AI SAFETY CONCERNS
Anthropic has opened discussions with NYC Comptroller Mark Levine after he urged the company to postpone its planned IPO until it addresses AI safety and governance risks.
Levine oversees public pension funds holding more than $325 billion in assets and warned that a major AI safety incident after listing could severely damage investors.
Anthropic is reportedly targeting an IPO as early as November, following quarterly revenue of $16.8 billion.
The key takeaway: AI safety concerns are emerging as a potential obstacle to one of the largest IPOs in history.
As written, it’s easy to dunk on this, but there is actually a valid point here.
In Asian cultures, parents often force their children to practice for hobbies and extracurricular that their children do not care about (“tiger parenting”). Whereas in American culture, parents generally encourage their kids to follow their passions.
So the point isn’t that American culture emphasizes laziness and talent, but that it emphasizes self determination and individual agency.
“You don’t study for the spelling bee” is a deep and important aspect of American culture. I was at a baseball game discussing some young players with a buddy one time and he said to me offhandedly, “hard work is fine, but nothing beats talent.”
Americans have this sort of sense that there’s something a little off about working really hard to get good at something that you’re not naturally good at. You’re never really going to be on the same level as the guy who has a natural talent. We have room in our hearts for scrappy little guys who work hard, but we are by default suspicious of grinders and strivers.
If you were really meant to do it, then you’d be naturally good at it. You sign your kids up for little league, but you don’t take it too seriously unless they show remarkable early talent.
Yes, this sounds sort of ridiculous if you take it too literally. No, nobody is saying that there’s any kind of practical test for this or that anybody should be excluded because we found out that they secretly studied too hard.
But this is a part of American culture. And if you can’t understand it, then that just marks you out as foreign to that culture to an even greater degree.
A lot of the worry behind open-source models comes from untrusted actors being able to cyberattack or synthesize drugs. But unequal access to capital and compute already prevents all but the most well-resourced labs and governments from having frontier intelligence.
In light of the latest Anthropic's latest not-particularly-oblique attempt to build a case against open weights, I thought I would write down how I currently think about open weight AI models.
(1) Open weights have been deeply, irreplaceably useful for AI safety.
@TaliaRinger@allTheYud Say more?
- Autonomous wet labs correct a market efficiency
- Markets correct inefficiencies
- Therefore markets will bring about the creation of autonomous wet labs
@TaliaRinger@allTheYud You don't need to believe that markets are efficient. You just need to believe that capitalism incentives markets to be efficient.
Your argument is circular:
“It is all rationalists who are bringing these prophecies to life” is sneaking in the idea that they wouldn’t have happened without rationalists.
I recommend you consider the counterfactual in which Yud, or Anthropic, didn’t write about or build an integrated wet lab. Do you think a startup could make money by doing it? (@Ginkgo may be interesting for you to study). If yes, do you think that somebody would have capitalized on it?
@TaliaRinger serious question, in good faith: if you believe that it is dangerous to hook up AI's to autonomous wet labs, what better strategy would you propose?
@TaliaRinger it is the exact same reason that the company that is worried about ASI, is building ASI. these are the same thing.
did you think that they wanted to pace the frontier against chatbots?
since a16z, Nvidia, Palantir, etc are lobbying hard against any AI regulation, it is not effective for anthropic to make what you call a "real response". so they work incrementally, getting political buy-in at each stage. dario was pretty explicit about this in https://t.co/j45Y2rAAjD
what doesn't make sense to you? how would you do it differently if you actually believed it, and why would it be more effective than dario's current strategy?