Jukay Hsu

Running on testnet | founder @joinpursuit | U.S. Army Veteran @227Wolfhounds | Board member @NYCEDC @nycwater

[Jukay Hsu @JukayHsu]

@schlaf Thank you brother! You’re one of those extraordinary humans. Hope you’re great!

[Jukay Hsu @JukayHsu]

@2ctv hi, hi! Been forever. Would love to connect. Will DM you

[Jukay Hsu @JukayHsu]

Great point in juxtaposition. The two pieces of news on the same day. If the labs can’t coordinate on a math problem, that most people don’t care about, and have no existential consequences, how do we expect coordination for any kind of potential p(doom) scenario

[Sholto Douglas @_sholtodouglas]

It is extremely sad that this didn't end up as an example of how the labs could cooperate/coordinate, because the stakes will be so much higher in the future.

[Jukay Hsu @JukayHsu]

Read the full story in TIME's 25th anniversary issue: https://t.co/kEcQpP0EpY

[Jukay Hsu @JukayHsu]

“I was in a Manhattan classroom, three blocks away on 9/11. I remember looking through these two-story plate-glass windows at this billowing black-gray kind of fog coming from one way as we went out the other way into a sunny sky.

Early on in high school, it was imprinted on me that what’s most important is what happens to all of us. And the ability to have agency in that. 9/11 helped cement that. Important critical junctures really can bring out the best in humanity, if we choose to respond.”

Thank you @TIME and @karl_vick for including me in the 9/11 special issue. I have a tremendous sense of gratitude as I reflect back on these past twenty-five years, and what I’ve been able to learn and contribute.

Most of all, I'm grateful for the extraordinary, kind humans I've met along the way.

Last year, I left my life’s work at @joinpursuit because I saw that AI alignment was becoming pressing and potentially more consequential.

I started Bright Futures to build cooperative AI systems for the benefit of humanity.

[Jukay Hsu @JukayHsu]

@snigdhasur ❤️❤️❤️

[Naval @naval]

👇

[Naval @naval]

I’m not an AI doomer, but my firsthand experience dating back to 2020 is that the researchers expressing concerns are sincere.

The closer they are to the research, the more worried they seem to be.

[Mark Bao @markbao]

A lot of people are missing Terence Tao’s point and thinking “mathematicians are upset that AI is better than them.” That’s not what he’s saying, and some people are forgetting that Tao is one of the most AI-pilled mathematicians out there.

His point is that when people work on discovering something, along the way they invent new concepts. Those concepts later become useful far beyond the original goal, and enables further inventions. Finding a solution does matter, but the intermediate idea is often what makes the field richer, because other people can share it and build the next thing from it.

In tech, we can use the analogy of collaborative software. We started with algorithms for merging changes in a Word document, and evolved that to concepts about versions, diffs, and merges, and later to real-time collaboration tools like Git, Google Docs, and Figma. Humans built upon these concepts and developed more powerful solutions.

Terence’s worry is that a machine automating a solution robs the field of the value of developing the intermediate discoveries in the pursuit of larger discoveries.

When automating a solution, the intermediate discoveries and invention of concepts can be buried or completely hidden in the black box. We don’t learn from them to build the next thing; it’s like we never made the invention of collaborative document editing and thus could not have the conceptual understanding to invent the next version – and since it’s hidden, we also don’t socialize them to allow other people to invent, too, a core tenet of collective discovery.

So then, in both code and math, this leads to the atrophy of development of concepts in the field.

In other words: pure ‘solution extraction’ that hides the process of discovery can leave the field with a checked-off theorem but little new insight or new questions to pursue. And it might prevent us from understanding a field deeper.

I am seeing, first-hand, that atrophying of skills in software development. We push buttons and get solutions. There is much less incentive to develop new concepts and human skill. The bet most software companies are making is that LLMs are so effective in writing code that you’re still shipping overwhelmingly more value even with human skill atrophy, and it’s the right bet IMO.

However, much of the software industry is built upon building things, not necessarily novel invention and research. In such an environment, you can say that you accept some atrophying of conceptual invention and human skill for more output.

On the other hand, sectors like math and pure sciences that are focused on invention and insight might be the hardest hit by this.

Practical/applied sciences might fall somewhere in the middle. An Alzheimer’s cure, room-temperature semiconductor, or highly effective carbon capture solution are far too valuable to sandbag and say only humans can do that to develop concepts in the ‘proper’ way. The outcome matters too much to treat the preservation of concept invention as the highest goal. Even there, though, hidden intermediates can slow the next breakthrough if nobody can see how the first one actually worked.

So the question is not “is AI allowed to solve hard problems?” It is “in this field (math, science, tech, etc.), is the answer itself the main point, or are the concepts and abstractions we use to get there also the thing we need to maintain?”​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

In pure math, there’s an argument that the intermediates are often more useful than the solution, and atrophy in concept development is highly detrimental to the field. Solving Navier–Stokes, contrary to what some people claim, has little practical application, and pure math might be one of those fields where just finding a solution isn’t the entire point, and can actually be contrary to the field, which is what Tao is worried about.

[Jukay Hsu @JukayHsu]

@nchirls ❤️❤️

[Jukay Hsu @JukayHsu]

25. Thinking of you, today and always.

[Joe Pease @joepease]

transfiguration

[Sholto Douglas @_sholtodouglas]

It is extremely sad that this didn't end up as an example of how the labs could cooperate/coordinate, because the stakes will be so much higher in the future.

[Jukay Hsu @JukayHsu]

@mkriak @Consensys Congrats Mike!! Excited for you and the ethereum ecosystem!

[Jacob Coxon @hilbertspaess]

If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” - or take this moment to call for different conditions?

[Jacob Coxon @hilbertspaess]

A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.

[Evan Hubinger @EvanHub]

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

[Jukay Hsu @JukayHsu]

@jalak @NYCEDC thanks Jalak! Been forever. Hope you're doing great. Let me know next time you're in NYC

[Jukay Hsu @JukayHsu]

I’m thrilled to have joined the Board of the NYC Economic Development Corporation (@NYCEDC). EDC is an agency I've long admired, and joining its board is a full-circle moment for me. I’m excited for the opportunity to learn, contribute, and serve, especially as the City thinks through the impact of AI in the upcoming years.

When I moved back to New York after the military, EDC's Cornell Tech initiative inspired me to start Pursuit. EDC went on to become one of Pursuit's first partners, and over the past decade I've had the chance to work alongside extraordinarily talented EDC leaders and staff.

EDC may be the most important agency most people have never heard of. It has been at the forefront of innovation as the architect of Cornell Tech and helped make NY the #2 technology hub in the world. As the City's largest landowner, EDC creates entirely new neighborhoods such as Willets Point, at a physical scale most of us rarely encounter. Having spent over a decade building financing models for public goods like Pursuit's Job Bond, I find EDC's combination of financial tools — equity, debt, tax incentives, grants — particularly compelling. This has unlocked tens of billions in private investment over the past decade, letting public capital move at the scale of private markets and achieve public benefits that private markets cannot do alone. This may seem like a laundry list but this real combination of assets and capabilities, particularly in government, is actually what makes EDC rare and distinctive.

But what excites me most is the moment we're in.

EDC's role in leading the City's AI strategy comes at a time when AI will touch every aspect of our economy and workforce. AI’s impact isn't predetermined. It will be shaped by the choices we make. As the financial capital, media and cultural capital, and the place where more recent graduates move to start their careers than anywhere else, what happens in New York will shape how AI plays out across the country and the world. New York should lead on this, and EDC is uniquely positioned to make sure we do.

I'm grateful for the opportunity to support Deputy Mayor @JulieSuLabor, Interim President Jeanny Pak, and NYC Mayor @ZohranKMamdani 's administration at a moment when the City’s leadership on these issues will reverberate. This includes all of us, and I’d welcome your ideas and suggestions in the months and years ahead.

Thank you to @EleonoraSrugo and @MisterSoo1 for their encouragement and support, without which this wouldn’t be possible. And to the EDC presidents I’ve had the privilege of learning from over the years – @sethpinsky, James Patchett, @MTorresSpringer, Kyle Kimball, Andrew Kimball – thank you for being models of public leadership.

[Alex Atallah @alexatallah]

OpenRouter now works in Claude Cowork!