Taher Haveliwala

Connecting dots • • • Helping startups startup 🚀

Focus on AI, machine learning, and search

[Aadit Sheth @aaditsh]

Jeff Bezos literally dropped the best advice on how to build a trillion dollar company.

[Nikunj Kothari @nikunj]

You can just fix things..

[Jon Erlichman @JonErlichman]

2024 valuations:

Nvidia: $2.2 trillion
Intel: $132 billion

2009 valuations:

Intel: $90 billion
Nvidia: $5 billion

[Taher Haveliwala @taherh]

@ylecun @pmarca

[Taher Haveliwala @taherh]

@KTmBoyle https://t.co/gh6qQm35Qg

[David Perell @david_perell]

I once asked Neil deGrasse Tyson how he speaks so eloquently.

The key is writing: “Hardly any sentence that comes out of my mouth unless I’ve written it down once before.”

[Taher Haveliwala @taherh]

@jeremyphoward Here’s their official explanation:

[Inflection AI @inflectionAI]

Pi’s responses are always generated by our own models, built in-house. On investigation, it appears the user prompted this particular response from Pi after copy-pasting the output from Claude earlier in the conversation with Pi.

Pi differs from other products, like Claude and ChatGPT, in that it remembers the main conversation even in threads. In this case, Pi reproduced the relevant message because it was included in its conversational memory.

[François Chollet @fchollet]

The skills needed for developing new ML techniques have little overlap with the skills needed for applying ML effectively. A bit like how chip design has little overlap with software engineering.

[Yann LeCun @ylecun]

Modems use an adaptive linear classifiers to turn analog signals into bit strings. It's called an "adaptive equalizer"
It is trained in two phases. The first one is supervised, the second one is self-supervised:
1. In the initial phase, when the modem first connects to another modem, the transmitting modem sends a known pseudo-random bit training sequence. The receiving model trains its equalizer in supervised mode to turn the mushy analog signal into clean bits. It's basically a linear binary classifier trained with MSE loss. The random sequence is what produces this horrible screeching noise we hear when the modems first connect.
2. the running phase where the equalizer adaptively "tracks" the changing characteristics of the transmission line using a self-supervised (or clustering) method in which a thresholded binary version of the weight sum is used as the equalizer's desired output. Basically, the equalizer trains itself to make its own output binary. If the properties of the line change slowly, this method can automatically adapt to the changes.
This adaptive equalizer algorithm was invented by Robert Lucky at Bell Labs in 1965. It enabled digital communication over phone lines above 2400 bauds to 9600 bauds.
Bob was the Executive Director of Bell Labs' division "113" in which I was hired in 1988 (he was 3 levels above me). He passed away last year.

Wikipedia: https://t.co/NyuXXyTZMp
Interview of Bob: https://t.co/M2U1haB8i8
Adaptive equalizers: https://t.co/RzblTwMCDX
Original paper: https://t.co/zeb6I8N9N4

[Santiago @svpino]

2 billion people use spreadsheets every month. Their lives are about to change.

Here is ChatGPT integrated natively on spreadsheets. It's one of the best applications of AI that will spread in 2024.

You now have access to:

• Generate content based on other cells
• Summarize, translate, and rewrite
• Analyze reviews at scale
• Reformatting and extracting information
• Enriching existing data

And it's as simple as using ChatGPT as a spreadsheet formula.

This is @NumerousAI. They collaborated with me on this post. Their tool came out in March last year. I think it will explode in the coming months.

Check it out here: https://t.co/kaS0Oqsa5e.

[Mckay Wrigley @mckaywrigley]

This is what AI chat will look like in 1yr.

The days of text-in, text-out are over.

AI apps will handle on-the-fly generation of custom UIs - microapps - that are best suited to help you with your task.

Here’s a preview from Google.

[Mckay Wrigley @mckaywrigley]

I gave ChatGPT a screenshot of a SaaS dashboard and it wrote the code for it.

This is the future.

[Pirate Wires @PirateWires]

Florida Has High Speed Rail and California Never Will

the high speed brightline train now connects miami and orlando, but california still has not laid a single mile of rail. what's the difference between these two projects?

by @river_is_nice

In 2008, California voters approved a plan to create a high-speed rail line connecting San Francisco to Los Angeles. They were told the project would be complete by 2020 and cost them $33 billion. Now, three years past the original deadline, not a single mileof track has been laid. The California High-Speed Rail Authority (CHSRA), which oversees the project, no longer even bothers to speculate when San Francisco and LA will be connected by rail, instead telling Californians that they hope to connect Merced, Fresno and Bakersfield by 2030 to 2033, for $128 billion dollars. This gave the impression that America was incapable of building high-speed rail. But Friday, in Florida, Brightline’s high-speed rail service began between Miami and Orlando. At 125 mph, it’s the fastest train in the country outside of the northeast. It’s starting to look like America’s high speed rail problems might have just been California's after all.

That’s not to say the Florida project was flawless. It’s completion is a relative success, rather than a concrete one. When Florida East Coast Industries, Brightline’s parent company, announced their plans to run passenger rail from Miami to Orlando in 2012, they said the project would be finished by 2014 and cost $1 billion. This estimate proved overly optimistic. The project was completed 9 years and $4 billion dollars later after facing numerous delays caused by bureaucratic red tape and lawsuits from two Florida counties in its path (Martin and Indian River), among other issues. The Transcontinental Railroad, which connected San Francisco and New York, began in 1863 and only took 5 years to complete — despite the ongoing civil war. So it is, frankly, a bit ridiculous that with all of our modern wealth and technology it should take more than twice the amount of time to lay the mere 240 miles of track connecting Miami and Orlando.

That said, it's worth repeating that California hasn’t laid a single mile of track in 15 years, and that somehow these zero miles of track have cost $9.8 billion dollars — nearly double the $5 billion Brightline spent to actually complete their project. What exactly this money was spent on is unclear. CHSRA’s website offers few clear answers, although it does assure voters that the money was wasted locally, noting that 97% of the $9.8 billion “invested” went to California firms and workers. Earlier this year, the agency released a document called “The Economic Impact of California High-Speed Rail,” which boasts that “more than 1,000 private sector firms have been contracted to work on the system.” You’d think at least one of them would have laid a little bit of track, if only by accident, but again, not a single mile.

It is difficult to think of an analogue for this level of government waste, incompetence and inefficiency. Soviet bureaucracy is the knee-jerk comparison, but it's deeply unfair to the Soviets. In the Stalinist era, for example, an apparatchik might arbitrarily order that 2 million porcelain Karl Marx figurines should be produced. (It could be any product or any number, this is a hypothetical example meant to demonstrate how the Soviet economic system worked). A factory would receive the order, and pay their workers by the USSR’s inefficient and ideologically motivated piece-rate system, which paid workers for each individual task they completed on the job. (Each individual worker also had a quota, and was paid a basic rate for meeting it, plus extra for any tasks completed above the quota.) This process could produce up to 18 pounds of paperwork per worker, per month, and the administrative cost of calculating one worker’s pay could cost the state up to a fifth of his monthly wage. Even so, the economic incentives of the piece-meal system on the part of workers and fear of Stalin on the part of managers assured that the quotas were almost always met — and often exceeded. Thus, at the end of this ridiculous process you would still have 2 million porcelain Marx figurines, maybe even 2.5 million, delivered on schedule. Purely by virtue of its ability to actually create things that existed in the physical world, Soviet bureaucracy was more efficient than what the California High-Speed Rail Authority has produced over the last 15 years — again, literally nothing — with a sum of money larger than the GDP of 33 sovereign countries. The absurdity defies comparison.

What can explain Brightline’s (relative) success versus CHSRA’s failure? Libertarian types might point to Brightline as an example of the free market succeeding where the government has failed. Unlike government bureaucrats, Brightline executives had a vested financial interest in making the Miami-Orlando line a reality. Additionally, after the company issued bonds to fund the project, this financial interest also became a legal, fiduciary responsibility. Theoretically, the state of California also has a legal obligation to build the high-speed system that voters approved via referendum. But as the last 15-years have proven, there’s no mechanism to force them to actually build it, much less finish it. So it could be that CHSRA’s insulation from the market forces that led to Brightline’s completion, are the reason for its failure. However, accepting this theory as the only explanation for CHSHRA’s failure tacitly suggests that government is incapable of finishing — or in this case,  even starting — construction on any significant infrastructure project, which is observably untrue. Although I’m sure some hardline libertarians would speculate every public work, from the Hoover Dam to the Interstate Highway System, would have been completed better, faster, and cheaper as private projects, such claims constitute speculation for a reason. Government did build these things. So the question remains: why did Brightline succeed where CHSRA failed?

In Brightline’s case, significant delays were caused by local governments and activists — specifically in Indian River and Martin counties — who were incredibly hostile to the project, seeing it as a potentially dangerous intrusion into their community that would bring little economic benefit. Brightline dealt with the local government and activists, fighting them in court and negotiating safety guarantees. Although I’m sure their PR person would say otherwise, the company’s position was always fundamentally oppositional, and they would have never acquiesced to any major demand (like abandoning the project or rerouting the track) unless ordered to do so in court.

California’s approach has been precisely the opposite. The state has continuously caved to local gripes and demands, producing ever-shifting plans as to where production would start and where it would go. By the end of it all, the train that was supposed to connect LA and San Francisco wouldn’t start in either place, but rather in west-central California, because it would be easier and cheaper to build on the cheap farmland there. Only then, they decided that they didn’t want to run the train through farmland, as this would “encourage new sprawl,” so they decided to start the project in the urban area of east-central California, as this would ensure that the line “served existing cities.” As it currently stands, this means that a regional line connecting the central California cities of Merced, Fresno and Bakersfield is supposed to be completed by 2030 to 2033. From there, at some unspecified date, the line is then supposed to expand north and south in an extremely complicated fashion, weaving through the state with deep regard for political considerations and little for cost or efficiency. Notably, one of the earliest detours approved involved rerouting the train across a mountain and into the Mojave desert. According to the New York Times, “the route's most salient advantage appeared to be that it ran through the district of a powerful Los Angeles County Supervisor.”

Early in the project, advisors from other countries came to California, hoping to win contracts to help devise the system. Among them was the SNCF, France’s state-run rail line. Their proposal — a direct line between San Francisco and LA — was quickly rejected. By 2011, the French advisors left, telling the state of California that they were “leaving for North Africa, which was less politically dysfunctional,” according to an official interviewed by the Times. Seven years later they completed a bullet train in Morocco. The lack of financial incentives inherent to the free market isn’t stopping California from laying rail. Being a democratic government that has to contend with a diverse range of politically organized constituencies isn’t the problem either — the French called California “politically dysfunctional,” and they live in a country where virtually every proposed law is met with a Molotov cocktail or a general strike.

Fundamentally, both the corporate suits at Brightline and the bureaucrats running France’s nationalized train service understand the same thing: you can’t please everyone. American Capitalism can build you a high-speed rail system. French socialism can build you a high-speed rail system. But Californiaism — the politics of endless deference and limitless inclusion — cannot build you anything. It is incapable of producing anything apart from mass homelessness, open-air drug markets, a housing crisis, sanctuary cities overrun with illegals, politically sanctioned race riots, mass looting, and yes, a $9.8 billion “investment” in rail that produces no rail. That is Californiaism, sailing aimlessly in a gulf of money, throwing its trash off the deck and imagining it built the sea instead of the ship.

You could also call that decadence. It’s all the same thing.

— River Page

━━━━━━━━
References for this post are on our website.

Images: Brightline train / Ft. Lauderdale Brightline station

[Joe Lonsdale @JTLonsdale]

Better idea: park cars around town with valuables in plain site and hidden cameras / cops nearby.

Arrest criminals who break in, give them multi-year jail sentences, and shame them publicly. Do this to 50 criminals and see how quickly the break-ins start to drop!

@DeanPreston

[Taher Haveliwala @taherh]

@Austen @typesfast https://t.co/CuqW8eTkKN

[Brooke Jenkins 謝安宜 @BrookeJenkinsSF]

Headlines suggesting that charges were dropped are misleading & suggest that the case has been dismissed, which could not be farther from the truth.

[Product Hunt 😸 @ProductHunt]

Don't have all day to watch Sam Altman's appearance before the Senate?

Check out this speed recap by @mreflow

[Taher Haveliwala @taherh]

@paulg @ylecun A literal read of the second paragraph would be that it (may be) sufficient, without implying that it is necessary.

[Michael Shellenberger @shellenberger]

Victory!

Gov. @GavinNewsom finally calls out the National Guard to shut down the open-air drug dealing in San Francisco!

LFG!

https://t.co/IuKeGNUqME

[James Campbell @jam3scampbell]

Larry Page in the year 2000:

“Artificial intelligence would be the ultimate version of Google…It would understand everything on the web, it would understand exactly what you want—and it would give you the right thing”

Wild that we’re now getting close to LLM’s replacing Google

[Taher Haveliwala @taherh]

@binarybits https://t.co/TULiYALuOf

[Dhruv Batra @DhruvBatra_]

Contemporary discussion (hype?) about LLMs and “pausing AGI development” seems oblivious of Moravec’s paradox.

We’ve hypothesized since the 80s — that the hardest problems in AI involve sensorimotor control, not abstract thought or reasoning.

It explains why AI is mastering games, dialogue, and scene generation. But robots are conspicuously missing from this revolution.

Today, we announce 2 major advancements on the embodied road to general intelligence: https://t.co/9JJyCuMBTN

[Taher Haveliwala @taherh]

@sdrapeau @ylecun Theory ≠ theorem