Victoria Johnston

Interested in software development, AI, web3 & language learning 🤖💖
Staff Eng @ stealth startup 🥷
Ex-Cofounder @littleatlas_xyz | Ex-Senior Eng @ Google

[Ben Lang @benln]

Paul Graham one-pager on how to start a startup:

[kache @yacineMTB]

after LLMs have made it 20 times faster to write code, it became 20 times more important to make your devloop extremely, extremely tight

[Andrej Karpathy @karpathy]

Programming is changing so fast... I'm trying VS Code Cursor + Sonnet 3.5 instead of GitHub Copilot again and I think it's now a net win. Just empirically, over the last few days most of my "programming" is now writing English (prompting and then reviewing and editing the generated diffs), and doing a bit of "half-coding" where you write the first chunk of the code you'd like, maybe comment it a bit so the LLM knows what the plan is, and then tab tab tab through completions. Sometimes you get a 100-line diff to your code that nails it, which could have taken 10+ minutes before.

I still don't think I got sufficiently used to all the features. It's a bit like learning to code all over again but I basically can't imagine going back to "unassisted" coding at this point, which was the only possibility just ~3 years ago.

[Ben Lang @benln]

The YC approach to finding startup ideas:

[DEV Community @ThePracticalDev]

AI Log #4: Vectorization & NumPy in Machine Learning

Wrap your head around vectorization and NumPy on your ML journey! 🤖

{ author: @ctrlaltvictoria } #DEVCommunity
https://t.co/EqrD6qBBfJ

[@levelsio @levelsio]

"You're just lucky"

[Victoria Johnston @ctrlaltvictoria]

@akshay_pachaar Great list! thank you 🙏😃

[Paul Graham @paulg]

"I think a life properly lived is just learn, learn, learn all the time."

— Charlie Munger

[Greg Brockman @gdb]

that feeling when you finally understand a piece of previously inscrutable code

[Victoria Johnston @ctrlaltvictoria]

@daboigbae lol what will happen to coding interviews 🤔🤔

[Victoria Johnston @ctrlaltvictoria]

@VisualCap Wow RIP UK 😵‍💫 do you know where they are going?

[Victoria Johnston @ctrlaltvictoria]

@NFT_GOD Interesting research, thanks for sharing 🫡

[Barsee 🐶 @heyBarsee]

The pace of AI is incredible right now. 🤯

Here's another groundbreaking week for AI:

Grok coming to 𝕏
Screenshot to code
Google's new Synth
Google Bard X Youtube
Stability AI new video AI
ChatGPT voice for free users

Here's everything you need to know: 🧵👇

[Alvaro Cintas @dr_cintas]

The future of AI in web development?

This app generates code and images to convert a screenshot into HTML/Tailwind CSS.

I just gave it a screenshot of my X feed, and look what it produced (Notes & link in the comments):

[Andrej Karpathy @karpathy]

New YouTube video: 1hr general-audience introduction to Large Language Models
https://t.co/Bl4WNuNyFJ

Based on a 30min talk I gave recently; It tries to be non-technical intro, covers mental models for LLM inference, training, finetuning, the emerging LLM OS and LLM Security.

[andi (twocents.com) @Nexuist]

interesting post from an OpenAI employee claiming that all large language models reach the same endpoint regardless of training strategy or clever tricks

this is of course what the bitter lesson teaches us but useful to get an up to date confirmation that it still holds true

[Santiago @svpino]

This is the end of online education as we know it.

Starting yesterday, Bard can answer any questions about YouTube videos.

Look at the attached screenshot. I'm asking to summarize a specific conversation in a 5-hour video. It took around 5 seconds for Bard to answer.

What!?

[Elon Musk @elonmusk]

Looks like https://t.co/Ojr8Kvjaem is still available

[Chamath Palihapitiya @chamath]

A Short History of OpenAI

On Friday, OpenAI ousted its co-founder Sam Altman as CEO. While OpenAI cites a lack of consistent candor in Altman’s dealings with the board as the key reason for his removal, there is widespread speculation about other motives behind his termination. These range from disputes concerning the profit vs nonprofit motives of the company to the discovery of artificial general intelligence, a type of AI that can surpass human intelligence for most tasks.

We wanted to look back at the history and corporate structure of OpenAI to understand how we got here. Here’s the story:

Inception and Early Strides (2015-2018)

OpenAI was initially founded in 2015 by Sam Altman, Elon Musk, Ilya Sutskever and Greg Brockman as a non-profit organization with the stated goal to “advance digital intelligence in the way that is most likely to benefit humanity as a whole.” The company assembled a team of the best researchers in the field of AI to pursue the goal of building AGI in a safe way.

The early years of OpenAI were marked with rapid experimentation. The company made significant progress on research in deep learning and reinforcement learning, and released ‘OpenAI Gym’ in 2016, a toolkit for developing and comparing reinforcement learning algorithms.

OpenAI showcased the capabilities of these reinforcement learning algorithms through its ‘OpenAI Five’ project in 2018, which trained five independent AI agents to play a complex multiplayer online battle arena game called ‘Dota 2’. Despite operating independently, these agents learned to work as a cohesive team to coordinate strategies within the game.

A crucial development occurred in June 2018. The company released a paper titled "Improving Language Understanding by Generative Pre-Training", which introduced the foundational architecture for the Generative Pre-trained Transformer model. This later evolved into ChatGPT, the company’s flagship product.

Transition From a Non-Profit (2019)

In 2019, OpenAI transitioned from a non-profit to a “capped-profit” model. According to the company’s blog post, OpenAI wanted to increase its ability to raise capital while still serving its mission, and “no pre-existing legal structure they knew of struck the right balance”. Per the IRS, for-profit entities and not-for-profit entities are fundamentally at odds with each other, so in order to combine the two competing concepts, OpenAI came up with a novel structure which allowed the non-profit to control the direction of a for-profit entity while providing the investors a "capped" upside of 100x. This culminated in a $1Bn investment from Microsoft, marking the beginning of a key strategic relationship, but complicating the company’s organizational structure and incentives.

The non-profit entity, OpenAI Inc., became the sole controlling shareholder of the new for-profit entity OpenAI Global LLC, which answered to the board of the nonprofit and retained a fiduciary responsibility to the company’s nonprofit charter. Crucially, the board was responsible for determining when OpenAI attained artificial general intelligence (AGI), which the company defines as a “highly autonomous system that outperforms humans at most economically valuable work.”

The structure of OpenAI is outlined below:

Becoming ChatGPT (2020-2023)

In 2020, bolstered by new funding, OpenAI unveiled GPT-3, a large language model (LLM) capable of understanding and generating convincing human-like text. This was a watershed moment for OpenAI and the broader AI community. As the company grew, its LLMs continued to become larger and more intelligent.

However, OpenAI's innovation didn't stop with language models. In 2021, the company expanded its horizons by launching Codex, a specialized AI model for programming, and DALL-E, an AI system adept at creating original artwork from text descriptions.

December 2022 marked another major milestone for OpenAI with the release of GPT-3, laying the groundwork for the consumer-focused application ‘Chat-GPT’. Chat-GPT rapidly captured global attention, becoming the fastest app to amass 100 million users within just two months of its launch. Capitalizing on this success, OpenAI introduced a subscription model and unveiled its most sophisticated model yet, GPT-4, ~10x more advanced than its predecessor and capable of analyzing text, images, and voice. Further developer tools and a turbocharged version of GPT-4 were announced at the company’s Developer Day on November 6th, 2023.

Removing Sam Altman (2023)

On Friday, OpenAI announced that it was removing its co-founder Sam Altman as CEO, citing a lack of consistent candor in his communications with the company’s board. According to the company's official statement, the board “no longer has confidence in Altman’s ability to continue leading OpenAI.”

OpenAI’s board of directors has undergone numerous changes since inception. Elon Musk resigned from his board seat in 2018, citing a “potential future conflict of interest” with Tesla’s AI development for driverless cars. Elon later expressed disappointment over the company’s for-profit motivations and dealings with Microsoft. Since Elon’s departure, a number of other board members have left the company, including former congressman Will Hurd who cited a Presidential bid, LinkedIn co-founder Reid Hoffman over an investment conflict, and Neuralink director Shivon Zilis.

The remaining board members who removed Altman are:

- Adam D’Angelo - CEO of Quora
- Tasha McCauley - Co-Founder of Fellow Robotics and adjunct senior management scientist at RAND Corporation
- Ilya Sutskever - Co-Founder and Chief Scientist of OpenAI
- Helen Toner - Director of Strategy and Foundational Research Grants at Georgetown University’s Center for Security and Emerging Technology

Altman’s removal as CEO prompted the resignation of President and Co-Founder Greg Brockman and three of the company’s senior scientists. Reports suggest that this may have been orchestrated by the company’s other Co-Founder and Chief Scientist Ilya Sutskever over concerns that Altman was pushing to commercialize the company too quickly. Sutskever was recruited to OpenAI from Google in 2015 by Elon Musk, who describes him as “the linchpin for OpenAI being successful”.

A tweet from Greg Brockman confirms that Ilya was a key figure in Altman’s removal.

https://t.co/x2KgdNi3VN

Since OpenAI’s for-profit entity was ultimately accountable to the charter of its non-profit parent, its rapid commercialization may have conflicted with the company’s primary goal of developing AGI in a safe way. According to their 2019 IRS filings, OpenAI does not have a written joint venture policy but the company’s structure explicitly prioritizes the purposes of its nonprofit entity over maximizing profits, preventing OpenAI from engaging in activities that would jeopardize the company's non-profit status.

As of this writing at 3pm PST on 11/19/2023, there are reports that the board is negotiating for Sam’s return, though it remains to be seen how OpenAI overcomes the challenges of its competing profit and nonprofit interests.

Conclusion:

While the details of Altman’s removal are still unfolding, it is becoming increasingly clear that OpenAI’s convoluted corporate structure led to conflicting motivations and incentives within the company. There is a key learning here. Whether you are a for-profit or non-profit entity, there are tried and true corporate structures to help you achieve your stated goal. Because just doing that is hard enough as it is. But once you decide what the goal is, you should work as hard as possible to achieve it; you should never compound unnecessary risk into this journey like iterating on corporate structure. While it can make a hero out of lawyers, it is one of these unnecessary risks that’s only blindingly obvious in the rear view mirror.

[Sam Altman @sama]

the mission continues