my saturday so far:
- used chatgpt to code python + devops
- used chatgpt to help me cook lunch
- used chatgpt + whisper to translate some anime
- used chatgpt to suggest chord modulations for a song
- used chatgpt + perplexity to do some bio research
This is honestly massive news. At least from the vantage of someone in the public, like me, Altman was probably the person who seemed most associated with the AI explosion of 2023.
Not implying whether or not I approve or disapprove. Just saying it feels very momentous.
i loved my time at openai. it was transformative for me personally, and hopefully the world a little bit. most of all i loved working with such talented people.
will have more to say about whatโs next later.
๐ซก
OK everyone's asking me for my take on the OpenAI stuff, so here it is. I have a strong feeling about what's going on, but no internal info so this is just me talking.
The first point to make is that the Dev Day was (IMO) an absolute embarrassment.
You shouldnโt play 20 Questions with an LLM.
Why? Because LLM behavior less resembles a simulation of a person than a superposition of many such simulacra.
In @Nature, Shanahan, McDonell, and Reynolds eloquently illustrate what this means:
So, I tried a test with the new GPTs, to see if the whole "it can generate a website from a single request" thing is accurate. I'd done a similar test a while back - define a VERY simple website using Go & HTML, HTMX with AlpineJS so it doesn't even need a React app or similar, Tailwind for styling and Phaser to see how far it can get with a simple game. Standard layout, content area with a header, sidebar and footer (hamburger on mobile)
Code Interpreter passed with flying colours. Couple of small things, but basically a solid start.
Now? That failed. It generated some templates & routes to load them - didn't actually connect up the main index.html to anything, forgot about styling entirely.
Going through and iterating over it & explicitly breaking out each goal was...ok, although there was stuff that it just wouldn't get.
So....I see these evals claiming it's better, I don't buy it. GPT-4 at the start of the year did this fine, as did code interpreter. It's literally just some static HTML served by a few routes.
What made it powerful was that you didn't need to do extensive prompt engineering or manage each step as much as 3.5. Looks like that's out the window
๐A Complete Guide to Creating and Storing Vector Embeddings for PostgreSQL Data (using Python, @langchain and @TimescaleDB Vector)
This fantastic article by @avthar not only cover basic RAG concepts like chunking, embedding, etc - but also covers advanced topics like how to keep those embeddings up to date over time
Give it a weekend read: https://t.co/Sd1hIjzmAg
๐๐ฒ๐ฎ๐ฟ๐ป ๐ณ๐๐ป๐ฑ๐ฎ๐บ๐ฒ๐ป๐๐ฎ๐น๐ ๐ป๐ผ๐ ๐ณ๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ๐
We, as developers, like to learn new stuff and try it ASAP. That stuff is mostly some new frameworks and tools (such as React, Angular, Spring, Web Forms, ...). Yet, those frameworks usually have ๐ฎ ๐๐ต๐ผ๐ฟ๐ ๐น๐ถ๐ณ๐ฒ, ๐ฎ ๐๐ผ ๐ฑ ๐๐ฒ๐ฎ๐ฟ๐ ๐ฎ๐ ๐๐ต๐ฒ๐ถ๐ฟ ๐ฏ๐ฒ๐๐. Instead of learning frameworks, which are needed to some extent, we should focus more on learning fundamentals.
๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด ๐๐ต๐ฒ ๐ณ๐๐ป๐ฑ๐ฎ๐บ๐ฒ๐ป๐๐ฎ๐น๐ ๐ผ๐ณ ๐๐ผ๐ณ๐๐๐ฎ๐ฟ๐ฒ ๐ฑ๐ฒ๐๐ฒ๐น๐ผ๐ฝ๐บ๐ฒ๐ป๐ allows a developer to understand the underlying principles and concepts common across different frameworks and programming languages. This understanding allows for more flexibility and adaptability when working with new technologies or when faced with problems that may not be easily solved by a specific framework.
Additionally, a strong understanding of the fundamentals can lead to ๐บ๐ผ๐ฟ๐ฒ ๐ฒ๐ณ๐ณ๐ถ๐ฐ๐ถ๐ฒ๐ป๐ ๐ฎ๐ป๐ฑ ๐ฒ๐ณ๐ณ๐ฒ๐ฐ๐๐ถ๐๐ฒ ๐๐๐ฒ ๐ผ๐ณ ๐ณ๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ๐, as the developer can better understand how to customize and extend them to meet specific needs.
Take an example of a web application that allows users to upload and share images, e.g., Ruby on Rails, and its functionalities for image processing. If the number of users increases, we could struggle with performance issues if we know only the framework well. Yet, if we understand the fundamentals of web development, we could try to identify bottlenecks and try different solutions, such as using CDN-s, optimizing image sizes, using different storage solutions, etc.
So, which fundamentals to learn:
๐น ๐๐น๐ด๐ผ๐ฟ๐ถ๐๐ต๐บ๐
๐น ๐๐ฎ๐๐ฎ ๐๐ผ๐๐ฟ๐ฐ๐ฒ๐
๐น ๐๐น๐ฒ๐ฎ๐ป ๐ฐ๐ผ๐ฑ๐ฒ
๐น ๐๐ฒ๐๐ถ๐ด๐ป ๐ฝ๐ฎ๐๐๐ฒ๐ฟ๐ป๐
๐น ๐๐ถ๐๐๐ฟ๐ถ๐ฏ๐๐๐ฒ๐ฑ ๐๐ผ๐บ๐ฝ๐๐๐ถ๐ป๐ด ๐ฃ๐ฎ๐๐๐ฒ๐ฟ๐ป๐
๐น ๐ง๐ฒ๐๐๐ถ๐ป๐ด
๐น ๐ฆ๐๐๐๐ฒ๐บ ๐๐ฒ๐๐ถ๐ด๐ป
Try to learn those fundamentals, buy and read these books:
๐ธ ๐๐น๐ฒ๐ฎ๐ป ๐๐ผ๐ฑ๐ฒ
๐ธ ๐ง๐ต๐ฒ ๐ฃ๐ฟ๐ฎ๐ด๐บ๐ฎ๐๐ถ๐ฐ ๐ฃ๐ฟ๐ผ๐ด๐ฟ๐ฎ๐บ๐บ๐ฒ๐ฟ
๐ธ ๐๐ผ๐ฑ๐ฒ ๐๐ผ๐บ๐ฝ๐น๐ฒ๐๐ฒ
๐ธ ๐๐ฒ๐๐ถ๐ด๐ป ๐ฃ๐ฎ๐๐๐ฒ๐ฟ๐ป๐: ๐๐น๐ฒ๐บ๐ฒ๐ป๐๐ ๐ผ๐ณ ๐ฅ๐ฒ๐๐๐ฎ๐ฏ๐น๐ฒ ๐ข๐ฏ๐ท๐ฒ๐ฐ๐-๐ข๐ฟ๐ถ๐ฒ๐ป๐๐ฒ๐ฑ ๐ฆ๐ผ๐ณ๐๐๐ฎ๐ฟ๐ฒ
๐ธ ๐๐ป๐๐ฟ๐ผ๐ฑ๐๐ฐ๐๐ถ๐ผ๐ป ๐๐ผ ๐๐น๐ด๐ผ๐ฟ๐ถ๐๐ต๐บ๐
#softwareengineering