Its miracle that models learn anything substantial from pretraining when you think about how much of the loss is actually dominated by non meaningful tokens
Still using GPT-4.5 for almost everything, even easy queries. Can’t put in words why it’s better but I like knowing that the most possible resources are being spent per token to get me the answer. I’ve lived the luxe life and can’t go back
You think I check my RL jobs constantly because I’m a researcher of diligence and dedication but the sad truth is its just another dopamine lottery. The job was stable for the past day and I know it didn’t crash in the past ten minutes, I just want my identity to be validated
My post-AI career interests:
- Raw travel youtuber
- Train in soccer and see if I can get to college level
- Learn to sing
- Doing science experiments on my own body (current one is how to improve nasal breathing flow, which I measure with an instrument)
Been chatting with a few friends in AI and you have no idea how often people admit to having the same intrusive thought of quitting to open a cafe / coffee shop when the ai wars are over 😂
Upon further reflection, I have a few quips about your writing course:
1. Putting "however" at the beginning of the sentence is actually better. Although the flow may sound better with "however" at the end, putting it at the beginning lets the reader know earlier that you're making a contrast, and reducing cognitive load for the reader is more important than sounding better.
2. Animations on slides are perfectly fine.
3. Redundant information in figures is also fine.
4. I would have gotten summa cum laude at Dartmouth if I had just fixed the darn capitalization in my bibliography for the final report for your class!
Overall, I enjoyed the course though, thanks for teaching it :)
In the past weeks I received many questions (from undergrads especially) about AI research, so I'm putting together a "Ask Me Anything" doc. Add any questions to the doc, I'll answer all of them: https://t.co/D1NL9MuVEI
Yes, I'll actually answer them all, because writing answers to questions scales way better than talking to people individually, and you know I love scaling. (Also, I have a few long flights coming up.)
Here are some of the questions that I've written up draft answers to (see the doc). Also open to feedback on my answers!
1. How did your journey in AI begin?
2. What research direction should I work on?
3. Where/how did you learn most of the stuff you needed to conduct effective research? Is it better to spend more time learning, or to jump straight into research if you have interesting ideas?
4. What would you say is the most important trait you need for research?
5. How much does self-learning (e.g. taking these more recent courses that you mention, etc.) play into the whole research process? Do you wish you went to a school that offered more in terms of academics and coursework? If so, do you wish there were more options or that the courses were more rigorous?
6. Is OpenAI offering internships?
7. I’m particularly interested in working with ML in an engineering way rather than a research way (working with pre-existing research rather than doing research on new things), so does OpenAI offer a full-time software engineering role that doesn’t require a graduate degree?
@agihippo Would love to see this plot
Sugary drinks are the worst. You only get to taste it for a mere second and they have so much sugar
Korean bbq and sushi are pretty pareto optimal IMO
Science is broken. I bought a tempur-pedic mattress, wear blue-light glasses, don't eat after 7pm, take melatonin and still can't get a good night's sleep. Meanwhile my girlfriend eats mcdonalds and drinks coffee right before bed and sleeps like a baby
5. Never seen people in the world so passionate about something as soccer. The Argentine kids next to us, from Argentina, went to 3 group stage games and stay in the US. How do so many people afford tickets to games like this? Even as an American $700 even feels like a lot
A week ago I watched Messi play in the Argentina vs Equador Copa America quarterfinal in Houston. Paid good money ($700/ticket) for a decent seat, so wanted to write down some lessons learned
4. I felt that I didn't know enough about soccer to fully appreciate all the nuances I saw in person. E.g., my first time watching pro tennis in person was profound, because I was good enough at tennis to comprehend how hard the things they did were. Should study more next time