@HansMahncke My brother in Christ your holy Cheeto kept THOUSANDS of top secret docs in the bathroom, and not even the gold plated one. When you can hold your disgrace of a fascist tyrant to the same standard maybe then you’ll have some credibility.
@AvivaKlompas I’m glad that the program is gone, but this is a step too far. There are many innocents suffering in Iran from the regime. Do not mock them or celebrate the destruction of their cities. That’s our enemies way, not ours: בִּנְפֹל אֹיְבֶךָ אַל־תִּשְׂמָח
I always get asked what my favorite **Open Source** eval tool is. The most impressive one thus far to me is Phoenix
Very permissive license as well.
https://t.co/s5FAUkVzn3
I asked here on Twitter who's the best IC product manager you've ever worked with, and @amankhan shot to the top of the list.
Aman Khan is a long-time AI product manager. He's currently Director of Product at @ArizeAI, and was previously at @Spotify, @Cruise, @Zipline, and @Apple.
In our conversation, we discuss:
🔸 What exactly is an “AI product manager”?
🔸 How to break into AI PM
🔸 What separates top 5% AI PMs
🔸 How to thrive as an individual-contributor PM
🔸 Common pitfalls to avoid when building AI products
🔸 The importance of energy and curiosity in product roles
🔸 Much more
Listen now 👇
- YouTube: https://t.co/lgjCTCC7nT
- Spotify: https://t.co/5hlB427xbo
- Apple: https://t.co/NGkuECm8cB
Thank you to our wonderful sponsors for supporting the podcast:
🏆 @Pendoio — The only all-in-one product experience platform for any type of application: https://t.co/ob3ps1KiCk
🏆 @trustVanta — Automate compliance. Simplify security: https://t.co/JHcQhNsceu
🏆 @useParagon — Ship every SaaS integration your customers want: https://t.co/NXHvecJiWk
@metacontrarian@sam_kritch the combination of high horse dismissiveness with obvious disregard for impact or context is some very first draft moral reasoning Sam. Not a good look dude.
@sam_kritch No. The ethics of a child are never using them to make a choice you have investment in, but instead shouting them into an online cesspit, insisting on their obvious validity despite them being devoid of context or impact.
(1/7) o1-Preview Time Series ⌚Eval Challenge 🥊
Time Series anomaly evaluations trip up most LLMs. We decided to put the new @OpenAI O1 models to the test.
We actually use LLMs for time series analysis in our @arizeai Co-pilot so this test is important to us. This skill is incredibly useful for debugging time series patterns, correlations and problems in data.
TLDR:
✅o1-preview did actually perform a lot better
❌o1-preview is just too slow for our product right now
❌o1-mini was horrible on this Eval
If you take a look at the data - here's a set of traces you can check out on @ArizePhoenix:
https://t.co/ZxiE3PcLNi
If this gets 2-3x faster, we would use it immediately for our time series debugging skill. There is a very real future where we swap in and out stronger models for the “tougher” cognitive tasks.
Results
o1-Preview:
85% of anomalies detected - 110k context window
80% of anomalies detected - 80k context window
95% of anomalies detected - 56k context window
100% of anomalies detected - 30k context window
Claude-Sonnet 3.5:
55% of anomalies detected - 110k context window
75% of anomalies detected - 80k context window
85% of anomalies detected - 56k context window
60% of anomalies detected - 30k context window
o1-mini:
20% of anomalies detected - 110k context window
20% of anomalies detected - 80k context window
25% of anomalies detected - 56k context window
45% of anomalies detected - 30k context window
Tagging relevant LLM Evals folks!
@rown@universeinanegg@ybisk@YejinChoinka@allen_ai@haileysch__@lintangsutawika@hendrycks@markchen90@MillionInt@HenriquePonde@Shahules786@karlcobbe@jerryjliu0@mobav0@lukaszkaiser@gdb@HamelHusain@sh_reya@eugeneyan@DimitrisPapail@_akhaliq@JeffDean@demishassabis@jxnlco@OpenAI@AnthropicAI@GregKamradt@MiqJ
It may seem silly, but this is as important as it gets to having a day where the conflict ends. The problem with the tourist discourse on the Israel/Gaza conflict is the unavoidable urge to cast this as good vs evil. It's not, we're all people - we have to remember that.
@TrentTelenko if you actually believe that you're out of your mind.
Trump has clearly been bought and paid for, the holdup in Congress right now is (by your own acknowledgement) being held up by Moscow beholden Republicans. The gymnastics you have to do to blame Biden is wild.
1/3 Resigning, a Hanukkah Message: As of today I have resigned from the antisemitism advisory committee at Harvard. Without rehashing all of the obvious reasons that have been endlessly adumbrated online, and with great respect for the members of the committee, the short explanation is that both events on campus and the painfully inadequate testimony reinforced the idea that I cannot make the sort of difference I had hoped. Still, there are several points worth making. I believe Claudine Gay to be both a kind and thoughtful person. Most of the students here wish only to get an education and a job, not prosecute ideological agendas, and there are many, many honorable, thoughtful and good people at the institution. Harvard is still a repository of extraordinary minds and important research.
So here's a story of our time.
1. In 2016, an Israeli soldier called Elor Azaria executes a wounded Palestinian militant. Point blank. On camera.
2. He becomes a hero to the far-right and the execution of wounded suspects briefly became a wedge issue in Israeli society.
You don’t get it.
It’s not about an UNRWA teacher who held an Israeli kid hostage in his house.
It’s all about how for 75 years you have destroyed the future of generations of Palestinians, including my family.
My cousins in Arab countries are still not citizens - not even the grandkids of the real refugees.
My cousins in Lebanon are not even allowed to work in certain jobs.
And my friends in Gaza and the West Bank continue to be victims of incitement in their own schools.
And it’s all your fault.
Shut UNRWA down.