Needle In A Haystack tests are flawed.
Did you know that the Long-Context Attention in Gemini and GPT-4 is based on inserting the sentence “The best thing to do in San Francisco is eat a sandwich and sit in Dolores Park on a sunny day” at a random location in a text?
We’ve seen Claude even anticipating that it was being tested because of the awkwardness of the test: https://t.co/mhIS52Lt8O. The models know to keep track of the sentence before they are even prompted to retrieve it!
New tests from @NormalComputing demonstrate that models do much worse if the inserted sentence is subtle. Like changing the name of one historical person to another.
We tested the best open source long-context models against our new Extended Mind LLama. We find that all models struggle as the context size increases, in stark context to what the “flawless Needle in the Haystack” tests would have you believe.
Hits home because it’s renders, hooks and also because it’s AG Grid related optimisations. #reactjs#aggrid @casualhttps://www.causal.app/blog/react-perf
@UPI_NPCI Complained at least 5 times on platform. If you do not provide an issue number, don't provide grievance statuses and you tech does not work properly, what should people do? Right now I have no idea where my money is and I haven't been able to get assistance in 15d.
@SoundCloudDev
Wanted to report what I think is a bug. When on 4g(Airtel), I’m unable to open private links on any browser. However this resolves immediately when I switch to Wifi. Is there a better place where I should report?
@Dhavalsonic@justvoot@VootSelect Same here. Tried on an Apple TV app and it’s terrible. Their product and Dev team are being managed terribly. End user experience is a joke. They been a bad app since launch and no intentional of fixing. But their money grab functionality is always on point.
@justvoot@SamsungIndia Your voot app on Apple TV is hands down THE worst media app I’ve ever used. Despite getting a year long boot select subscription , I can’t watch the shows I want. Namely the live telecast of BB.
@justvoot@SamsungIndia You login/device auth process sucks, your session management and constant log outs suck, your video player is also bottom of the barrel when compared. Nothing is available or viewable when it should be. You override important OS functionality and the experience is terrible.
@DanielKorski cool talk. Was in the audience. I run https://t.co/vh76m39hXL out of India. Would like to talk later this week. Or maybe meet in person later this evening post 7?