@SuperMemoWoz Great! I see a lot of AlterBrain/IR parallels. With regard to any future integration, I wonder which frictions of classic IR are costs AlterBrain should eliminate, and which are load-bearing for the learn drive? How will AlterBrain tell the difference?
@biedalak@SuperMemoWoz@GuillemPalauS You too. We probably got caught in slightly unneeded back and forth here due to crossed wires. But genuinely looking forward to your own upcoming algorithm, and future contributions to algorithm comparisons.
@biedalak@SuperMemoWoz@GuillemPalauS If OpenAI had the current leading model per open evaluations, and Google then published a blog full of posturing about how *actually* our model is "better by far" but with no evidence, obviously their social media would be full of people saying, "ok, let's see the receipts".
@SuperMemoWoz@biedalak@GuillemPalauS I can see why you might think that, having only taken the plunge in social media recently, but no, the opposite is true for a private company. That would be the quickest way for a company to turn the social tide against them. No that it's a worry here, there's so few eyes on this
@biedalak@SuperMemoWoz@GuillemPalauS Great, hope to see it. My point is simply no myths are busted, until they're actually busted.
I was surprised by the 'guns blazing' tone of the article. I prefer to see claims proven with data, and encouraging each other to excellence. Agree with your other post,live and let live
@biedalak@GuillemPalauS@SuperMemoWoz Totally agree, it needs a reaction. I hope you can demonstrate it to be incorrect. But it needs that demonstration/proof to stop it, not just assertion.
@biedalak@SuperMemoWoz@GuillemPalauS Sure, the point is not open source. There's science as method, and science as a body of knowledge. You could be effectively applying scientific, but the point is how can anyone know with nothing verifiable/repeating/auditable/peer reviewed.
@SuperMemoWoz@biedalak@GuillemPalauS Perhaps there's a linguistic misunderstanding here. When I say "what’s needed now" I don't mean today. My point is you need something verifiable to correct the record and fight the meme. Not just "we're the best, trust us".
@SuperMemoWoz@biedalak@GuillemPalauS I’m a SuperMemo user, so I want SM to succeed commercially. But I also want a healthy ecosystem. Open eval is non‑negotiable for credible claims. Extra openness (white‑paper/ref impl) would accelerate the whole field and, IMO, still leave SM far ahead on incremental reading.
@SuperMemoWoz@biedalak@GuillemPalauS You don’t need open‑source to do science. What’s needed now is open evaluation (dataset/aggregates + scoring code + method). Any SRS system: closed API, binary, or OSS, can run against it. There a good reasons to request open‑sourcing the algo, but that's a separate topic.
@SuperMemoWoz@GuillemPalauS This is why I think Jarrett should be commended, he's pushing in that direction and making good faith attempts to compare the algorithms.
@SuperMemoWoz@GuillemPalauS In AI, most frontier models are closed, yet the field compares them via open evaluations (public datasets + public scoring code + documented methods). Models plug in via API, binary, or open code. SR needs its first such eval. That’s step one.
@biedalak@GuillemPalauS@SuperMemoWoz I agree, any statistically valid dataset works. That’s exactly why we need reproducible evidence. I'm not sure if you were referring above to Jarrett’s work "manipulative", but if so, that feels off. He's published code/methods, is adding UM, and has responded to criticism.
@SuperMemoWoz@GuillemPalauS Better for marketing, leaves a bad taste in my mouth in terms of taking SM seriously on the science. The SM articles make strong claims, but on scientific grounds Jarrett's work is more convincing. I say this as a SuperMemo user, so I'd like to see you guys step up to the plate.
@SuperMemoWoz Great. It's certainly niche, but I think there could be some wider interest. I recall others having some curiosity about how to handle fiction in SM from time-to-time.