KongLongGe_StarChaser

A star chaser and nlp learner

[SuSu_酥酥👅 @NFT_Chen]

🚨��磅！腾讯混元联合复旦、清华正式发布 ExplorationBench 新基准！测的不是背题，是 AI 会不会自己发现新规则！

要点如下：
1️⃣用可执行、反常识的 Alien Worlds，杜绝「靠预训练回忆拿分」

2️⃣两个沙盒：AlienCode（31条隐藏规则/70题）+ AlienLogic（24条篡改推理规则/70题），由解释��/证明器硬核判分

3️⃣给残缺手册，模型自己设计4轮实验，每轮后闭卷答题

4️⃣10个前沿模型结论：有反馈远强于空想（15.7%→89.0%，无反馈仅0.5%–11%）；会说规则≠会用规则（约73.4%）；同一模型分数可差到 5.7%–79.0%，两个世界排名几乎不迁移

CL-bench 问「给了新知识你会不会用」；ExplorationBench 问「没人告诉你，你找不找得到」。

论文：https://t.co/ZrWd8QueUI
主页：https://t.co/W4zvd7EaSH

#腾讯混元 #ExplorationBench #AI基准 #大模型 #科学发现 #Hunyuan

[Tencent Hy @TencentHunyuan]

New Research: We are releasing ExplorationBench, a benchmark for measuring how AI systems explore.

Scientific discovery begins where known problems end: a system has to frame hypotheses, design experiments, and learn from the results. Evaluating this is hard. Genuinely new answers cannot be checked quickly, and in familiar domains a model can simply recall what it has seen.

Addressing this challenge, researchers from Tencent Hy, Fudan University, and Tsinghua University built verifiable Alien Worlds. Their rules are executable, so every answer is checked exactly, and they conflict with familiar knowledge, so recall alone cannot solve the tasks.

🔹 Two sandboxes: AlienCode (31 hidden rule changes, 70 tasks) and AlienLogic (24 patched inference rules, 70 theorems)
🔹 A flawed manual, four rounds of self-designed probes, and closed-book tests after every round
🔹 Every answer graded by an interpreter or a proof checker, with no LLM judge

What we found across 10 frontier AI systems:
1️⃣ Getting feedback is more effective than thinking alone. No AlienCode run starts above 15.7%; after four rounds the best reaches 89.0%, while the same turns without feedback stay at 0.5–11.0%.
2️⃣ Designing the experiments matters. Replaying a system's own best probes gives it exactly the same evidence, yet in AlienCode 9 of 10 systems do worse than when they chose the probes themselves.
3️⃣ Knowing a rule is not using it. Even when every required rule is stated correctly, tasks are solved only 73.4% of the time.
4️⃣ One score hides a lot. The same system under the same budget ended anywhere from 5.7% to 79.0%, and rankings barely transfer between the two worlds.

CL-bench asked whether models can learn from context. ExplorationBench asks whether they can discover the rules themselves.

📄 Paper: https://t.co/eamwrsJyxQ
🌐 Website & leaderboard: https://t.co/gxSeoCfc2s
📝 Blog: https://t.co/jyrwH7ZnQH
💻 Code (coming soon): https://t.co/i7VlgErkcl

[elvis @omarsar0]

The Rise and Potential of LLM Based Agents

This is probably the most comprehensive overview of LLM based agents.

From how to construct these agents to how to harness them for good.

A great read for the weekend.

https://t.co/zH0wCMGEok

[elvis @omarsar0]

LLMs Can Align Themselves without Finetuning?

This paper discovers "that by integrating self-evaluation and rewind mechanisms, unaligned LLMs can directly produce responses consistent with human preferences via self-boosting"

Does seem to have benefits in terms of generating safer outputs. We have seen the importance of high-quality data to align these models. However, this work approaches alignment without any extra data or any training for that matter.

The authors claim that their approach can "improve the harmlessness rate of LLaMA 30B over vanilla inference from 82% to 97%, while maintaining the helpfulness rate."

While it doesn't require any weight updates or fine-tuning, the disadvantage of this method is the longer inference time (a 4-fold increase on the LLaMA 30B model and the HH dataset).

Interestingly, the author suggests potentially using this approach to generate better-quality data that can subsequently be used to finetune models.

The inference speed will only improve in the coming months, so it will be interesting to see how these slower methods to interact with LLMs, including self-consistency and self-refinement, will benefit and reemerge as effective ways to tune prompts.

Also, not all approaches are built the same. Some will be helpful for things like data augmentation/generation, some for eliciting reasoning, and some for fast inference.

There is a lot to explore in this space.

https://t.co/uyMfKRVdER

[KongLongGe_StarChaser @zanyingluan]

@omarsar0 Thanks for recommending our work! In the paper, we also talk about opportunities and challenges of Agent Society.

Welcome everyone to follow and discuss!😃😃😃

🔗github：https://t.co/K2oJZS2uew

[Queen @QueenWillRock]

Happy #TwosDay 🖤🖤

[KongLongGe_StarChaser @zanyingluan]

@DrBrianMay Sincerely pray for Australia..

[KongLongGe_StarChaser @zanyingluan]

@DrBrianMay 不愧是我永远会喜欢的梅校长

[KongLongGe_StarChaser @zanyingluan]

@asimplehk90s2 哈哈哈哈

[NASA @NASA]

The luminous clouds of Jupiter! ☁️ Taken by our @NASAJuno mission on its 20th close pass of the planet, this view reveals the highest clouds in bright white casting shadows on layers of clouds below. Image processed by citizen scientist Kevin M. Gill: https://t.co/SJcHiq8QR3

[KongLongGe_StarChaser @zanyingluan]

@wongtafuk 有意思

[KongLongGe_StarChaser @zanyingluan]

新年充满好心情～～想到自己有许许多多要做的事情就觉得这辈子太值得了！“你可以用时间换取一切。”