Noeleptes

spaitial architecture programming model | formal verification | agentic coding for Programming Language

[Superman @thesupermannx]

Meta published a paper that might end the current LLM era.

They proved that we can kill the "tokenizer" forever and it actually makes the models stronger.

Every AI you use today (ChatGPT, Claude, Gemini) reads text the exact same way. It chops your words into arbitrary chunks called tokens.

It relies on a massive, bloated vocabulary of around 100,000 specific chunks just to understand a sentence.

We assumed this was the only way to build AI. We treated it as the fundamental speed limit of language models.

Meta proved we were completely wrong.

Their new paper, "Breaking the Token Ceiling," ditches tokens entirely.

Instead, they forced the AI to process raw bytes. A vocabulary of just 256 values.

The results rewrite the rules of model efficiency.

By training on pure bytes and distilling the knowledge, these tiny models didn't just survive. They dominated.

Meta's distilled 1B-parameter byte models asymptotically surpassed state-of-the-art token models like Llama 3.2-1B and Gemma-3-1B across downstream tasks.

They use a fraction of the vocabulary. They require less data. They take up less storage.

It forces the model to actually learn the fundamental relationships between raw data instead of relying on pre-packaged text chunks.

This is a massive shift for anyone building, training, or deploying AI.

We’ve spent the last five years obsessively optimizing how we chop up text to feed these models.

It turns out, the bottleneck wasn't the intelligence of the model.

It was the tokenizer.

[P Equity Research 📰 @pequityresearch]

It is incredible that by 2027, the U.S. CSP capex could be nearly 10x more than that of China.

$GOOGL $META $MSFT $ORCL $AMZN

[Math Cafe @Riazi_Cafe_en]

University of Toronto's "Neural Networks for Machine Learning"
by Geoffrey Hinton

Videos: https://t.co/gr2u0qTsUn
Slides: https://t.co/Hz5jpWabfx

[Jerry Fang @Jerry_fsy]

想深入 Agent Infra，别只盯着 Prompt 和框架。

我落地过 2 个生产级 Agent。在这些项目中，我发现最难的不是模型本身，而是重试、状态管理、沙箱隔离和监控这些工程问题。Agent 能不能在生产环境中稳定运行，很大程度上取决于底层 Infra。

我把Agent Infra的学习之路拆成了 6 步，从 Runtime 一直到生产平台。下面是完整的学习路径。

开始前先说清楚 4 件事。
•这是一份公开的学习计划，大约按 6 个月的节奏走。重点是吃透，不比速度。
•Agent Infra 不是 AI Infra。这里不讲 CUDA，也不讲训练和推理。
•已经熟悉 Runtime 的人，可以跳过第 1 步。
•第 2 步最容易被低估。

6 步依次是 Runtime、可靠性、编排、沙箱、可观测与评估、生产平台。

1. 先自己写一个 Agent Runtime
目标：不依赖现成框架，搞懂 Agent 是如何完成一个用户任务的。

学什么：从零写一个 Agent Loop，也就是"思考、调用工具、拿到结果、再思考"这一圈。然后学 Tool Calling、Context Engineering、Streaming、State 和 MCP。可以跟着三元同学的 Super Agent 实战课走一遍，也可以跟着Datawhale的hello-agents做一遍（hello-agents链接：https://t.co/JI5NhvLCoB）。我这里只分享整体思路，不贴课程里的代码。
有个捷径是读 Pi 的源码。重点看 Loop、消息结构、消息发出的时机、session、缓存、compact、hook 和 MCP。底层的 LLM 封装可以跳过。

怎么练：搭一个最小的 Loop，跑通一个需要多个工具的任务。再对照 Pi 画一张图，标出消息什么时候发出、session 怎么存、什么时候触发 compact。

验收：你能讲清一次工具调用从头到尾发生了什么。只会调 SDK 可不算过关。

2. 可靠性靠设计，不用先学分布式
目标：进程挂了、网络断了，没跑完的任务还能接着跑。
我的结论是，入门不必先学分布式。可靠性是设计出来的。

学什么：HTTP、SSE 和 WebSocket，进程、线程和协程，还有队列、状态机、超时、重试和幂等。资料先看 System Design 101。MIT 6.1800 和 6.5840 挑需要的部分补。DDIA 先别硬啃。

怎么练：写一个能恢复的小型任务执行系统。杀掉进程再重新启动，做了一半的任务要能接着跑。
我特别看重两点。第一，重新启动一个Agent进程，任务也能接着跑。第二，失败要如实上报，不能当成成功吞掉。

验收：重启以后任务不丢失，重试不会造成重复的业务副作用，任务状态能对得上。

3. 从一个 Loop 升级到编排多个 Agent
目标：从管一个 Loop，升级到管多个子任务和多个 Agent。

学什么：Subagent 的任务分发、结果收集和上下文隔离，异步任务与并发控制，目标拆解与结果对齐，以及更高层的多 Agent 编排工具。

怎么练：把一个任务拆成 3 个子任务，跑通后汇总结果。再故意让其中一个子任务失败，看编排层能不能把整个任务收住。

验收：你能说清哪些工作该交给子 Agent，哪些该留在主 Loop。

4. 危险操作一律放进沙箱
目标：工具可以读写文件、执行命令。出了事，坏掉的是沙箱，不是宿主机。

学什么：入门阶段会用 k8s 调度就够了。入门可以先从 Container + Volume 搭建隔离环境，再逐步了解文件系统隔离、资源限制、网络策略和权限控制。

怎么练：给 Agent 配一个独立的沙箱，故意把里面的目录写坏。确认宿主机没有受影响，任务状态也还能恢复。

验收：危险操作默认在沙箱里执行。权限写在 Tool Definition 里，不靠 Prompt 求模型"小心一点"。

5. 先看得见 Agent 在做什么，再让它变好
目标：看得见 Agent 在做什么，并且能持续改进。

学什么：直接看 Langfuse。可观测（Observability）的核心，把轨迹、工具调用、耗时和失败原因都存下来。评估（Evaluation）是一条"先优化、再验证"的流水线，不是偶尔手动测两个用例。

怎么练：接入 tracing。固定 10 到 20 个用例做回归测试。每次改了 Prompt、工具或编排，都用同一套评估看结果变好还是变差。

验收：出了问题，能定位到是哪一步、哪个工具。每次改动都有改动前后的对比。

6. 把前 5 步拼成能上线的平台
目标：把前面几块拼成一个能上线的平台，而不是一堆 Demo 脚本。

学什么：读 FenixAgent的源码。编排、调度和 Agent 协议对接，都是生产里绕不开的脏活。

怎么练：对照 FenixAgent列一份缺口清单，包括调度、协议、多租户和配额，然后按优先级补。别一上来就把整套系统重写。

验收：别人照着文档，就能把你的服务启动起来并跑通。不能只在你自己的电脑上能跑。

[Arash Vahdat @ArashVahdat]

🚀 Are you a PhD student pushing the boundaries of AI, robotics, graphics, or accelerated computing?

NVIDIA's Graduate Fellowship offers up to $60K in funding, mentorship from NVIDIA researchers, and a summer research internship!

⏳ Apply by October 30!

https://t.co/HkNUpQQL0e

[Noeleptes @fx_gx_rx]

@cerebras too expensive the pure SRAM architecture...wish cs5 has the proposed stacked DRAM to solve the memory bottleneck. and release the compiler, i/o runtime to public access so we can develop software for you

[实践哥 Li @MinLiBuilds]

🚨 惊呆了，1521 tok/s，DeepSeek 这速度。

美国这还怎么封？谁他妈受得了啊！

Fireworks 直接把 DeepSeek V4.1 Flash 跑到了 1521 tok/s，11 秒搓出一个俄罗斯方块。

奥特曼跟扎克伯格都要搞抑郁了。

我这边的 OpenAI，在 30 tok/s 挤牙膏。

更搞笑的是，扎克伯格估计看完更难受。

Fireworks 不是普通中转站，是一家估值 175 亿美元的美国 AI 推理公司。

创始团队 7 个人，6 个有 Meta 背景，CEO 林乔以前就是 Meta 的 PyTorch 负责人。

从 Meta 出来创业，反手把 DeepSeek 跑出了这么恐怖的速度。

🚀 梁子前脚开源 DeepInfer，后脚就看到这样的极限性能。虽然不能证明 Fireworks 用了 DeepInfer，但这才是最可怕的地方。

模型开源，推理技术也开源。全球顶级工程师都能参与优化，越卷越快。

快到极致，本身就是一种能力。🫡

[paco xu @xu_paco]

vLLM Q4 Roadmap有三个方向：核心重新设计、模型优化、Agentic 负载，其中核心重构会大量借助 Agent 编码。他把 vLLM 的目标表述为”加速开放智能：跨生态协作，让开放推理对所有人都更快”。https://t.co/mx8ytADT1g From PyTorchCon 2026

[Matthias Niessner @MattNiessner]

Thrilled to announce the @SpAItial_AI PhD Intern program.

Work with a team of exceptional researchers. Build foundation models with lots of GPUs and publish the results at a top-tier conference.

Incredibly happy to strengthen the connection between academia and frontier AI🤝

[Kevin Weil 🇺🇸 @kevinweil]

💥 Introducing Warp Sciences, a new company applying AI and robotics to help the world develop new medicines faster.

We’ve raised a $215M Series A led by @lightspeedvp to bring frontier AI out of the browser and into the lab, and build scientific capacity here in America.

I co-founded Warp with my long-time friend @dpatil, FAIR co-founder @manohar_paluri, and drug hunter Sol Reisberg. The early team is full of incredible people like @bkspears9, who helped achieve Q>1 fusion at LLNL.

[Yichi Zhang @YichiZ03]

I wrote a blog to introduce SGLang HiCache (Hierarchical Cache)

HiCache started with CPU offloading in February 2025. It now preserves reusable history across GPU, CPU, and external storage, including hybrid-model state.

The blog will walk through mechanism under HiCache, and how it achieves 15.5× effective input throughput and ~98% cache hit rate in mass production serving.

Read the visualized deep dive 👇
https://t.co/gCL72ELCa9

[Mrs M @MrsScotty_M]

🔬 Google Student Researcher Program 2027: Paid Research Opportunities in AI, Computer Science & Data Science!

Are you a bachelor's, master's, or PhD student passionate about artificial intelligence, machine learning, data science, or scientific research? Google is inviting eligible students to apply for its 2027 Student Researcher Program, offering opportunities to work on innovative research projects addressing real-world challenges. 🌍💻 Successful candidates may be matched with research teams across Google, including Google Research, DeepMind, and Google Cloud.

🎯 What Are the Benefits?
✅ Gain practical experience in advanced research and technology.
✅ Work on real-world scientific and computing challenges.
✅ Explore areas such as AI, machine learning, computer vision, data science, and software engineering.
✅ Collaborate with researchers, engineers, and scientists.
✅ Develop technical, analytical, and research skills.
✅ Explore opportunities across multiple international locations.

📅 Application Deadline: 13 December 2026
🔗 Read the full opportunity and application guide: https://t.co/aqI6yrAdtM

[SuSu_酥酥👅 @NFT_Chen]

💥不容小觑！英伟达悄悄开源Flare！纯Rust本地推理服务器，专治DGX Spark和Blackwell长对话+Agent！DeepSeek也能直接上！

亮点直接拉满：
🔹运行时纯Rust，零Python开销
🔹内存提前规划 + 默认FP8 KV Cache
🔹CUDA Graph预捕获，启动后极速响应
🔹专为长上下文、工具调用、多轮Agent优化
🔹已内置DeepSeek配方

以DeepSeek V4.1 Flash 1017为例，关键操作步骤：
1️⃣克隆仓库并准备环境（Rust + uv + 依赖）

2️⃣用ignition导出运行时包：
python -m ignition export deepseek-v4.1-flash --gpus 0,1,2,3 --max-concurrency 4 --mtp-draft-tokens 5 --output dsv41.safetensors

3️⃣Rust引擎直接服务：
./flare run dsv41.safetensors --gpus 0,1,2,3 --port 8001

4️⃣（多机）4台Spark TP4部署，用orbit做集群服务，KV池可拉满超长上下文

这波NVIDIA明显在抢本地Agent赛道，和DeepSeek几乎同时出手，有意思。

仓库：https://t.co/01lJ5C5bvK

#NVIDIA #Flare #DeepSeek #V41Flash #本地推理 #DGXSpark #Blackwell #Agent #Rust #开源 #黄仁勋 #梁文峰

[FHILY👑 @Oluwaphilemon1]

Qwen3.8-27B reportedly jumped from 82 to 381 tokens/sec on a single RTX 3090 with 24GB VRAM.

That’s a 4.6× increase without changing the GPU or the model.

The developer, TeksEdge, achieved this by optimizing the inference stack rather than upgrading the hardware.

Here’s how the reported performance improved:

Baseline: 82 tokens/sec
Optimized MTP: 114 tokens/sec
DFlash2 + context lookup: 138 tokens/sec
Optimized lookup verification: 381 tokens/sec

Three techniques explain the improvements.

1. Speculative decoding

Normally, a language model generates tokens sequentially, with each new token depending on the previous ones.

Speculative decoding introduces a faster drafting process that proposes multiple tokens for the main model to verify. When several proposals are accepted, the system can generate more output per verification cycle.

In this experiment, optimized MTP increased throughput from 82 to 114 tokens/sec. Adding DFlash2 and context lookup pushed it to 138 tokens/sec.

2. Context lookup

Why generate text from scratch when the same text already exists in the input?

Context lookup identifies sequences that can be reused and proposes them for verification instead of generating every token individually.

The reported implementation processed blocks of up to 16 tokens, accepting roughly 15 per block.

On a 25,000-token document reproduction task, throughput reportedly climbed from around 260 to 382 tokens/sec.

There’s a catch: this technique works particularly well when the output resembles existing text. Open-ended conversations generally offer fewer opportunities for reuse.

3. Prefix caching

Coding agents frequently resend identical system instructions, repository context, and code snippets across multiple requests.

Prefix caching stores reusable computations from previously processed prompts, reducing the work required when those same prefixes appear again.

In TeksEdge’s reported test, time to first token fell from 22.4 seconds to just 0.56 seconds.

That’s approximately a 40× reduction in initial waiting time.

For agents working with large codebases, this can make repeated interactions substantially more responsive.

One important distinction: 381 tokens/sec isn’t ordinary chat performance.

The developer reports around 133 tokens/sec for regular chat. The higher figure comes from a workload that benefits heavily from context reuse and optimized verification.

Those numbers measure different conditions, so they shouldn’t be treated as interchangeable.

The interesting part is that the RTX 3090 didn’t change. The model didn’t change either.

The improvement came from making the inference stack do less redundant work and verify proposed tokens more efficiently.

Local AI performance isn’t determined by GPU specifications alone. The software running on that hardware can make a substantial difference.

Project: TeksEdge’s Qwen3.8-27B RTX 3090 implementation.

[小龙先生 @Dragonhau66]

普通人的第一桶金，到底怎么赚？

这个问题，我思考了很久。

中国有20万人，资产过亿。
他们中的大多数，起点跟我们一样——没资源，没人脉，没本金。

但他们做对了一件事：
资源拼缝。

你不需要“拥有资源”，
你只需要“连接资源”。

信息差，是拼缝的起点，
价格差，是拼缝的利润，
资源差，是拼缝的护城河。

这种赚钱方式的成本最低，风险最小。

这跟做投资和交易，是同一套逻辑，
你真正赚的不是“低买高卖”的钱，
你赚的是认知差、信息差甚至是资源差的钱。

穷则思变，
但变的，不是勤奋，
是你看待资源和利用资源的方式。

视频里，这位老板把“拼缝”的完整方法，讲透了。

看完你就知道，为什么有些人，能白手起家了。

[AYi @AYi_AInotes]

卧槽兄弟们，这个真的牛逼，感觉大模型诞生至今最憋屈的七年失忆史可能要翻篇了！！

甚至我想说个暴论，这大概是今年大模型架构演进里最性感、但也最击中行业要害的一次底层突围。

刚刚开源的 MemT-1.0 第一次在 Transformer 底层焊入了一颗真正的生物级海马体，

让 AI 从聊完即忘的失忆金鱼，真正进化成了能把经验沉淀进血液里的生命体。

靠暴力堆显卡、拿几个 G 的 KV Cache 硬撑长文本的旧石器时代，今天被这个开源原生记忆大模型正式砸穿了。

他们放出的 66 页技术报告和最高 288B 的开源权重直接甩出了一组降维打击的工程账单：

把记忆变成前向计算流的原生状态后，256K 超长历史下的显存开销从 5.06 个 G 暴降到恒定的 40 兆，
直接把 KV 缓存砍掉了整整一百二十倍以上，而且首字延迟更是不管历史拉得多长，始终稳稳压在一百毫秒出头。

最先击穿传统路径的，是它彻底重构了模型的前向计算流。

以前大家为了让 AI 记住上下文，只能硬着头皮堆长文本或者外挂向量库，代价就是显存撑爆且丢失多步推理因果，它在标准网络旁并联了一套原生记忆模块，在输入进入的瞬间，擦除门、写入门和遗忘门就像生物海马体一样自动运转，不再保留所有的死文字，而是把信息实时消化成动态的内部记忆矩阵。

更绝的是它在真实智能体场景下带来的降维打击，团队把历史纠错中沉淀的一千多条代码经验，直接写入内部的原生记忆状态，完全不占用任何系统提示词的 Token 额度，

结果在标准代码测试集上，连续三次均通过的硬核指标直接提升了百分之八，稳定性甚至反超了直接塞明文提示词的做法。

这意味着经验不再是一段需要模型每次重新阅读理解的生硬规则，而是直接内化成了它的本能直觉，真是牛逼!

我觉得这才是通往真正的递归自我改进最底层的物理逻辑啊, 参数只定义了模型的出厂下限，但唯有持久内化的原生记忆，才能为智能打开复利增长的空间。

以前我们买的模型是聊完即焚的一次性计算工具，从今天起模型正在变成越用越懂业务、越用越值钱的自进化资产。

大家在做智能体或者长对话应用时，最头疼的是长上下文带来的昂贵显存和卡顿延迟，还是模型转头就忘业务规范的失忆症？
对我来说，光是把 256K 显存压缩一百二十倍且延迟恒定这一项，就已经能让无数原本算不过账的长程业务直接原地复活了hh~

[Noeleptes @fx_gx_rx]

C/C++ are portable assembly of the CPU ISA where other languages are purpose built middlewares

[灰狐 @huihoo]

AWS 早期持续押注 Rust，如今来看是极具远见的战略决策
AWS 早期不仅在内部重用 Rust，资助 Rust 基金会、还招募了大量 Rust 核心编译器开发者

AWS 押注 Rust
极致控制云资源成本，堵住基础设施漏洞

Google 押注 Go, Kotlin（被 Oracle 索赔需要有种语言替代）
降低大规模高并发开发门槛，实现云原生工业标准化和 Android 语言生态的连续性

Apple 押注 Swift
统一生态开发体验，彻底告别 OC 的陈旧

Microsoft 押注 C#
早期是为了对抗 Java，现在是企业级全栈语言和生态系统
通过 Unity 引擎，C# 垄断了移动游戏和独立游戏开发的半壁江山

Meta 押注 OCaml
构建最强代码静态分析，并衍生出了 ReasonML/ReScript

当然，C/C++ 依然是所有这些科技巨头绝对无法放弃的
它们会向 C/C++ 生态（编译器、标准委员会、核心库）持续注入资金
大家心里跟明镜似的，C/C++ 是无法撼动的核心历史资产
新语言负责开疆拓土，C/C++ 依然是现代算力的物理底线

[Nous Research @NousResearch]

Jensen closed his GTC Berlin 2026 keynote with this Hermes Agent demo as the finale.

He introduced it by saying the way to build an agent is to ask an agent to build it for you, so he installed and set up a customized Hermes in one prompt.

Then he used Hermes, running on @nvidia RTX Spark, to create a simulated training environment for the @huggingface Microduck. Hermes planned an obstacle course with Nemotron and had GPT-6 Astra build it using FreeCAD, Omniverse libraries, Blender, Newton, and Isaac Lab. Hermes trained thousands of simulated ducks on that course until the validated policy was ready to run on the real robot in the physical world.

At Nous, we believe open source is a practical tool as much as a philosophical ideal. It gives users the ability to build powerful, complex workflows chaining numerous local+cloud models, devices, and tools themselves instead of waiting for a company to enable them.

[Chris Choy, Ph.D. @realChrisChoy]

Interested in Internship at NVIDIA? Apply for a 2027 internship at LPR https://t.co/2nYVnR2KGC! 🤖

We’re looking for interns to help shape the future of intelligent systems that interact with the physical world.

Apply: https://t.co/stKrVFetDk

#EmbodiedAI #NVIDIA #Internship

[Noeleptes @fx_gx_rx]

lol...

[vLLM @vllm_project]

High throughput on the grill. Zero latency on the drinks.

vLLM maintainers and friends gathered during the PyTorch Conference for an evening of Brazilian steak, drinks, and great conversation! 🥩🥂

Grateful for this community and everyone who keeps vLLM moving forward. 🚀