In Qwen, we find that Chinese tokens can encode more complex concepts than English tokens, so we can use J-lens to find abstract tokens that describe model computations.
Due to this, meta-tokens are often Chinese and causally relevant to the model’s algorithm.
This kind of thing to me is an indication of @leanprover ‘s success. Before, such issues were obscure, restricted to type theorists type. But now they matter. But something had to become popular first.
sitting in a rooftop restaurant in Shibuya, contemplating the time I trekked to @berniemoreno’s Mercedes Benz dealership on the Cleveland automile to convince him to reorient his Cleveland tech campaign “Blockland” to be more AI focused
what a strange timeline