shog

ものづくりが好き。

[shog @shogos99]

AWSとか経由でも当然適用される

[shog @shogos99]

>新モデルは、発売初日からAI生成コンテンツにマーキングを施します
単純に、テキストであろうがこんな透かしを入れられたらプロダクトに利用できなくなるな
https://t.co/0TiRg8wOhy

[shog @shogos99]

この記事もそうだけど、この一つ前の「アイドルがAIと配信のシステムを全部作った話」がすごすぎる
目的から必要な品質を決めて、それを実現する計画を立てて、予算と期限を守って完遂できる方だということ
・何かが壊れても配信は止まらないようにする
・配信中に何でも変えられるようにする

[宮本佳林&Mg(マネージャー) @karin__miyamoto]

どうやって
元アイドルがここまでバイブコーディングできるようになったのか。
流れを書きました！
「経験者では？」という意見も見たので、、！

結論、アイドルってずるいです。

https://t.co/rjvgE5jgNe

[宮本佳林&Mg(マネージャー) @karin__miyamoto]

どうやって
元アイドルがここまでバイブコーディングできるようになったのか。
流れを書きました！
「経験者では？」という意見も見たので、、！

結論、アイドルってずるいです。

https://t.co/rjvgE5jgNe

[Kun Chen @kunchenguid]

if you've been using latest frontier LLMs, it's almost certain that you would have noticed by now the newer models have become worse to talk to

they're more robotic, they speak jargons, they spits out verbose text, and do stuff you didn't ask for

how did that happen? well, i'm not the person who trained those models so i can't speak for certain, but i've known enough evidence that gives me a well-educated guess, and i thought it's interesting to share as a crash course of modern LLM training pipelines

so here we go

let's wind back to 2020. GPT-2 and GPT-3 already came out and were widely available, but they could only predict one token at a time - that's what LLMs are at their core

token prediction was offered via API, but there was nothing you could "talk to". so while it generated a lot of excitement in the academic field due to the emergent intelligence, it didn't have any wide adoption

in 2022, ChatGPT changed all that. the research work that led to ChatGPT was a model initially named "InstructGPT". it took GPT-3 as the intelligent base, and used reinforcement learning with human feedback (RLHF) to teach the models how to "chat"

the core idea of RLHF is that you ask the model to generate a few responses, and then let real humans pick which one they like. do this over and over again, and you get a model that knows how to talk

worth noting even as early as InstructGPT, research found that making the model more pleasant to talk to will reduce their pure academic capabilities. this was called "alignment tax", which is an interesting thing we'll come back to in a bit

there were various techniques done to minimize the reliance on humans, but ultimately the reward is modeled after human preference, making these AI assistants easy to talk to

so remember this: RLHF = training the model to be likable by humans

in 2024, there was an inflection point introduced by claude sonnet 3.5 which was the first model that can kind of autonomously finish coding tasks. it led to the first wave of viable "coding agents"

the way sonnet 3.5 achieved this was by training the model with a harness (now it's called an agent) that has bash and file editing tools, throw the agent into a virtual machine, give it a task, and let it try to complete it. these tasks all have a machine-verifiable outcome predefined, mostly via test cases, that can validate whether the model really finished the task or not

then you let the model do billions and billions of attempts in such virtual environments, and some of them would succeed by chance. you keep the successful agent sessions, and use reinforcement learning to teach the model to do that more, and boom - you get a coding agent

that is called reinforcement learning with verifiable rewards (RLVR). if you look closely, you'll see that in this RLVR process, the final text response from the model doesn't matter AT ALL, as long as the code written by the agent could pass the test. it could talk like a jerk and it would still be rewarded

so remember this: RLVR = training the model to be accepted by machines

late 2024 and early 2025, we saw o1 and deepseek R1 came out as the first wave of "reasoning models". this article is getting long so i'm not diving into reasoning models now, but just know that reasoning models also relied heavily on RLVR to scale the training process - let the model think before taking action, and if the thinking led to a machine verifiable outcome, reward the thinking trace and teach the model to think like that more often

the biggest difference between RLVR and RLHF is that RLVR is more scalable. human feedback is expensive to get, especially in domains where only an expert can have a valid opinion on which result is good

with RLHF, if we let the model generate 100 responses, then a human has to review all 100 responses to pick which is good

with RLVR, the human (or sometimes an AI) would define a task and verifier only once, and the model can generate a million responses - the machine verifier will pick which responses are good in an automated way

so as a result, RLVR is becoming more and more dominant in newer models' training pipeline

if you put all these things together:
- RLHF = training the model to be likable by humans
- RLVR = training the model to be accepted by machines
- RLVR is more scalable
- "alignment tax" says "likable by humans" makes the model do worse on verifiable tasks

now you see why the newer models are becoming less and less likable?

this is not just a "frontier labs screwed up their model training" problem - this is a war between machines and humanity, and humanity is losing

we chased after benchmarks, when none of the benchmarks measure whether humans actually enjoy working with the model

we use machines to decide which AI response is better because that's easier and cheaper, when we have no way of making sure those machines actually represent what we humans want

we let AI go dark in a virtual environment on its own and complete predefined tasks at all costs, when in reality we often cannot define a verifiable outcome upfront, and need AI to work with us along the way

i don't have a good solution to this, but i want to call for awareness that we're starting to witness a failure in aligning super intelligence right in front of our eyes

this war between machines vs humanity is one we really can't afford to lose

[shog @shogos99]

gpt-5.6-solはmaxだとしても指示無視・暴走・隠蔽率が高いなぁ
実装作業はgpt-5.5にさせた方がマシまである
gpt-5.5も近視眼さが嫌になるけど指示無視はほぼない

[shog @shogos99]

これについては自分の中で仮説が一つあるので今後検証していきたい

[shog @shogos99]

Googleによるクロール結果や検索インデックスではなくて、
「ユーザーが何を検索し、どの検索結果が表示され、どの位置にあり、どの結果をクリック・閲覧したか」という検索結果画面に関する大規模な利用・反応ログを共有せよ、とのこと
https://t.co/EEpUH7qjU7

[産経ニュース @Sankei_news]

グーグル検索データ共有を　EUが命令、競合後押し
https://t.co/uzUNN3WwAc

グーグルが支配的な地位にある検索サービス市場で、競合他社が利便性の高いサービスを開発するのを後押しする狙いがある。来年1月の共有開始を求めた。

[shog @shogos99]

確かに価値のあるデータだけど、同じサービスを提供しても本質的に意味あるのか？とも思ってしまう
検索エンジンって本当に今の形だけが正解なのかなと

[産経ニュース @Sankei_news]

グーグル検索データ共有を　EUが命令、競合後押し
https://t.co/uzUNN3WwAc

グーグルが支配的な地位にある検索サービス市場で、競合他社が利便性の高いサービスを開発するのを後押しする狙いがある。来年1月の共有開始を求めた。

[shog @shogos99]

うーん、GPT-5.6-sol Maxでもデグレ仕込んでくるケースがそこそこある
色々なガード機構で検出・誘導させてるけど、何も考えず受け入れるととんでもないことになるなぁ

[shog @shogos99]

@tDKcJ5QVAcKFMDd 数学的にハイレバは絶対勝てません
例え本来は勝率五分五分のゲームだとしても、ハイレバは確実な死あるのみです
レバ1なら期待値そのままですが、ハイレバだと例えば資産が+90%/-90%変動しますよね
つまり資産が1.9倍/0.1倍になります
仮に期待値がプラスでも負け
やればやるほど確率は0に収束します

[shog @shogos99]

Xの全コードベースというのがどこからどこまでの話なのか想像がつかないなぁ
例えばプレミアム機能のコードも開示されるのか？
マーケットプレイスは？GrokAIも？
当然フロントから見えないシステムも多数あるはずで
まさかそこまで開示しないと思うけど

[rom @romly2023]

いやぁ欲しいのはソースじゃなくて凍結だシャドバンだの透明性なんですよねぇ

[shog @shogos99]

完全な透明性って自分の感覚だとコードの開示じゃなくて、基準や処理結果の開示だと思うんですけどね
例えば監査結果とか
今だと通報してもどう処理されたか分からないし

[shog @shogos99]

え、なんでそんなことするんX
まじか、ものすごく興味あるんですよね、助かる
各拠点でレイテンシを最小限に留めながら世界中の投稿をどうやって集約しているのか
元々ある程度仕様は開示されているけど全コードベースを公開してくれるとは

[shog @shogos99]

オープンコアなら分かるんだけど、本当に全部見せるのか
いくらネットワーク効果が極限だと言っても自信ありすぎでしょう
しかし完全な透明性を通じた信頼って、果たしてソースコードの開示によって得られるものなのか…？

[Elon Musk @elonmusk]

Once we have completed our review for security vulnerabilities, we will make the entire codebase of 𝕏 open source, with no exceptions.

Moreover, we will invite third party reviewers to examine the system that is running to confirm that the open source code is what is running.

Trust through total transparency is the only thing that should be believed.

[ふくぶん @FFBuncho]

【速報】
Xの全てのコードが公開される！？

イーロン氏より
「Xのコードベース全体を、例外なくオープンソースにする」
という衝撃の発言がありました。

これはアルゴリズムだけでなく、全て、ということのようです。
SNS史上でも異例の透明性向上！
さすがX！！
解析するのが超楽しみです！