Yoss

all we ever see of stars are their old photographs

[Chubby♨️ @kimmonismus]

Absolute cinema

[Hiruni Liyanage @the_liyanage]

US data scientists are so cooked right now. they can't import pandas anymore

[Deedy @deedydas]

Stanford just published a comprehensive 456 page report about AI in 2025.

It's a pretty easy document to scroll through. This is one of my favorite charts from it.

If you find it tough to keep up with the rapid pace of AI news, this doc is the resource for you!

[ハンス @okinawa90s]

Buku fiksi seperti Vinland Saga, Bakuman, Blue Period, Kanokari, Solanin, GTO, Dandadan etc mengajariku menjadi manusia yang lebih baik dibanding buku2 motivational high intellectual type shit kayak Atomic Habits, Psychology of Money, Filosofi Teras etc

[BURKOV @burkov]

If it weren’t for DeepSeek, they would still be feeding us incremental improvements on benchmarks by increasing by 1 percentage point one after another for 5 more years. Now they had to reveal all they had and give it for free. DeepSeek saved us 5 years. Thank you, @deepseek_ai.

[Vivek Naskar @vivek_naskar]

This is seriously getting out of hand.

[unusual_whales @unusual_whales]

Today was the worst day in the stock market in 2025.

[Vaibhav (VB) Srivastav @reach_vb]

Summary of the DeepSeek models released today!

DeepSeek-R1-Zero

> Base Model: DeepSeek-V3-Base
> Training Approach: Pure reinforcement learning (RL) without any supervised fine-tuning (SFT) as a preliminary step
> RL Algorithm: Group Relative Policy Optimization (GRPO), which foregoes the critic model and estimates the baseline from group scores
> Reward Modeling: Uses a rule-based reward system with accuracy rewards (for correct responses) and format rewards (for adhering to specified output formats)
> Training Template: A simple template that requires the model to produce a reasoning process followed by the final answer, enclosed in specific tags
> Performance: Achieves significant improvements in reasoning benchmarks, such as increasing the pass@1 score on AIME 2024 from 15.6% to 71.0%
> Challenges: Faces issues like poor readability and language mixing

DeepSeek-R1

> Base Model: DeepSeek-V3-Base
> Training Approach: Incorporates multi-stage training and cold-start data before RL
> Cold Start: Collects thousands of long Chain-of-Thought (CoT) data to fine-tune the model as the initial RL actor
> Reasoning-oriented RL: Applies large-scale RL training to enhance reasoning capabilities, focusing on tasks like coding, mathematics, and logic reasoning
> Rejection Sampling and SFT: Uses rejection sampling to collect SFT data from the RL checkpoint, combined with supervised data from DeepSeek-V3 in various domains
> Secondary RL Stage: Implements a secondary RL stage to align the model with human preferences, improving helpfulness and harmlessness while refining reasoning capabilities

> Performance: Achieves performance comparable to OpenAI-o1-1217 on reasoning tasks and excels in various benchmarks, including MMLU, MATH-500, and Codeforces

Distillation: Smaller Models

> Base Models: Qwen2.5 and Llama series (1.5B, 7B, 8B, 14B, 32B, 70B)
> Training Approach: Direct distillation from DeepSeek-R1 to smaller dense models using 800k curated samples
> Performance: Distilled models, such as R1-Distill-Qwen-7B and R1-Distill-Qwen-32B, outperform non-reasoning models like GPT-4o-0513 and set new records on reasoning benchmark

[trish @TrisH0x2A]

Software Development is 10x easier when you work on the projects you genuinely care about.

[Yoss @crypter70]

🚀🚀🚀

[Elon Musk @elonmusk]

arXiv papers mentioning @Grok in the abstract

[Kevin Aluwi @kaluwi]

Belakangan banyak spekulasi mengenai alasan2 kenapa industri teknologi di Indonesia lesu. Suku bunga AS, kebanyakan bakar duit, kurang investor, dst. Semua itu ga salah, tapi akar dari masalahnya adalah: masyarakat Indonesia daya belinya masih rendah

[Leonie @helloiamleonie]

If you embed an entire document, you'll lose retrieval precision.

If you chunk a document, you'll lose contextual information between chunks.

These are some concerns when you're building long-context RAG applications.

But "Late chunking" may just be the sweet spot in the middle.

Late chunking:
• a new approach introduced by @JinaAI_
• helps preserve context across large documents
• applies chunking later in the embedding process

Read more: https://t.co/THzK6KAckx

[Yoss @crypter70]

Do you remember when you joined X? I do! #MyXAnniversary

[Andrej Karpathy @karpathy]

Seeing as I published my Tokenizer video yesterday, I thought it could be fun to take a deepdive into the Gemma tokenizer.

First, the Gemma technical report [pdf]:
https://t.co/iPVo3iLXQC
says: "We use a subset of the SentencePiece tokenizer (Kudo and Richardson, 2018) of Gemini for com- patibility. It splits digits, does not remove extra whitespace, and relies on byte-level encodings for unknown tokens, following the techniques used for both (Chowdhery et al., 2022) and (Gemini Team, 2023). The vocabulary size is 256k tokens."

The tokenizer.model file is with this code release:
https://t.co/SwcVU2nkkS

I decoded this model protobuf in Python and here is the diff with the Llama 2 tokenizer:
https://t.co/4HoAeYJsZz

Notes:
- vocab size is quite large: 32K -> 256K
- add_dummy_prefix is False. Different from Llama but consistent with GPT. This is a bit more consistent w.r.t. "leave the data alone", as there is no preprocessing step that adds a space to the encoding text.
- the model_prefix is the path of the training dataset, which is amusing to look at: "/cns/mf-d/home/gemini-data-access/tokenizers/final_v1_51GB_run1/bpe_coverage_0_999995_v5/255969". Seems to indicate the tokenizer training corpus was ~51GB (?).
- a lot of user_defined symbols (i.e. special tokens) are present, e.g. "hardcoding" a sequence of up to 31 newlines as tokens, and a large number of other unclear tokens. I tried decoding the octal representations but it's not clear what's happening here. Also a lot of more special tokens for what look like html elements, e.g. <table>, <tr>, <td>, <i>, <b>, etc. Not 100% sure what the unused tokens are for, maybe this is pre-allocated space to make easier future finetunes that try to add more special tokens, as there is no need to resize vocabularies and perform model surgeries (?).

TLDR this is basically the Llama 2 tokenizer, except bigger (32K -> 256K), with a lot more special tokens, and the only functional departure is that add_dummy_prefix is turned off to False. So e.g. tokenizing:

"hello world" becomes:
[17534, 2134]
['hello', '▁world']

which otherwise would have been preprocessed to " hello world" (note leading space) and tokenized as:
[25612, 2134]
['▁hello', '▁world']

cool

[Mikael Dewabrata 📊 @MikaelDewabrata]

Buat yang pernah bekerja, kerja sama, atau menggunakan produk Zenius terutama waktu masih sekolah dulu, mari kita di sini kasih apresiasi.

Sejauh mana Zenius ngebantu dan memberikan kesempatan untuk kita bisa lebih baik.

[☠️ @apriLIARies]

@irwndfrry bentar nak, udah mulai transfer window

[Extra Time Indonesia @idextratime]

Kamus nama-nama julukan "El"

[A THREAD]

1. El Dear God

[Papan (Necromancer Bare Metal) Berjalan @papanberjalan]

buat kalian yg ngerjain project python, please pretty please pake package manager.

paling tidak pake Poetry Python Package Manager

jadi lu ga usah install versi python yg beda beda, ga perlu install virtual environment, ga perlu mikirin dependency clash, enak manage config, dan distribusi package.

kalo masih pake requirement.txt atau bare pip install, please consider using poetry untuk memudah kan hidup lu dan org di sekitar lu 🙇