fab2s

[fab2s @flodl_dev]

A declarative graph DSL for neural networks. Tag streams, build residuals, compose trained subgraphs as frozen Modules in larger architectures. Every level addressable by name.
"The structure IS the text."
Full post: https://t.co/RbWAkaLLPq

[fab2s @flodl_dev]

Phase one was loading. Phase two was fine-tuning, now plumbed end-to-end.

Phase three is the demo on the hardware people actually own.

Next: ModernBERT, LLaMA, LoRA, ViT, plus a flagship El Che fine-tune benchmark on heterogeneous consumer GPUs.

[fab2s @flodl_dev]

Three new families:

ALBERT (factorised embeddings, cross-layer sharing)
XLM-RoBERTa (multilingual SentencePiece, ~250k vocab)
DeBERTa-v2/v3 (disentangled attention, mask-gated embeddings)
MLM heads across all six families. fill_mask one-call for any checkpoint.

[fab2s @flodl_dev]

Universal Trainer for transparent fine-tuning.

You write one closure (forward + loss); the framework owns the loop, backward, optimizer step, gradient sync.

Same code on CPU, single GPU, or heterogeneous multi-GPU. El Che cadence auto-tunes the slow card.

[fab2s @flodl_dev]

Round-trip export is the flagship.

fdl flodl-hf export --hub <repo> --out staged/
fdl flodl-hf verify-export staged/

Re-emits any flodl-hf checkpoint as an HF-canonical dir that loads back into HF Python's AutoModelFor* with bit-exact agreement on every head output.

[fab2s @flodl_dev]

🧵 flodl 0.5.3: HuggingFace, both ways.
30 head cells, bit-exact round-trip with HF Python (max abs diff = 0).
6 BERT-family architectures × 5 head shapes. In Rust.
https://t.co/YsttCzWytF
#rustlang #huggingface #deeplearning

[fab2s @flodl_dev]

flodl is two months old.
Tensor + autograd, full nn parity, declarative graph DSL, transparent multi-GPU on heterogeneous hardware.
The line that's stayed true:
"With flodl I don't rewrite when I pivot. I add or remove a graph member."
Full post: https://t.co/f81f5w7zJp

[fab2s @flodl_dev]

The bet: libtorch FFI through a thin C++ shim.
Not pure Rust. Inherits libtorch's footprint. In exchange:
CUDA parity today. NCCL today. Tensor Cores today. Mixed precision, CUDA Graphs, fused optimizers.
Not in six months. Now.

[fab2s @flodl_dev]

Two false starts.
Go first. The project was called goDL. GC and GPU memory don't compose. You end up with tensors the GC thinks are dead but the GPU is still using, or the inverse.
Rust was the answer. Ownership instead of GC.

[fab2s @flodl_dev]

The shape: FBRL (Feedback Recursive Loops).
Letters read by attention, classified, reproduced. Words compose frozen letters. Lines compose words. Each level frozen as an oracle for the next.
Nested. Partially-frozen. Graph-shaped.
Python did not enjoy this.

[fab2s @flodl_dev]

I'd never trained a deep learning model in my life.
Then I built one in Python, pivoted three times, and watched the script bloat with freezing and recomposition boilerplate.
So I wrote my own framework. In Rust.
🧵

[fab2s @flodl_dev]

Write-up: https://t.co/u8DfN4Yl0B
Tutorial: https://t.co/Y9bTGWB2Dk
Crate: https://t.co/5pmkpY8HEz
Repo: https://t.co/6FoabgCx1F
Feedback and PRs welcome.

[fab2s @flodl_dev]

Foundation for the fine-tuning arc: these three families + AutoModel are the gateway to fine-tuning published checkpoints on heterogeneous consumer GPUs with ElChe.

Next on the roadmap: ModernBERT, LLaMA, LoRA, ViT. Then the fine-tuning loop itself.

[fab2s @flodl_dev]

One command scaffolds a playground inside any flodl project:

fdl add flodl-hf
cd flodl-hf
fdl classify

Drops a side crate pinned to your flodl version with an AutoModel example ready to run. `fdl init my-model --with-hf` for fresh projects.

[fab2s @flodl_dev]

Numerical parity against the HuggingFace Python reference: 9 pinned checkpoints, max_abs_diff ≤ 1e-5 across all three families and three task heads.

Observed on reference: bert-base-uncased pooler 9.835e-7, DistilBERT SeqCls 2.384e-7. Reproducible via `fdl test-live`.

[fab2s @flodl_dev]

One line, any family: AutoModelForSequenceClassification::from_pretrained(repo_id) inspects config.json's model_type and dispatches BERT/RoBERTa/DistilBERT automatically.

Same three-line caller for bert-base-uncased, roberta-base, distilbert-base-uncased, or any fine-tune.

[fab2s @flodl_dev]

BERT in flodl is now `from_pretrained("bert-base-uncased")?` 🧵

flodl 0.5.2 ships flodl-hf: BERT, RoBERTa, DistilBERT + three task heads each (seqcls, NER, QA), AutoModel dispatch, PyTorch parity at max_abs_diff < 1e-5.

[fab2s @flodl_dev]

Full changelog: https://t.co/ItlXC8pDgc

Setup needs Docker + a matching libtorch variant; `fdl setup` walks through GPU detection and image builds.

Rust + DL folks, feedback welcome, especially on `fdl bench` and the DDP surface.

[fab2s @flodl_dev]

`fdl init my-project` now asks: Docker, or native?

Docker gets host-mounted or baked libtorch. Native skips the Dockerfiles, cargo builds on the host.

Three self-consistent scaffolds, one interactive pick. The scaffolded Makefile is gone.

[fab2s @flodl_dev]

Rust binaries using `#[derive(FdlArgs)]` have always exposed `--fdl-schema` so `fdl` help + completion + validation come from one struct.
0.5.1 extends it to scripts. `benchmarks/run.sh` emits the same JSON via a heredoc at the top. `fdl` auto-probes and caches it.