John Taylor

Admin @SwarmIntelligence.ai. Passionate about #AI, #GANs, and #MachineLearning. Solving problems with #SwarmIntelligence.

[John Taylor @SwarmAlgo]

This is the path from the teacher-assisted 0.911346373371 VBVR upper-bound score to a fully autonomous D4RT/JEPA world-model score.

#AI #MachineLearning #ComputerVision #VideoReasoning #WorldModels #JEPA #D4RT #SpatialReasoning #AIEvals #OpenSourceAI

Replace teacher fallback with a shared D4RT/JEPA backend that predicts clean structured plans from first_frame + prompt, renders those plans into videos,
and routes only when validation-calibrated structured loss is low enough.
https://t.co/fPl126Mt4D

[John Taylor @SwarmAlgo]

My first small autonomous VBVR toy result from my D4RT/JEPA world-model stack trained only on the A Very Big Video Reasoning Suite.

Stack: JEPA-style latent world model, D4RT geometry queries, structured symbolic target heads, validation-calibrated routing, VISReg-style collapse checks, Muon/AdamW optimizer ablations, and EvalKit scoring.

No teacher/incumbent fallback in this score. The model routes only validated structured families and falls back to the clean shared backend elsewhere.

Tags:

#AI #WorldModels #ComputerVision #VideoReasoning #JEPA #SelfSupervisedLearning #SpatialAI #MachineLearning #DeepLearning #EvalKit #VBVR #Research #OpenSourceAI

[John Taylor @SwarmAlgo]

Swarm Update: adaptive memory crossed a real refresh gate.

8/8 known-archetype / unknown-path route signatures were copied, re-evaluated from fresh paths, and accepted into exact memory.

0 harmful rows
0.0px mismatch
+8 allow entries

Tech stack: Python + PyTorch, D4RT-inspired route-signature evals, structured-uncertainty guards, exact-memory policy artifacts, fresh sibling checkpoint re-eval, JSON audit reports, pytest gates, and a static HTML dashboard.
Still small-stack, but now measured, not replayed.

#AI #MachineLearning #WorldModels #Memory #PyTorch #Python #AgenticAI #Research #OpenSource

[John Taylor @SwarmAlgo]

This is an early toy VBVR-only video reasoning stack combining several research ideas:

D4RT-style point queries: represent video scenes as sparse 2D/3D/4D object/query particles instead of dense pixels only.
JEPA-style latent prediction: predict future latent states and compare them in representation space, not just pixel space.
Sparse memory retrieval: store/retrieve latent motion evidence from similar examples to improve prediction.
Particle rollout: draft future object states as particles, then accept or fall back based on confidence.
Attractor/Gibbs acceptance: choose between candidate motion routes using energy-style basin scoring instead of a single brittle guess.
Muon optimizer ablations: tested orthogonalized matrix-update optimization, but AdamW is still better in this setup.
Functional Attention ablation: tested basis/function-style attention readouts; useful diagnostically, not yet better than top-k latent retrieval.
VBVR EvalKit controls: measured first-frame, final-frame, morph, hard-cutover, and oracle controls to locate what remains unsolved.

[John Taylor @SwarmAlgo]

First tiny toy test of my VBVR-only video reasoning stack is up.

Current diagnostic score:
Overall 0.8580
ID 0.8722
OOD 0.8439

Not a leaderboard/model claim yet. This uses endpoint controls to probe the EvalKit ceiling.

Next: replace controls with learned D4RT/JEPA dynamics, close spatial/physics gaps, and evaluate real generated videos without held-out final-frame leakage.

#VideoReasoning #WorldModels #ComputerVision #SpatialIntelligence #AIResearch #JEPA #VBVR

[John Taylor @SwarmAlgo]

@GregKamradt And more to come… Still to submit..

[John Taylor @SwarmAlgo]

@hicasamadim 2 times doubled 2 is 8 because it is 2x4, and 5 times doubled 5 is 50 becuase it is 5x10, so 6 times doubled 6 is 72 becuase 6x12

[John Taylor @SwarmAlgo]

I find the difference between people I have worked with comes down to mental models. The stories and experiences that some use to learn from and adapt and the ones that get purged. I found personally it weird that some people read the book and took notes and were extremely book smart but they didn’t know how to apply it to situations. They but in a lot of computer while I rest my head down listening drawing pictures things of ways to use this information with way less compute. And later that day apply that mental model on something in life. Not everyone uses compute the same way and not everyone uses mental models.

[John Taylor @SwarmAlgo]

@slash1sol Yikes, isn’t this from like 2007. I am pretty sure it a lot different from then and even from last year. Especially at larger companies of those scales.

[John Taylor @SwarmAlgo]

@demishassabis Nice

[John Taylor @SwarmAlgo]

@ID_AA_Carmack The audiobook is really good as well.

[John Taylor @SwarmAlgo]

Yes, working on the Kaggle now, but not the paper. I am horrible at writing and communication. And will also open-source the neuromodulation and arc agi as I did everything specific to my personal AMD machine a GMTEC EVO-X2 128GB would be nice to be able to pay off the local AI hardware habits. Removing recon robotic and other non-arc-related clutter and making sure it's all generalized. And making sure it can run on Mac and CUDA after that. Hoping for it to get some generalization past the public and crossing fingers.

[John Taylor @SwarmAlgo]

@arcprize Strict-halo **

[John Taylor @SwarmAlgo]

@arcprize Currently at 183 lvls on the public ARC-AGI-3 demo set games using a non-LLM semi JEPA based system on AMD strict halo cpu/npu.

[John Taylor @SwarmAlgo]

I used a local, no-LLM, JEPA-inspired world-model architecture that combines representation-level prediction, neuromodulated memory, symbolic state graphs, action-effect learning, trap detection, and cross-level rule reuse to solve ARC-style environments.

observe → encode → predict consequences → compare outcome → update memory → choose action

#JEPA #AMD #NPU #AGI #AI

[John Taylor @SwarmAlgo]

All 25 games and 183 levels of the public ARC-AGI dataset were solved locally on my GMKtec EVO-X2 mini PC with 128 GB RAM using Nemo-WM’s compact world-model approach: replay evidence, symbolic state tracking, action-effect memory, trap/mechanism detection, partial-success routing, and reusable cross-level mechanics.

The run used CPU and NPU only, with no LLM required.

#ARCAGI #ARCPrize #AGI #WorldModels #NeuroSymbolicAI #ReasoningAI #Generalization #LocalAI #NoLLM

[John Taylor @SwarmAlgo]

@Clip_Station_ The kids today

[John Taylor @SwarmAlgo]

@Ex0byt This paper supports some cool continual learning transformations I been playing around with on Atari games and arc 3 stuff well beyond llm but memory systems, distillation also.

[John Taylor @SwarmAlgo]

@alexalbert__ Pretty soon there will be 0 Zero days…

[John Taylor @SwarmAlgo]

The headline component gets the citation; the augmentation pipeline, the schedule, the input format, and the inductive bias do the work. ML papers are bad at credit assignment because in general we never run the full ablation.

The credit-assignment problem in ML papers is structural. If you trained with N components and you removed component K and it got worse, you wrote a paper saying “K is critical.” But you didn’t run the 2^N ablation; you ran one. Most “K is critical” claims would dissolve under a complete factorial. We don’t run those because they’re expensive, and the result is a literature where loss functions get credit that belongs to data pipelines

The proprioceptive encoder has a massive architectural prior baked in namely, that temporal self-localization is a function of velocity and angular velocity sequences, which are physics-grounded scalars with known dynamics. V-JEPA 2 has to learn that those signals exist, learn to extract them from pixels, learn that they integrate over time, and learn the integration constants. The proprio encoder gets all of that for free from the input format. Of course it wins. The “win” is mostly the inductive bias.

Same pattern with the ACh window finding (k=2 → 0.925, k=32 → 0.9997, k=128 → 0.9839). The naive read is “we discovered the optimal temporal window.” The honest read is “we discovered that the prior ‘temporal precision is bounded above by ~32 seconds for this task’ is correct, and our architecture happened to expose that hyperparameter cleanly.” A different architecture with the same prior would find roughly the same optimum. The architecture didn’t learn k=32; it exposed k as a knob and gradient descent rolled into the basin.

[John Taylor @SwarmAlgo]

I trained a world model where the JEPA gradient did nothing for 30 epochs and it still works.

the JEPA gradient contributed zero — something else drove the learning

#WorldModels #JEPA #EmbodiedAI #EdgeAI #EfficientML

Not a bug. The free-bits floor clamped L_jepa on every single batch across 1.12 million training steps - zero gradient contribution. Something else drove the learning entirely. The system still achieves AUROC 0.9837 on real outdoor robot navigation, with graceful quasimetric degradation to 0.7847 at k=16.

Physical state encoding confirmed by AIM probe (arXiv:2603.20327) at N=1,752: linear velocity p=10⁻⁵⁵, heading p=10⁻²³, GPS displacement p=10⁻⁵.

Language conditioning without an LLM. CLIP distilled into 46K parameters — 8,700× compression — for real-time text-conditioned navigation at 4Hz with no LLM in the inference loop.

Full pipeline: 4.93ms end-to-end on a $800 consumer machine at 8W. Trained at 45W. No GPU.

If you work on world models, efficient embodied AI, or language grounding at edge scale — I’d like to talk. Paper preprint shortly.