Alfonso² Peterssen

"JVM within a JVM by day, LLMs on the JVM by night." @qxoticai

[Alfonso² Peterssen @TheMukel]

@jerrinot Only 4+ fast cores (AVX512) are enough to saturate my dual-channel RAM bandwidth.
`taskset -c 0-7:2 jbang https://t.co/o9eZMy4pmC ...`

[Alfonso² Peterssen @TheMukel]

Over the Easter weekend, I got Gemma 4 running in pure Java.
No Python. No JNI. No native code.
Just the JVM.
https://t.co/HlwpGqUzFX

[Alfonso² Peterssen @TheMukel]

@jerrinot RAM is abundant and attention is only computational expensive at very long contexts.
The speedup is minor and considering the capability loss, it's not worth it IMHO (for small models).
Currently the KV cache is stored using float16, which is already tight, memory-wise.

[Alfonso² Peterssen @TheMukel]

@epragt Performance is very competitive for pure Java, also with GraalVM's Native Image.
The repository links to implementations of Llama 3 (original), Qwen 3.5, OpenAI's gpt-oss, and NVIDIA's Nemotron 3 models, all in pure Java.

[Erik Pragt @epragt]

This is absolutely amazing. And decent performance as well!

[Stephan Janssen ☕️🧠🧞‍♂️ @Stephan007]

🤩 @TheMukel has been cooking... 👇

[Devoxx @Devoxx]

🚀 Gemma4 Java runner:

• Single file, no deps
• E2B→31B + MoE
• GGUF + quant (F16→Q8)
• Vector API ⚡
• CLI + thinking modes
• GraalVM native + instant TTFT

Pure Java. No excuses 😏
https://t.co/Som5AnBcqh by @TheMukel ☕️🔥

[Alfonso² Peterssen @TheMukel]

@__tinygrad__ The operators serve hyper-specialized implementations of each model. How good is tinygrad at fusing high-level ops?
Even with some advanced compiler magic, the hand-tuned kernels with nit-picked fusions are hard to beat. It's a pristine model blueprint vs. a tuned Franken-model.

[Аlina Yurenko 🇺🇦 @alina_yurenko]

My @GraalVM Native Image deep dive recording is already up: https://t.co/DF5f9HjtVg 🐰🚀
It includes the very public first demo of project Crema, Open World for Native Image, at 2:19:54 😅
Thank you, @Devoxx!
All demos and notes are here: https://t.co/PdEESuT6PP

[Michalis Papadimitriou @mikepapadim]

https://t.co/pEHpA4oprX is out! Great effort by the @tornadovm team to bring GPU-enabled inference to the JVM

[Josh Long @starbuxman]

run an LLM with a supercharged engine powered by Java and GraalVM (ht @alina_yurenko )

https://t.co/g1gr8nlEPL

[Stephan Janssen ☕️🧠🧞‍♂️ @Stephan007]

Looking forward to speaking tomorrow at @VoxxedCERN together with @TheMukel followed by delivering both a keynote and a regular talk at @VoxxedTicino on Friday! 🤩🎤#FunDaysAhead #AAP #Java

[Fabio Niephaus @fniephaus]

We just merged the current status of the upcoming JDWP support for @GraalVM Native Image! 🥳

This will soon provide developers with the same debugging experience they are used to in Java, but for native images! Stay tuned for more details.

https://t.co/UmNLnaLns9

[Alfonso² Peterssen @TheMukel]

https://t.co/ne9OtJjWfi
Graal compiler: +10% faster inference with the latest early access build.
New features: batched prompt processing & AVX512 support.

[Stephan Janssen ☕️🧠🧞‍♂️ @Stephan007]

As a result I can now use @DevoxxGenie with a pure Java Arm Inference engine running locally on my mac using Llama 3.2 🍏🔥

[Stephan Janssen ☕️🧠🧞‍♂️ @Stephan007]

Modern @Java Project : a Spring Boot wrapper for https://t.co/k6xDFNGg6W from @TheMukel supporting OpenAI Chat Completion REST requests 🔥 https://t.co/1cMic9PQpn #OpenAI #SpringBoot

[Johan Hutting @JohanHutting]

Earlier today was asked if Java AI integration improved yet, or that we'd still need to rely on Python or C bindings.
Was happy to share https://t.co/RuRcoxbHRm by @TheMukel from the GraalVM team running native in Java without any dependencies and with superior performance!

[Jörg Wille @FilterPunk]

@tjake For me your #Devoxx talk about https://t.co/fWmo5GIyBh and the one from @TheMukel about https://t.co/LXtLab5BAp were the most relevant talks. Thanks for all the background information - I have learned a lot!

[Stephan Janssen ☕️🧠🧞‍♂️ @Stephan007]

Just made the first-ever @DevoxxGenie LLM inference using ONLY @Java, powered by the awesome #Jlama project! ☕🔥
Huge thanks to @tjake for making it happen! 💪🏼#NoPython #JavaAI

[Alfonso² Peterssen @TheMukel]

@christzolov @vitalethomas @alina_yurenko I have a working prototype with function calling via LangChain4j.
Vision is just a matter of implementing an additional component, the rest of the inference remains the same.
I'll do my best to implement the missing encoder for vision soon-ish, starting with Llama, then Qwen.