freedom Koan-Sin Tan

Coder?

[freedom Koan-Sin Tan @koansin]

@anemll FYI, a note on fp8, winograd, and thermal throttling on iPhone 17/18 Pro. https://t.co/IeNndnMpPn

[freedom Koan-Sin Tan @koansin]

ANE INT8 on A20 Pro is about 105 TOPS for INT8. Measured with my little MPSGraph based app, https://t.co/f2EbaZGlkI. Note that these numbers are likely Winograd 1D F(2, 3) "accelerated".

[freedom Koan-Sin Tan @koansin]

@anemll DetectZero is not new. It’s there back from M5/iPhone 16 days. See https://t.co/XLkX6rBytK

[freedom Koan-Sin Tan @koansin]

@anemll Mostly NO. From .hwx and reverse engineering of the ANECompiler, current 1D Winograd implementation doesn’t support fp8. If Winograd is enabled, you should be able to find it in .hwx file.

[freedom Koan-Sin Tan @koansin]

@combin8or AFAICT, MPSGraph's conv2d kernel, which I used for the app now, doesn't support FP8 (yet). Need to find another way (maybe CoreAI pipe).

[freedom Koan-Sin Tan @koansin]

@anemll So far it’s for A20 Pro (h19) and M6 (h18g) only from the ANECompiler in 27.0 RC. I guess although M5 Ultra has 2 ANE units, there is fast enough bus/interconnect between them.

[freedom Koan-Sin Tan @koansin]

@anemll Assuming the question is: does the ANECompiler handle cases with > 2 ANE units? Nope, not for the bonded network of the ANECompiler, which is hardcoded to support exactly 2 units in macOS/iOS 27.0 RC. BTW, I have a short writeup on the bonded network at https://t.co/nNilmHs4dT

[freedom Koan-Sin Tan @koansin]

Dug into ANECompiler.framework of iOS/macOS 27 RC and found “bonded networks” on H18g/M6 and H19/A20 Pro. Before, multi-ANE chips like the M-Ultra just split up the work at the driver level across separate devices. (1/2)

[freedom Koan-Sin Tan @koansin]

@atiorh for the multi-ANE part, there is something called Bonded Network in ANECompiler, see https://t.co/saBxe1yEqt

[freedom Koan-Sin Tan @koansin]

Dug into ANECompiler.framework of iOS/macOS 27 RC and found “bonded networks” on H18g/M6 and H19/A20 Pro. Before, multi-ANE chips like the M-Ultra just split up the work at the driver level across separate devices. (1/2)

[freedom Koan-Sin Tan @koansin]

But with bonded networks, the compiler actually splits a single model across two physical ANE units itself. So, I guess there is relatively high-speed interconnect between the two ANE units. The compiler spits out a .hwx file with both bonded and non-bonded versions. (2/2)

[freedom Koan-Sin Tan @koansin]

@anemll fp8 and fp4 are in the MLIR path already. As far as I can rember, they are in ANECompiler, too. I didn't check the CoreML and MIL part. I mean it should be possible to use FP8 with CoreAI or MPSGraph.

[freedom Koan-Sin Tan @koansin]

Finally figured out how to read Apple ANE's PMU counters. In short, you have to change boot-args. I put down some example cdoe at https://t.co/25Fmd7L6uH. Tested on M1 and M4 only.

[freedom Koan-Sin Tan @koansin]

@anemll @originalmaderix with my hwx parser, you can see INT8 is supported https://t.co/6zoG4J8yui

[freedom Koan-Sin Tan @koansin]

@anemll @originalmaderix from my previous reverse engineering, as far as I can recall, MIL doesn't support int8 natively.

[freedom Koan-Sin Tan @koansin]

@anemll @originalmaderix I know why. I measure ANE compute capacities couple months ago. AFAICR, MIL doesn't support INT8. With MPSGraph, which uses MLIR, you can get expected INT8 performance.

https://t.co/L3gNigJdHA

[freedom Koan-Sin Tan @koansin]

@Wunkolo in recently iOS 26 libsytem_platform.dylib, there are __sme_mem{chr,cpy,move,set}, but, unlike MTE ones, it seems they are not used.

[MLCommons @MLCommons]

MLCommons® just launched MLPerf® Mobile on the Google Play Store! 📱
Benchmark your Android device’s AI performance on real-world ML tasks with this free, open-source app.
Try it now: https://t.co/YKN4OtzU1k

[LaurieWired @lauriewired]

Just built an MCP for Ghidra.

Now basically any LLM (Claude, Gemini, local...) can Reverse Engineer malware for you. With the right prompting, it automates a *ton* of tedious tasks.

One-shot markups of entire binaries with just a click.

Open source, on Github now.