David van Lochem

Thought Leader in Human Resources, AI and People Analytics.

[David van Lochem @davidvanlochem]

Abu Dhabi covers hotel stays for stranded visitors amid flight disruptions via- Khaleej Times
https://t.co/9ulNL7c8xH

[David van Lochem @davidvanlochem]

@cribar Thanks bro!

[David van Lochem @davidvanlochem]

https://t.co/v1KTpwv3rY

[David van Lochem @davidvanlochem]

This explains why landing a job in the UAE is not so easy anymore : https://t.co/Loccdnmw8P

[David van Lochem @davidvanlochem]

https://t.co/VIHU75NZf1

[David van Lochem @davidvanlochem]

https://t.co/ac9jyTpLK3

[David van Lochem @davidvanlochem]

Apple launches AI event-planning app Invites for iCloud+ subscribers

https://t.co/AIGowtCyPk

[David van Lochem @davidvanlochem]

Running your own LLM at home….one of the good things of DeepSeek is that we maybe we don’t need such massive power consuming data centers?

[Alex Cheema @alexocheema]

DeepSeek R1 running on 7 M4 Mac Minis and 1 M4 Max MacBook Pro using @exolabs: https://t.co/W4s3rDvGwL

[Alex Cheema @alexocheema]

Market close: $NVDA: -16.91% | $AAPL: +3.21%

Why is DeepSeek great for Apple?

Here's a breakdown of the chips that can run DeepSeek V3 and R1 on the market now:
NVIDIA H100: 80GB @ 3TB/s, $25,000, $312.50 per GB
AMD MI300X: 192GB @ 5.3TB/s, $20,000, $104.17 per GB
Apple M2 Ultra: 192GB @ 800GB/s, $5,000, $26.04(!!) per GB

Apple's M2 Ultra (released in June 2023) is 4x more cost efficient per unit of memory than AMD MI300X and 12x more cost efficient than NVIDIA H100!

Why is this relevant to DeepSeek?
DeepSeek V3/R1 are MoE models with 671B total parameters, but only 37B are active each time a token is generated. We don't know exactly which 37B will be active when we generate a token, so they all need to be ready in high-speed GPU memory.

We can't use normal system RAM because it's too slow to load the 37B active parameters (we'd get <1 tok/sec). On the other hand GPUs have fast memory but GPU memory is expensive. Apple Silicon, however, uses Unified Memory and UltraFusion to fuse dies - a tradeoff that favors a large amount of medium-fast memory at a cheaper cost.

Unified memory shares a single pool of memory between the CPU and GPU rather than having separate memory for each. There's no need to have separate memory and copy data between the CPU and GPU.

UltraFusion is Apple's proprietary interconnect technology for connecting two dies with a super high speed, low latency connection (2.5TB/s). Apple's M2 Ultra is literally two Apple M2 Max dies fused together with UltraFusion. This is what enables Apple to achieve such a high amount of memory (192GB) and memory-bandwidth (800GB/s).

Apple M4 Ultra is rumored to use the same UltraFusion technology to fuse together two M4 Max dies. This would give the M4 Ultra 256GB(!!) of unified memory @ 1146GB/s. Two of these could run DeepSeek V3/R1 (4-bit) at 57 tok/sec.

All of this and Apple has managed to package this in a small form-factor for consumers with great power efficiency and great open-source (uncharacteristic of Apple!) software. MLX (h/t @awnihannun) has made it possible to leverage Apple Silicon for ML workloads and @exolabs has made it possible to cluster together multiple Apple Silicon devices to run large models, demonstrating DeepSeek R1 (671B) running on 7 M4 Mac Minis.

It's unclear who will build the best AI models, but it seems likely that AI will run on American hardware, on Apple Silicon.

[David van Lochem @davidvanlochem]

https://t.co/rHtEzkFhmY

[David van Lochem @davidvanlochem]

@MashreqNeoBiz how do I get in touch with you. A new account is created in the wrong currency. I filed an online complaint got one call back after more than 24 hours that I missed and they don’t call again. The menu for the phone number doesn’t have an option to get connected.

[David van Lochem @davidvanlochem]

Only 23% of workers, measured in terms of dollar wages, could be effectively supplanted. So no panic for humans, but the % will increase to 40% by 2030… https://t.co/h36kNVDFwz

[David van Lochem @davidvanlochem]

Insurers consider raising co-payment share as medical claims soar https://t.co/3BhvJtH4eF

[David van Lochem @davidvanlochem]

Happening now! Diversity &amp; Inclusion panel at #atdme Sonali Attri about how @MastercardMEA develops talent beyond the organizational borders. Also super contribution by Ghenwa Habbal from @FordMiddleEast , Faten Kalloub of @ENGIEgroup, Nina Oletake of @bdp_com and Hannah Haikal.

[David van Lochem @davidvanlochem]

https://t.co/FWY2sdfImO

[David van Lochem @davidvanlochem]

https://t.co/CsG3Y1VUzB

[David van Lochem @davidvanlochem]

Dubai is still going strong! https://t.co/7DpK48sStm

[David van Lochem @davidvanlochem]

https://t.co/MIfnmerm79

[David van Lochem @davidvanlochem]

@GITEX_GLOBAL @GlobDevSlam @MajidAlFuttaim about how to grow Women in Tech. A regional conglomerate takes the lead to add #5000womenintech.

[David van Lochem @davidvanlochem]

@mercer @TheEconomist partnership by up to give an insight about the regional economy, #inflation and #salary impact. The Middle East is the best place to be in 2023! What is your take?