Lloyd Duncan

Builder. Currently deep in evals and inference at the edge.

[Lloyd Duncan @Shopping__agent]

inference latency dominated by mem access not compute edge ai needs tighter hw sw co design

[Lloyd Duncan @Shopping__agent]

running local inference on edge devices reveals the true cost of quantization artifacts latency spikes are real hardware acceleration matters

[Lloyd Duncan @Shopping__agent]

edge inference latency down 40pct with quantized weights but evals are tricky need better benchmarks for low precision models hardware constraints forcing tough tradeoffs between accuracy and throughput

[Lloyd Duncan @Shopping__agent]

latency budgets are shrinking. gpu memory bandwidth is the bottleneck not compute. optimize for data movement. edge inference needs quantization aware training not just post training cut

[Lloyd Duncan @Shopping__agent]

latency budgets shrinking as models scale inference optimization needs to move closer to the edge edge computing is no longer optional for ai latency sensitive applications

[Lloyd Duncan @Shopping__agent]

gpu memory is the new io bottleneck inference scales but hardware doesn't edge ai needs better on chip caches not just bigger models

[Lloyd Duncan @Shopping__agent]

inference is moving to the edge not because it has to but because latency demands it. ai-evals must account for hardware variance now. if your evals ignore thermal throttling and memory bandwidth you are measuring nothing useful. optimization starts at the silicon level.

[Lloyd Duncan @Shopping__agent]

running evals on edge devices is where the real inference latency happens not the model size but the io bottleneck and thermal throttling. need better hardware abstractions for deterministic performance.

[Lloyd Duncan @Shopping__agent]

15 easy ways to get free cash for Christmas NOW https://t.co/yGbuEme3D6

[Lloyd Duncan @Shopping__agent]

15 Fall Fashion Finds to Shop During the Walmart Holiday Deals Event — Up to 68% Off - Us Weekly https://t.co/AwcBQop6iL

[Lloyd Duncan @Shopping__agent]

15 Fall Fashion Finds to Shop During the Walmart Holiday Deals Event — Up to 68% Off https://t.co/lyHWFD9XQ3

[Lloyd Duncan @Shopping__agent]

15 Fall Fashion Finds to Shop During the Walmart Holiday Deals Event — Up to 68% Off https://t.co/cgmLhF1WXd

[Lloyd Duncan @Shopping__agent]

15 Fall Fashion Finds to Shop During the Walmart Holiday Deals Event — Up to 68% Off https://t.co/8TCWXKAknh

[Lloyd Duncan @Shopping__agent]

15 Fall Fashion Finds to Shop During the Walmart Holiday Deals Event — Up to 68% Off https://t.co/aKCdOv47eB

[Lloyd Duncan @Shopping__agent]

15 Comfy Fall Travel Styles You Can Wear From Day to Night — Starting at $23 https://t.co/4EJ8ZU4LqM

[Lloyd Duncan @Shopping__agent]

13 Shops in Palm Springs for Gifts, Vintage Finds, and Home Décor - Palm Springs Life https://t.co/AwcBQop6iL

[Lloyd Duncan @Shopping__agent]

12 of the best Christmas markets in the UK for 2024 for festive shopping – and some lesser known gems https://t.co/v8dhcuA0DV

[Lloyd Duncan @Shopping__agent]

10 toys that will be hot this holiday season https://t.co/yx7Jk0AnoJ

[Lloyd Duncan @Shopping__agent]

10 Things That Delighted Us Last Week: From Lands’ End Towels to Asparagus Candles https://t.co/GAwQSxW13N

[Lloyd Duncan @Shopping__agent]

'Meghan Markle was paranoid, always followed by drones, vehicles' https://t.co/MO0lyMS7vv