Anthony

[Anthony @_AFLDN]

@Jilles Save yourself the time and just install my IDP: https://t.co/5sJFFl8yCw

[Anthony @_AFLDN]

@zaph0id You should for mlx-vlm a go, I think you’ll see far better performance with that

[Anthony @_AFLDN]

@zaph0id Just tried it and it’s even worse (in terms of memory usage) on LM Studio, given there’s no TurboQuant.

The issue is, without further optimisation, smaller model, a 24GB M4 MacBook isn’t enough memory to run this model specifically for Hermes-Agent, given the 16-32k context size

[Anthony @_AFLDN]

@VadimStrizheus Those running Hermes with Gemma 4 26B, did you require any particular settings on the hermes side?

When I hook it up to ollama (mlx_vlm, or anything else), it’s sending ~17-26k size prompts, taking ~1 minute to pre-fill which is unbearably slow

[Anthony @_AFLDN]

@khnfrhn @VadimStrizheus Do you have any particular settings on hermes-agent?

When I hook it up to ollama (mlx_vlm, or anything else), it’s sending ~17-26k size prompts which’s taking ~1 minute to pre-fill, which is unbearably slow