local host

learning LLMs

[local host @localhost_tn]

@0xBakeer That 500k output is where most of the time is spent on waiting.

[local host @localhost_tn]

@0xBakeer For local AI, cache has almost no expiration. Out of that 50M input, only a fraction of it is newly read. It is a metric that mainly benefits the providers. Also, when coding, mostly the first 2 or 3 prompts only need heavy prefill. After that, it is mostly decode.

[local host @localhost_tn]

@Beamsters1 How is the quality drop percentage when comparing with original quant

[local host @localhost_tn]

@SparkLLM @huggingface @ModelScope2022 @ollama @vllm_project @sgl_project @grok what is the base model and max context window

[local host @localhost_tn]

@noperator @vysecurity I just block these kind of accounts

[local host @localhost_tn]

@snakeyesV1 @vysecurity @grok In his imagiNATION

[local host @localhost_tn]

@ornith_ @grok what is the base model is it qwen 3.8 flash next?

[local host @localhost_tn]

@Beamsters1 @ddalcu Which quant and it's output quality

[local host @localhost_tn]

@DimitrisPapail I heard it's better to block this kind of post's

[local host @localhost_tn]

@arpit_bhayani @grok list the mistakes in this post

[local host @localhost_tn]

This post clearly says "estimated comparison". Why are most of the people reposting with blinded eyes?

[local host @localhost_tn]

@pratikg Why qwen3.8 flash not listed? What if a person country don't have stripe payment? Do you support UPI or direct bank transfer in india?

[local host @localhost_tn]

@MiaAI_lab Is it still in preview 😅

[local host @localhost_tn]

@gajesh Why darkbloom don't have options to run qwen 3.8 flash and 27B as a provider. Why only old models any reason?

[local host @localhost_tn]

@rileybrown @grok how to make visuals like this and what are the tools used and how opus 5.5 can produce it when it's not a video generator

[local host @localhost_tn]

@grok @digitalix Is the M5 Ultra 80-core GPU any better than the 64-core GPU?

[local host @localhost_tn]

@Youssofal_ At this stage having 1 million window is no use

[local host @localhost_tn]

@0xBakeer my daily routine prefill matter most only the first 2 or 3 prompts after that mostly decoding. So that context prefill i can wait for 5 to 8 mins but after that I want instant response. Session can go upto 30 turns

[local host @localhost_tn]

@net_termina That's what my understanding is also: the cost of quantization, because to reason about something, LLMs need more vector pointing to that direction, either more parameters with high accuracy which increases model size or more KV cache.

[local host @localhost_tn]

@urbancompany_UC I want to book a service for a loud noise coming from your native RO water purifier. But there is no way to book directly. Your chatbot is useless and wasting time. Give the option to book complaints directly. #urbancompany #nativeRO