sucre

Software Engineer | Blockchain | ⚡️| upcoming quant Zaddy

something @growthrail

[sucre @hamzzygeek]

future vanity plate number - Hoxedf

[Mayowa | Haildesk @craneoh4]

Built a tool so Windows devs can ship iOS apps without a Mac.
Works with @expo , @flutterdev and @reactnative projects.

https://t.co/FdXADuERnd

[Ryan Lopopolo @_lopopolo]

I distilled myself and my work to twelve theses of harness engineering. This represents the last year of my work. This is the practice and technique. Go nuts fam. https://t.co/7ZqSMIFN9H

[Kevin Simback 🍷 @KSimback]

This is a good thread, but we need to correct the semantics - ontology, knowledge bases, and company brains are NOT the same thing

They are closely related and often bleed into each other in practice so I understand why they can get confused

> ontology is a formal semantic model of the concepts, entities, attributes, relationships, and rules in a domain

It doesn't contain the actual data, think of it as a rule book for what kinds of things exist here and how they are allowed to connect

An ontology without data or a knowledge base is useless, so they can't be the same thing

> a knowledge base is the actual content repository

It can be structured (e.g., a knowledge graph built on top of an ontology), semi-structured, or just raw unstructured documents + embeddings

A KB can be useful on it's own because you can search it or run retrieval-based methods on it to get answers, but without an ontology you might struggle to understand how things in the KB relate or follow company rules

> a company brain is the actual system on top that people and agents interact with

It's usually an LLM + retrieval layer (often combined with orchestration, memory, tools, etc.) that makes the KB useful for answering questions, surfacing context, enforcing processes, etc.

In the Cerebras write-up, what they built looks like a very well-engineered retrieval-focused knowledge base, and hats off to them for sharing it!

It's an excellent practical 'company brain' implementation, but it doesn't appear to rely on a formal ontology, so again these are not all the same thing

That's not a criticism, just pointing out the semantics

I think the best modern systems will increasingly be hybrids - they have an ontology, a well-engineered knowledge base, and company brain

They're just 3 layers in the stack to getting organizational truth into a place where you can reliably build agentic automation on top

[OpenShip @openshipio]

Introducing OpenShip.

An open-source application platform for building, deploying, operating, and scaling applications on infrastructure you own.

Replace deployment tools, managed services, and infrastructure workflows with one open-source platform.

Available today:

- Mail Server (Built in. One click).

• Runs on your own VPS
• Unlimited domains
• Unlimited mailboxes
• Modern webmail included
• Connect with Gmail, Outlook, Apple Mail, Thunderbird, or any IMAP/SMTP client
• Send email directly from your applications using SMTP
• No mailbox subscriptions or API fees
• High email deliverability

- Deployment:

• Deploy any stack
• Git-based deployments
• Zero-downtime deployments
• One-click rollbacks
• Development, Staging, and Production environments
• Multi-branch deployments with isolated environments • Deploy to VPSs, dedicated servers, cloud VMs, or your homelab

- Services:
Provision the services your applications need in one click.
Replace multiple managed providers with services running on your own infrastructure.

• Supabase
• PostgreSQL
• MySQL
• MariaDB
• MongoDB
• Redis
• MinIO
• Meilisearch
• Qdrant
• RabbitMQ
• Kafka
• ClickHouse
• Elasticsearch
...and deploy any other service alongside your applications.

- Operate:

• Live deployment logs
• Live request logs
• Real-time traffic analytics
• Automated backups
• Monitoring
• Secrets management
• Domains
• Automatic SSL
• Environment variables
• Scheduled jobs
• Health checks

-Multi Environments:

• Separate Development, Staging, and Production environments
• Deploy every branch independently
• Test changes before production
• Isolated services, secrets, and configuration per environment

- Security & Teams:

• Team management
• Role-based access control
• IP allow/block rules
• Rate limiting
• Security rules

- Developer Experience:

• Web dashboard
• Native desktop application
• CLI
• REST API
• MCP support for AI agents, just add the mcp and your agent can do the work for you
• Manage your infrastructure without living in SSH

- Coming Soon:

• Multi-server clustering for applications and databases
• One-click load balancing
• Horizontal scaling across multiple servers
• Built-in high availability and failover
• Scale from a single VPS to a cluster with the same workflow
• Just add servers. OpenShip handles the rest.
Open source.

[max fu @maxxfuu]

Day 11/90 of Inference Engineering

How does vLLM work and how is it used in production?

Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users.

With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user.

The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating.

After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming.

The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks.

All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization.

Heres a little animation that visualizes everything!
-
I've also completed the forward pass for my mnist.c project. I had a nice chat with @shreybirmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

[sucre @hamzzygeek]

@kuppylee

[sucre @hamzzygeek]

@DevFolorunso Hold jor 🤪

[sucre @hamzzygeek]

@bello_sophiat

[sucre @hamzzygeek]

Muscle daddy

[sucre @hamzzygeek]

@biggybear_dairo 🫡, my liege

[Kay 🪔 @PiousGeek]

Don’t forget to download my book here and share with you friends 😎

[sucre @hamzzygeek]

@bello_sophiat Fuwa tan 😝

[Midddleman @OurMidddleman]

Spot the price difference: Temu vs 1688

Temu = 18,792
1688 = 4,488

[sucre @hamzzygeek]

@bello_sophiat Like , na for only insta I dey tail you 😂🤪, it was like as if everyone just disappeared. I thought I was shadow ban for months

[Ewgi @Ssaasquatch]

r/Wallstreetbets literally changed my life.

[Sani Yusuf @saniyusuf]

So I just created a new @googlechrome Extension for my new tool https://t.co/C64qUd1M19 that allows you to grab metrics from any page, especially for sources that do not have public APIs like @Medium or expensive sources like @X

Chrome exention is coming today

https://t.co/hLKUHVAeNe

[sucre @hamzzygeek]

@chiefkalu_ Sweet , you never onboard me sef , Cydm

[Olabisi| Web3 Marketing @auntybisi_]

Happy Birthday Olabisi ❤️

[sucre @hamzzygeek]

@mikailhamzat @afro_launch @WatchupLTD @MuzePay @ExamBuddie @LooprailHQ https://t.co/Efv7rQ5n8n