The big bitter lesson question in the applied AI layer is that as models get better and better, what is the best way to diffuse that intelligence into an enterprise.
Ironically, I’d argue that as models are getting more capable, and can handle longer running tasks and better tool use, the ability to differentiate on a domain by domain basis goes up instead of down.
When models were quite limited in what they could do, ironically all you could be literally was a “wrapper” because you were effectively delivering tokens back to the user with a custom prompt or some limited RAG.
But in a world of agents using tools, tasks that run on for hours or days, workflows that kick off agents, agent swarms that need to be coordinated against a goal, data sources that are domain specific, model routing that requires domain-specific routing, implementation that requires real change management of the workflow instead of “just type into this chat box”, it’s very very possible that specialization of these systems goes up more.
Obviously this doesn’t apply to everything - the closer you are to the labs core use-cases the tougher this play gets, but I’d argue we’re going to see autonomous systems for a wide range of job functions that weren’t practical to build out before. We’ll see this in security, life sciences, healthcare, finance, manufacturing, and many other fields. Either way going to be a fun and wild couple of years.
BOX CEO 对于 AI 和软件的关系的判断:
大部分人最大的误判是以为「世界只需要那么多软件」,自动化一来工程岗位就会变少。现实正好相反:正因为 agents 会用工具、操作电脑、即时写软件、又以远超人类的速度处理数据,以前根本没法数字化的问题现在都能「扔软件」去解决。结果是工程在经济里的权重上升,银行新交易形态、生命科学实验、制造业供应链自动化,越来越多工作开始长成软件问题的形状。他承认未来不会再大量手写代码,但几乎一切都会变成 code,懂它的人处于极有利位置。
The biggest misread of anyone predicting the end of software engineering was imagining that the world only needed a certain amount of software, and bringing automation to the field would reduce the need for labor.
Exactly the opposite is happening where essentially most things are actually turning into software. Because agents can use tools, work with computers, write software on the fly, and process data like humans (but far faster), we can now throw “software” at problems that never could reasonably be digitized before.
What that leads to is the role of engineering taking on greater importance in the economy, because we can now engineer far more systems and processes than were possible before.
If you’re a bank trying to do new forms of trading, engineers are often your first bet. If you are in life sciences, you expect more and more research and lab work to be engineering centric. If you’re in manufacturing, you’re trying to automate more of the supply chain, all that is engineering. More and more work is shaped like software problems.
Yes we’re not going to be coding manually in the future. But almost everything will be code, which means those that understand it are in a fantastic position.
If you learn Forward Deployed Engineering, you will never be unemployed again.
I’d do this.
Stage 1: Enterprise Identity & Access (The Gatekeeper)
- Learn: OAuth2, SAML, OIDC, SCIM provisioning, Row-Level Security (RLS) and multi-tenant data isolation.
- Practice: Build an auth bridge connecting your AI app to a simulated customer's Okta/Azure AD, ensuring User A can only query data their specific RBAC role permits.
- Why: No SSO and strict data isolation means no enterprise contract. Security review is the first gate.
Stage 2: Air-Gapped & VPC Deployments (The Environment)
- Learn: Kubernetes (K8s), Helm, Terraform, private container registries, air-gapped networking and egress filtering.
- Practice: Script a one-click Terraform/Helm deployment of your entire AI stack into a simulated customer VPC with zero egress to the public internet.
- Why: Banks, hospitals and governments don't let your AI talk to the public cloud. You must deploy in their basement.
Stage 3: Legacy Data Extraction & ETL (The Fuel)
- Learn: OCR, unstructured PDF parsing, mainframe DB connectors, Apache Airflow, dbt and data reconciliation.
- Practice: Build a pipeline that ingests 10 years of messy, scanned customer invoices and turns them into a clean, queryable hybrid (vector + relational) store.
- Why: Enterprise AI starves without clean data. The customer's data will always be a mess.
Stage 4: Secure RAG & Permission-Filtered Retrieval (The Brain)
- Learn: Hybrid search, document-level access control, citation grounding, metadata tagging and abstention logic.
- Practice: Build a RAG pipeline where the LLM dynamically filters out retrieved chunks that the specific user asking the question doesn't have clearance to see.
- Why: An AI that leaks HR salaries or M&A docs to a junior employee is an immediate breach of contract.
Stage 5: MCP & Legacy System Integration (The Hands)
- Learn: Model Context Protocol (MCP), SOAP/REST translation, web scraping, headless browsers and API rate-limit handling.
- Practice: Build an MCP server that lets an LLM safely query and write to the customer's 15-year-old on-prem SAP or Oracle instance without exposing raw credentials.
- Why: Agents are useless if they can't touch the customer's actual systems of record.
Stage 6: AI Guardrails & PII Redaction (The Shield)
- Learn: NER (Named Entity Recognition), deterministic regex filtering, LLM output validation, UUID masking, and audit logging.
- Practice: Build a middleware proxy that intercepts LLM inputs/outputs, masks SSNs and credit cards with UUIDs before they hit the model, and restores them post-generation.
- Why: Compliance teams will kill the deal if a single PII token hits an external frontier API.
Stage 7: Human-in-the-Loop (HITL) Workflows (The Trust)
- Learn: Temporal {dot} io, webhook routing, state machines, durable execution and UI approval gates.
- Practice: Build a workflow where the AI drafts a $50k purchase order, but physically pauses execution until a human manager clicks "Approve" in Slack or Teams.
- Why: Enterprises adopt AI only when humans remain legally accountable for the final action.
Stage 8: Edge & On-Prem Inference (The Silicon)
- Learn: vLLM, TensorRT-LLM, Ollama, GPU partitioning (MIG), and local quantized models (AWQ/GGUF).
- Practice: Deploy a quantized 8B model on a single customer-owned GPU node to handle highly sensitive triage locally without network latency.
- Why: Sometimes the data is too hot to move. You must bring the model to the data.
Stage 9: Observability & Customer Health (The Pulse)
- Learn: OpenTelemetry, Langfuse, custom Grafana dashboards, usage telemetry and cost-attribution.
- Practice: Build a "Customer Health" dashboard showing the exact cost-savings, time-saved and token-spend the AI has generated for their specific team this month.
- Why: If the customer doesn't see the ROI quantified, they will churn at renewal.
Stage 10: Evals on Customer Data (The Proof)
- Learn: Golden dataset creation, trajectory grading, shadow testing, and LLM-as-a-judge calibration.
- Practice: Take 100 real past customer support tickets, run them through your agent in shadow mode, and prove to the VP of Support that it resolved 80% accurately.
- Why: Generic benchmarks mean nothing to a buyer. You must prove the AI works on *their* messy data.
Stage 11: Technical Discovery & Scoping (The Translation)
- Learn: Stakeholder interviewing, architecture diagramming, SOW (Statement of Work) writing and expectation management.
- Practice: Translate a vague "we want AI for supply chain" executive request into a scoped, 4-week technical deployment plan with defined, measurable success metrics.
- Why: FDEs are the translators between revenue (Sales) and reality (Engineering).
Stage 12: The Handoff & Self-Serve Admin (The Scale)
- Learn: Technical writing, runbooks, admin portals, automated onboarding and chaos testing.
- Practice: Write a deployment runbook and build an admin UI so the customer's internal IT team can manage the AI stack, update prompts and view logs without calling you.
- Why: Your leverage is measured by how little the customer needs you for day-to-day operations.
AI labs build the engine. FDEs build the roads.
The modern Forward Deployed Engineer builds the secure, durable nervous system that connects frontier intelligence to legacy reality.
Bookmark and Repost!