offset pagination gets slower the deeper you go.
OFFSET 100000 makes the db read 100,020 rows and throw away 100,000 of them.
cursor pagination says WHERE id > last_seen LIMIT 20. it uses the index and reads only 20.
page 1 or page 5000, same speed.
the two bigquery settings that cut your query bill:
partitioning splits the table by date. a query filtered on that date only scans those partitions.
clustering sorts data within each partition so filters skip blocks too.
partitioning must be set at creation. plan early.
retry logic without jitter is how you take down your own service.
something fails, every client backs off the exact same 1s, 2s, 4s, and retries in perfect sync. your backend gets hit by the same spike over and over.
add randomness to the delay. that's the whole fix.
batch vs streaming, the honest version:
batch: cheaper, simpler, runs on a schedule. you wait.
streaming: real-time, but you pay in complexity, ordering, and late-arriving data.
most "we need real-time" requirements are fine with a 5 minute batch.
how apis actually do rate limiting: the token bucket.
the bucket refills at a fixed rate. each request spends one token. empty bucket, you get a 429.
it allows short bursts but caps the sustained rate.
that's why you can fire 10 requests at once and then get throttled.
@wiz_io (now part of @googlecloud) shipped AI-APP for securing ai apps code to runtime.
three agents: red hunts vulns like an attacker, blue investigates, green remediates.
sleeper feature is AI-BOM. it inventories every model and ide extension writing code in your stack.
github shipped stacked pull requests in public preview.
instead of one giant pr, you chain small ones. each pr targets the branch below it, so reviewers see only that layer's diff.
the good part: branch protections and CI evaluate against main, not the branch below.
the cache-aside pattern, simply:
app checks the cache first. hit? return it. miss? read the db, write it back to the cache, return it.
your db stops answering the same question a thousand times a second.
the hard part isn't caching. it's knowing when to invalidate.
why columnar databases like bigquery are fast:
row stores read the whole row just to get one column. column stores keep each column together.
so SELECT avg(price) reads one column, not your entire table.
it's the same data, stored sideways.
opening a new db connection per request is silently expensive. tcp handshake, tls, auth, every single time.
a connection pool keeps a set open and reuses them.
it's often the single biggest fix for an api that falls over under load.
claude opus 5 dropped today. anthropic says near-frontier at half the price.
the launch matters less than one boring habit: keep a frozen eval set.
new model drops, you swap it, re-run the set. nothing regressed? ship. something did? you caught it before prod did.
a database index is just a sorted lookup pointing at your rows.
without one, WHERE email = x scans every row. 10 rows: fine. 10 million: timeout.
with an index, it jumps straight to the row.
a shocking number of "my api is slow" bugs are just a missing index.
the n+1 query problem, in one line:
you fetch 100 users, then loop and run 1 query per user to get their orders. that's 101 queries.
one join or a batched IN query does it in 1.
it looks fine on 10 rows in dev. it quietly kills your api under real load.
everyone's comparing claude code vs codex vs gpt-5.6 like the model is the whole game.
it isn't. the real lever is your setup: a tight CLAUDE.md, fresh sessions, scoped context.
a good harness makes an average model useful. a bad one wastes the best one you can buy.
a decades-old symlink trick just broke 6 of the top ai coding agents.
a repo hides a symlink where a config file should be. you approve "edit config", the agent writes to your ssh key file instead.
worst part: the agent knew. the approval prompt just didn't show you.
you need an MCP less than you think.
the github mcp loads hundreds of tools into context. every lookup burns thousands of tokens.
2-5 tools? just use tool calls.
deterministic? skip the llm.
mcp earns its place when many agents share one integration.
a clean claude code repo setup, in one picture.
CLAUDE.md for shared context. .mcp.json for tool connections. .claude/settings.json for permissions.
and the trio people mix up:
commands = you trigger them
skills = claude triggers them
hooks = run automatically on events
there's a new http method being standardized: QUERY.
basically GET with a request body — safe, idempotent, cacheable.
finally a clean way to send complex search filters without stuffing them in the url or abusing POST.
still an ietf draft, but it fixes a real pain.
quick http reminder:
never put tokens, passwords, or api keys in query strings.
query params get saved in server logs, browser history, and referer headers. anyone with log access can read them.
put sensitive data in headers or the request body instead.
every api that creates or charges something needs idempotency keys.
client sends a unique key. server checks if it's seen it before. if yes, return the saved result instead of running it twice.
this is how you stop double charges on a retry.