Jon Polaski

Software Eng @Disney @ESPN

[Anuj @anujcodes_21]

🚨 Anthropic just showed a 27-minute workshop on how to actually do prompts for Claude.

Taught by the people who built it.

Free. No registration. No paywall.

I've seen $300 courses that don't cover what they teach in the first 8 minutes.

Watch it and bookmark it now.

[Movez @0xMovez]

Spotify's Chief Architect just showed how they ship 4,5K deployments /day with Claude at Anthropic stage

27-minutes. free. By #1 music app dev

"More than 99% of our engineers use AI coding tools. Adoption took off after Opus 4.5"

Worth more than any $500 vibe-coding course.

[ESPN Front Row @ESPNFrontRow]

“The best part is, we’re just getting started.”

ESPN Chairman Jimmy Pitaro reflects on last week's direct-to-consumer & enhanced app launch

See how ESPN employees celebrated the debut ➡️ https://t.co/RkDBxKMKAv

[Jon Polaski @polaskj]

@JustinThomas34 shank hole 2, #travelerschampionship

[Jon Polaski @polaskj]

#travelerschampionship Tree narrowly avoids hitting patrons at the Treehouse Brewing beer garden.

[Gwen (Chen) Shapira @gwenshap]

A few months ago, I knew almost nothing about AI. Last week, I completed my first AI-based product: AI Code Assistant. I decided to share my learnings in a blog series. Starting with basic concepts - because reading a single tweet shouldn't require a PhD.
https://t.co/RldzrgXeda

[@norootcause.surfingcomplexity.com on Bluesky @norootcause]

In a complex system, there isn’t a “safety” knob that you can just turn to the right to increase safety. Safety features increase complexity (new failure modes!) and have opportunity costs (finite resources!). Every intervention involves a tradeoff.

[terminally onλine εngineer @tekbog]

it's just sending one json from one service to another how hard can it be?

[Stanislav Kozlovski @kozlovski]

I wish I had this clear of an explanation of Kafka's Consumer Group rebalance protocol when I was starting out...

But I didn't. So I'm making it now. (2 minute read)

The protocol is very intricate.

It was intentionally designed with clear separation of concerns:

🟣 the broker knows about group membership & subscribed topics
🟣 the consumers know about partition assignment

Notably, the clients themselves choose how to distribute the load among themselves - they pass this information through an embedded protocol within the general rebalance.

Design-wise, this makes sense because every application is different and it knows best how to split its work appropriately. 🍱

✌️ The rebalance consists of two steps:
1. Collect or reaffirm group membership information.
2. Compute and propagate partition assignment.

For the pictured example, imagine the following 3/3/3 scenario:

We have 3 consumers consuming from a topic with 3 partitions, and 3 new partitions were created in the same topic. 3️⃣

🎬 Phase 0 - Initiate the Rebalance?
How do the clients know that a rebalance is taking place?

There can be different triggers for a rebalance - here it is a notification of new partitions, but it can be new consumer clients coming online or leaving the group, etc.

In any case - the same path is used.

📷 Not pictured here: every consumer sends lightweight heartbeat requests to the Group Coordinator broker.

When the broker realizes that a rebalance should happen, it starts responding to the heartbeats with a special response that informs the clients. The consumers then start the rebalance dance.

🎬 Phase 1 - JoinGroup

A JoinGroup request is sent to the Group Coordinator by every consumer. This informs the Group Coordinator broker of what consumer clients are part of the group.

The very first JoinGroup request the Coordinator receives is special:

⏱ - after it, the max poll interval timeout starts ticking...
👑 - the consumer that sent this first request is crowned the Group Leader!

The JoinGroup request sent by each consumer contains the protocol type and the protocol-specific metadata. This is the embedded protocol.

For regular Kafka consumers - that’s usually the partition assignment that the consumer is assigned, among other things like rack id, consumer instance member id, and other things specific to the configured partition assignor.

For Kafka Connect - there's other information like URL and offset of the worker.

The broker is agnostic to this information - it doesn’t parse it, it simply acts as a simple proxy and re-sends the same bytes.

The Group Coordinator broker responds when either:
🟣 - all known members have sent a request
🟣 - max_poll_interval_ms passes

Once this is fulfilled, the Coordinator starts responding to all the JoinGroup requests it received so far.

Note that this acts as a sort of synchronization barrier for the protocol. 🛡

At this point of response, we know that no member of the group is consuming from any partition:

🟣 - if the member sent a request, it willfully stopped consumption. (note there is a different type of rebalance - the incremental cooperative one - that changes details here)
🟣 - if the timeout passed, the member isn't part of the consumer group anymore.

The JoinGroup response sent to the Group Leader client is special - it has all of the other JoinGroup request information sent by the other members. ⭐️🧳

The GroupLeader then uses this information to craft a partition assignment for every consumer, and the second phase starts:

🎬Phase 2 - SyncGroup

A SyncGroup request is sent by each consumer immediately after they process the JoinGroup response.

The Group Leader’s SyncGroup request sends all the data (the computed partition assignment) that should be sent to each member.

💿 When the broker receives that request, it immediately persists this new group state to disk by appending a new record to the `__consumer_offsets` partition it leads.

Then, the Coordinator starts responding to the requests - propagating the particular consumer's assignment in its specific SyncGroup response.

🏁 The rebalance is then over

[Ryan Els @RyanEls4]

Do you agree 🫴🏻🤓

[Alex Cohen @anothercohen]

My wife: please start making your coffee at home so we can afford to buy a house one day

Me: 👍

[Riccardo Causo @rixlabs]

If you re a Senior Engineer you should always be available to answer questions.

The answer to: "Do you have a minute?" should in most cases be YES.

Now, all the hyper focus people will be mad at me but follow me for a minute.

The benefit to the team of putting someone in a position to work is incomparably greater than the benefit to the project of your time.

Cases where this does not apply are extremely rare and generally involve being busy with a client or otherwise engaged in an important meeting.

Your role as a senior is to be an enabler. You unblock people, you mentor and you guide people in the direction of a solution and the understanding of it.

But pay attention! Do not promote babysitting. If you get interrupted with lazy question you need to point it out.

You should reserve some focus time and communicate this schedule to the team, I'm not a promoter of hard core context switching, but a moment of your time can save hours of head bashing against the wall.

This is part of the responsibility of a Senior. You don't need to take this responsibility if you don't want it but remember that these are the things that will make you progress. Personally, as an engineer and most probably in the salary range.

[Jon Polaski @polaskj]

@CHBernasconiC “It depends” There’s a few online calculators that can help guesstimate based on index & data sizes, but generally accepted answer is 70%-80% RAM. You might find your working set has outgrown your hardware Think of it this way - anything not served from there is paging to disk

[Josh Long @starbuxman]

I love @SpringBoot 3.2!!

- reactive cache/scheduled abstraction
- virtual threads
- CRaC
- Spring for @apache_pulsar
- RestClient / JdbcClient
- reloadable SSL
- @GraalVM
- @java 21
- observability
- spring.main.keep-alive=true
- @Docker for @neo4j, activemq, etc
- new dependencies
- TransactionalExecutionListener

Try it out: https://t.co/1zelhVJx8l

For the full details see release notes for @springboot (1) and @springframework (2) and check out this @springtipslive video (3)

1. https://t.co/F6wMyU4wZP
2. https://t.co/F6CuXdbGol
3. https://t.co/0CZAhHEtil

[@jason @Jason]

Write-first culture is a game changer. I highly recommend learning about @JeffBezos’ management strategy around six pagers/memos — it really works for important decisions https://t.co/fZ9azNodC2

[Mitchell Hashimoto @mitchellh]

10x engineers blah blah blah who cares. The engineers that impress me the most are the ones that fearlessly dive into code, domains they know absolutely nothing about and end up producing amazing work in a short amount of time anyways. Truly awe-inspiring every time I see it.

[Adam Dymitruk @adymitruk]

Agile promised to make software 'nimble'. But have we become agile dancers or just headless chickens? Iteration without deep reflection is just motion, not progress - retrospectives held so often become toothless. Remember, a methodology is only as good as the minds applying it. Don't let 'Agile' be an excuse for lack of vision.

[Jon Polaski @polaskj]

@MarkSailes3 Would love to see progress & support for strongly-typed EventBridge events. Holding out for this🤞https://t.co/quKrdSyqgw