Ben

Yes you can put a database in a container... fight me | Unemployed Bum | Sold @instaclustr to @NetApp | @rust is cool

[Ben @BenBromhead]

It's official!

[NetApp @NetApp]

We just closed our acquisition of @Instaclustr, paving the way for NetApp as a PaaS.

Read more from @AnthonyLye on how we can deliver a best-of-suite platform for CloudOps, making it easier for our customers to succeed in complex operational environments. https://t.co/eitvSlFQuH

[Ben @BenBromhead]

@campbellclaret Off the back of your episode about fishing and the recent EU neogtiations - https://t.co/rSzqbqBD7s

[Ben @BenBromhead]

Today is my last day at @Instaclustr. Sad to leave such a rad bunch of amazing humans, excited to spend more time with another rad bunch of amazing humans ( family and friends).

[Tom Hacohen @TomHacohen]

Looks like Redis is trying to take over the all the OSS Redis libraries.

Jedis, Lettuce, and redis-py are down, they are now threatening redis-rs (link in reply).

[Ben @BenBromhead]

Whoever is responsible for the Microsoft Authenticator push notification flow for otp has done a damn fine job

[Ben @BenBromhead]

@mike_salter They are pretty good at working out your rough age for ad targeting… I think they will manage

[Ben @BenBromhead]

Super excited to be a part of the OpenSearch foundation as it creates license stability and confidence in collaboration.

[NetApp Instaclustr @Instaclustr]

We are pleased to announce the launch of the OpenSearch Software Foundation, a community-driven initiative that will support OpenSearch and its search software, which is used by developers around the world.

View the announcement: https://t.co/sE2YuT6b3O

#opensource #opensearch

[NetApp Instaclustr @Instaclustr]

We are pleased to announce the launch of the OpenSearch Software Foundation, a community-driven initiative that will support OpenSearch and its search software, which is used by developers around the world.

View the announcement: https://t.co/sE2YuT6b3O

#opensource #opensearch

[Ben @BenBromhead]

Bwahahahahah good luck https://t.co/TgzJNJxxUj.... Watch this space.. the reason why they are really doing this will become evident soon

[Ben @BenBromhead]

You know something is a giant turd when crypto/nft people use it

[Ben @BenBromhead]

The number of people who thought Telegram was a secure e2e messaging platform with no government ties and not just an unmoderated social network with "interesting" investors is hilarious

[Ben @BenBromhead]

This whole post is the best example of how SQL is not great, but it's the best solution we have to the problem right now

[Craig Kerstiens @craigkerstiens]

For as powerful as it is, SQL isn't exactly a user friendly language to work in. Debugging SQL can be slow and painful. But, you can write more legible SQL by taking advantage of one of the least used functionalities: CTEs (Common Table Expressions).

CTEs are essentially a temporary view within a query.

CTEs are named, and then can be reference later.

CTEs can build upon each other, essentially you get to create your own smaller foundational building blocks for a query just like you would in code.

There are also recursive CTEs, but we'll save that one for another day.

Lets start to take a look at them.

I'm going to start with a query for a project management tracking app that will give me all users current tasks:

WITH users_tasks AS (
SELECT
https://t.co/CT9ZaxvKrl,
array_agg(https://t.co/IevDRZJ7y1) as task_list,
projects.title
FROM
users,
tasks,
project
WHERE
https://t.co/pQ8RTsEqgt = tasks.user_id
projects.title = tasks.project_id
GROUP BY
https://t.co/CT9ZaxvKrl,
projects.title
)

Using the above CTE I could query it with:

SELECT *
FROM users_tasks;

But where it gets more interesting is when you want to chain various pieces together. So while I have all tasks assigned to each user here, perhaps I want to then find which users are responsible for more than 50% of the tasks on a given project, thus being the bottleneck. To oversimplify this we could do it a couple of ways, total up the tasks for each project, and then total up the tasks for each user per project:

total_tasks_per_project AS (
SELECT
project_id,
count(*) as task_count
FROM tasks
GROUP BY project_id
),

tasks_per_project_per_user AS (
SELECT
user_id,
project_id,
count(*) as task_count
FROM tasks
GROUP BY user_id, project_id
),

Then we would want to combine and find the users that are now over that 50%:

overloaded_users AS (
SELECT tasks_per_project_per_user.user_id,

FROM tasks_per_project_per_user,
total_tasks_per_project
WHERE tasks_per_project_per_user.task_count > (total_tasks_per_project / 2)
)

Now let me combing all of it to get a comma separated list of tasks from all over loaded people so I can then go and figure out how to reprioritize.

--- Initial query to grab project title and tasks per user
WITH users_tasks AS (
SELECT
https://t.co/pQ8RTsEqgt as user_id,
https://t.co/CT9ZaxvKrl,
array_agg(https://t.co/IevDRZJ7y1) as task_list,
projects.title
FROM
users,
tasks,
project
WHERE
https://t.co/pQ8RTsEqgt = tasks.user_id
projects.title = tasks.project_id
GROUP BY
https://t.co/CT9ZaxvKrl,
projects.title
),

--- Calculates the total tasks per each project
total_tasks_per_project AS (
SELECT
project_id,
count(*) as task_count
FROM tasks
GROUP BY project_id
),

--- Calculates the projects per each user
tasks_per_project_per_user AS (
SELECT
user_id,
project_id,
count(*) as task_count
FROM tasks
GROUP BY user_id, project_id
),

--- Gets user ids that have over 50% of tasks assigned
overloaded_users AS (
SELECT tasks_per_project_per_user.user_id,

FROM tasks_per_project_per_user,
total_tasks_per_project
WHERE tasks_per_project_per_user.task_count > (total_tasks_per_project / 2)
)

SELECT
email,
task_list,
title
FROM
users_tasks,
overloaded_users
WHERE
users_tasks.user_id = overloaded_users.user_id

CTEs won’t always be quite as performant as optimizing your SQL to be as concise as possible. But generally what they will be is more readable and thus easier to debug and collaborate with others on.

While the formatting may not look amazing on twitter, give them a shot the next time you have to write some SQL report.

[Ben @BenBromhead]

Truth. You do enough serious GenAI stuff, you end up spending most of your time on the same old ML and data science problems

[Scott Hanselman 🌮 @shanselman]

It's fun to say AI. "Artificial Intelligence" sells newspapers. But Machine Learning and Data Science are the base of the pyramid. Be like Kelsey (and me 😁) - focus on the big picture and the fundamentals.

[Ben @BenBromhead]

Happy fucking freedom from getting emails from American colleagues day

[Colt McNealy @coltmcnealy]

The difference between Kafka 4.0 and 3.0 will be awesome:

- No ZooKeeper
- Tiered Storage
- New Rebalance Protocol (848)
- EOS support in Connect
- State Updater Thread in Streams
- Improved Txn Protocol (890)
- Custom Stream Task Assignment (924)

#apachekafka is on a roll!

[Ben @BenBromhead]

Amazing hard hitting data journalism right here - https://t.co/nG4yLDORwz

[Ben @BenBromhead]

The amount of genius level attempts at the neuralink compression challenge who get rid of the noise and still call it lossless is... *chefs kiss*

[Chris Albon @chrisalbon]

Years ago @vboykis sent me a wonderful essay on exactly this.

Sadly, it's only available on the internet archive now: https://t.co/pmg2sTFTFB

[Ben @BenBromhead]

Not sure why you would exclude this. Most things a commercial enterprise would open source are enabling and not core…. That’s usually why they do it (e.g Cassandra, Kafka, Clickhouse, Cadence etc)

[Ben Kehoe @ben11kehoe]

Things like Firecracker don't count, they are enabling technologies but aren't a product themselves

[Ben @BenBromhead]

I’d be interested to see what the impact of their other distribution channels are like on this number. E.g consumption via Azure, Bing etc

[swyx @swyx]

🆕 The Unbundling of ChatGPT

https://t.co/6cLKLY7OwS

A whole year has passed with ~0 growth in ChatGPT user numbers. Instead, users are exploring a whole host of verticalized players for Imagegen, Writing, Coding, Research, Agents, Voice and RAG.

@OpenAI still wins in that scenario, but probably needs to release Sora and GPT-5 soon to stall mass unsubscriptions from ChatGPT.

[Ben @BenBromhead]

Please tell any schools outside the US that ties died 50 years ago…

[derek guy @dieworkwear]

not convinced that anyone who talks like this has worn a tie in this century. ties died 50 years ago. saying you hate ties when they haven't been relevant in your lifetime is pure performance. if you want to be counter-cultural, hate business casual