Kevin Benton

distributed systems and networking

[SpaceX @SpaceX]

Mechazilla has caught the Super Heavy booster!

[Starlink @Starlink]

Reliable high-speed, low-latency internet around the world—even while traveling at 27,000 km/h through a plasma field 🛰️🚀

[Kevin Benton @ktbenton]

@ArizonaDOT will the new SR-24 extension due to complete this year open in phases (e.g. just to Williams field rd) or will it only open once the path all of the way to Iron Wood is ready?

[Kevin Benton @ktbenton]

@bradfitz Yeah… an OUTPUT packet has to match a route first to even make it to the mangle table where you could mark it and use an ip rule to give it the real route you want to steer it. There is something slightly off in that packet pipeline design that makes it hard to internalize

[Kevin Benton @ktbenton]

@thockin Changing images is definitely fine. Just want to know what folks end up doing to share fate with sidecars.

[Kevin Benton @ktbenton]

@thockin I came across a mega bug thread re: k8s Deployments only supporting “Always” for restartPolicy. That linked to a feature request where you said people are doing it wrong.

What is the right way to deal with a pod where one container crashing breaks all of them?

[Kevin Benton @ktbenton]

@netpeeker @Ben_Pfaff So a new VM coming online would notify every compute node with a VM in the same sec group and all of those nodes would then have to re-pull the security group. N^2 during a launch of 1k VMs in the same group generates some good hate mail.

[Kevin Benton @ktbenton]

@netpeeker @Ben_Pfaff Including the full state was a big improvement that wasn’t originally there. When everyone was feeling the pain and hating on rabbitmq it was just basically like “UUID X changed”, then all of the agents would then have to hammer the server with a query to grab X.

[Kevin Benton @ktbenton]

@Ben_Pfaff Nah, if a new one came up it wouldn’t get old stuff. It would start the subscription for new events and then do a query for the state of the world.

[Kevin Benton @ktbenton]

@Ben_Pfaff Additionally, our patterns were very inefficient early on (notify every SG member something changed and then all need full refresh which was also done over msg queues).

[Kevin Benton @ktbenton]

@Ben_Pfaff I don’t think there were fundamental issues. The primary issue was that there wasn’t clarity on the durability requirements of the various queues so to be safe all queues had to be persisted to disk and that was a scaling nightmare because of msg volume.

[Kevin Benton @ktbenton]

@mikal Published in 2016 in case anyone else is reading and wondering why Google+ is being talked about like it’s a thing :)

[Kevin Benton @ktbenton]

@ArizonaDOT Oh, too bad. I’ve heard it’s a nice scenic drive on that section near Roosevelt. Thanks for the quick reply and info

[Kevin Benton @ktbenton]

@ArizonaDOT is it possible to get from Phoenix to Roosevelt lake on SR-88 or are there still closures that prevent driving all of the way through?

[Kevin Benton @ktbenton]

@mikal Yeah, I think if you’ve already properly committed to managing instances as cattle you would have the scaffolding in place to fan out events to all of your instances to update the iptables rules on membership changes. That wasn’t the case for many customers early on though.

[Kevin Benton @ktbenton]

@mikal Openstack security groups can also refer to a group ID in the source/dest so rules are dynamically updated as instances in a reference group come and go.

[Kevin Benton @ktbenton]

@nickbender Thanks. Did you get the chair too?

[Kevin Benton @ktbenton]

@nickbender What desk is that one? You recommend?

[Kevin Benton @ktbenton]

@distroguy @mikal Yeah, that qbr in the compute node side serves yet another purpose (security groups filtering with iptables). Later versions of openstack have a native OVS security groups option (enforcement in OF rules with conntrack actions) that get rid of that intermediary Linux bridge.