Mark Gordon

20-year tradesman turned agency founder. FieldTech helps fellow owners automate, market, and streamline their business.

[Mark Gordon @FieldTechMark]

Most people still think the freezers were just turned off.
They weren’t.

Someone forced the cases into continuous defrost and silenced the temperature alarms. Normal defrost is 10–15 minutes. Left on, the cases climb to 50°C fast while the system can still look okay from the outside.

That’s why stores didn’t get the usual early warning. The same access that controls the equipment also controls the alarms.

This isn’t complicated. A lot of food infrastructure is still running on older remote-access setups that were never designed for this. When it fails, the cost doesn’t stay with the chain, it’s passed on to the consumer!

[Mark Gordon @FieldTechMark]

I’ve worked in commercial refrigeration for 20 years and have been seeing a lot of comments asking why freezers and coolers are connected to the internet, and how they could get this warm. A bit of context:

Commercial refrigeration has used DDC (Direct Digital Control) and BAS (Building Automation System) platforms for decades. These systems continuously monitor temperatures, pressures, and equipment status. If a freezer fails at 11 p.m., they automatically alarm and dispatch a technician before product is lost. Across grocery stores and cold storage facilities, this prevents billions of dollars in food loss every year. They also let technicians remotely diagnose and fix issues before they become failures.

What appears to have happened here is that the attackers forced the cases into a prolonged defrost cycle instead of simply turning them off. In freezers, defrost normally uses electric heaters or hot-gas defrost for 10 to 15 minutes every few hours (or as needed by sensors) to clear ice off the evaporator. In coolers it’s usually just stopping the refrigerant flow while the fans run. These cycles are short and controlled so the product stays in a safe temperature range. If someone overrides that and leaves the heaters or hot gas running, temperatures climb quickly, which matches the 50 °C reports.

It’s fair to ask whether these systems should be online. In practice, modern grocery and industrial plants are too complex to operate without remote monitoring and control. Removing that connectivity would raise food costs and increase spoilage. These platforms are normally locked down with multiple layers of security; seeing them successfully attacked at this scale is genuinely concerning.

Hope that helps explain how something like this can happen.

[Mark Gordon @FieldTechMark]

@SYBIXBT @coinbureau Defrost cycles. They use heaters to melt ice off the evaporator when manually over ridden on they essentially turn the insulated box ment to keep things cold into a sauna

[Mark Gordon @FieldTechMark]

@Charlesvoicema1 @TheInsiderPaper Literal tons of waste water are dumped into the St Laurent River. Treated (still gross and full of chemicals) but if treatment facilities go offline I would recommend sourcing water from tributaries not any major river.

[Mark Gordon @FieldTechMark]

@Hodlwithnoah Never ! Lol those are dangerous 😅

[Mark Gordon @FieldTechMark]

Fair point and raises the question of where all the good people have gone. People use to make an honest living at a grocery store, worked there for years or decades held and passed down a standard but a separate issue to this that seems mostly gone. All about squeezing margins and cutting labour and wages. My honest take

[Mark Gordon @FieldTechMark]

@Working0115 @HormuzLetter When overriding a defrost cycle like this, it’s just as easy to override or silence the temperature alarms in the same control system. These hackers clearly had a strong understanding of the setup and how to maximize damage

[Mark Gordon @FieldTechMark]

@smerkinkones @HormuzLetter Sorry mis read your comment. But I agree

[Mark Gordon @FieldTechMark]

@comrade_mog @DanEastman2023 @HormuzLetter I’m not an expert on the security side, can only say some companies put a lot more into it than others and what I do know is it’s not cheap at all. Will be very expensive I’m sure.

[Mark Gordon @FieldTechMark]

@smerkinkones @HormuzLetter Apparently it was danfoss. But I’m not 100% sure.

[Mark Gordon @FieldTechMark]

I agree. The IT side of it moves away from my specialties but anything connected to the internet needs to be on a secure vpn. I think we have assumed for too long that hackers only want money or ID information, ignoring the possibility of wide spread destruction on critical infrastructure

[Mark Gordon @FieldTechMark]

@JakeKnight2767 @HormuzLetter Fair point. No argument. Grocery stores need to take notice. This kind of food waste is scary but also upsetting

[Mark Gordon @FieldTechMark]

@JexiumIslandJem @ascshank @HormuzLetter Open facing ips rather than secure vpns, security from my experience was always focused more on separation to online shopping and payment information

[Mark Gordon @FieldTechMark]

Pretty sure they just weren’t properly secured. Most systems now sit behind VPNs. These were almost certainly open-facing IPs, which is stupid for a big chain. Remote access is still essential though. A tech needs to be able to get in and stabilize a system before driving to the store. The fix is locking that access down, not removing it.

[Mark Gordon @FieldTechMark]

I’ve worked in commercial refrigeration for 20 years. It wasn’t that the temperature was set to 50°C. The attackers forced the cases into a prolonged defrost cycle instead of just turning them off. In freezers, defrost normally uses heaters or hot gas for 10–15 minutes to clear ice off the evaporator. If that stays on, temperatures climb quickly, which is how the cases got that warm.

[Mark Gordon @FieldTechMark]

I’ve worked in commercial refrigeration for 20 years. These systems have used remote monitoring for decades so a tech can catch a failure overnight and fix it before product is lost. What happened here was someone forcing the cases into a prolonged defrost cycle instead of just turning them off. In freezers that means heaters or hot gas running way longer than the normal 10–15 minutes, which is how temperatures climb that fast.

The systems are online for a practical reason. The problem is a lot of them are still sitting on public IPs. A secured VPN so a tech can still get in when something’s failing, without the controllers being reachable from the open internet, is the minimum at this point.

[Mark Gordon @FieldTechMark]

I’ve worked in commercial refrigeration for 20 years. These systems have used remote monitoring and control for decades so a tech can catch a failure overnight and fix it before product is lost. What happened in Europe was someone forcing the cases into a prolonged defrost cycle instead of just turning them off. In freezers that means heaters or hot gas running way longer than the normal 10–15 minutes, which is how temperatures climb that fast.

Disconnecting everything isn’t realistic for most stores. The practical fix is that these controllers stop sitting on public IPs. A secured VPN so a tech can still get in when something’s failing, without the systems being reachable from the open internet, is the minimum at this point.

[Mark Gordon @FieldTechMark]

You’re not missing much on this one. The attack caused a lot of spoilage and will raise insurance premiums for the stores that got hit. The difference is between one deliberate incident and the everyday failures that remote monitoring normally catches before product is lost. Both are real.

What should be non-negotiable now is that these systems stop sitting on public IPs. A properly secured VPN so a tech can still get in when something’s failing, without the controllers being reachable from the open internet, is the minimum at this point.

[Mark Gordon @FieldTechMark]

True. I’ve seen the tighter security around payment systems after the Target breach and other credit card incidents. Companies locked that side down. My wording was misleading though. The concern was never really on the mechanical side of these systems being hacked. That part was designed and left online long before anyone was thinking about it as a target.

[Mark Gordon @FieldTechMark]

I get the idea. On oil or refrigerant pressure failures there are usually hard safeties that shut things down. Defrost is different.

Hot gas defrost, if you force it long enough, will eventually raise head pressure as the hot gas makes its way back to the compressor and trip those safeties. Electric defrost is more of a problem. If someone overrides the controller, the normal time and temperature termination can be bypassed. Without knowing the exact heater high-limit design on every case, those heaters can essentially be forced on well past what the system was meant to allow. Temperature alarms and setpoints can also be changed from the same controller.

Whoever did this clearly understood how these systems work and how to maximize the damage.

[Mark Gordon @FieldTechMark]

@ButterWeenie I agree on the collusion. In commercial refrigeration controls it’s really a small handful of companies, with Danfoss being the biggest by far. That concentration keeps costs high and locks in a lot of the design choices.

[Mark Gordon @FieldTechMark]

I’ve worked in commercial refrigeration for 20 years. Watching the freezer stories today and reading the comments has been interesting.

A lot of people are suddenly experts on systems they’ve never touched. Commercial refrigeration has used remote monitoring and control for decades for a reason. It catches failures before product is lost and keeps the stores running. When someone overrides the controls and forces a defrost to stay on, temperatures climb fast. That’s not complicated. It’s just how the equipment works.

The bigger issue is the same one I see on the job all the time. New tools get treated like they can replace the person who actually knows the system. They can’t. Used carefully they help. Rushed in or set up on the cheap, they create new problems or leave you exposed when something goes wrong.

The tech is useful. The person who understands the equipment, the job, and what happens when it fails is still the part that matters. Cheap and fast usually costs more later.