Chris Jamieson

CTO at #FinTech CCS Group, doing cool things with payments. Cyclist and runner in remission.

[Chris Jamieson @cj_iom]

@asksalesforce Thanks for the response. Can you confirm if this incident could be related? https://t.co/MI3S8Aajli

[Chris Jamieson @cj_iom]

For the past few hours we've been seeing lots of H10 errors on Heroku apps in the EU common runtime. Apps crashing randomly (every few minutes), seemingly no reason for it. Anyone else on @heroku seeing this? Nothing on @herokustatus / @asksalesforce yet

[Chris Jamieson @cj_iom]

@asksalesforce @jamesmgillard @herokustatus @heroku Thank you @asksalesforce - as far as I can see this is a very similar incident to yesterday's (incident 2868 - https://t.co/Bz0DpQMEtF) we are seeing the same behaviour across all of our applications in EU common runtime. Would be great to see something on Heroku status page

[Chris Jamieson @cj_iom]

Seeing lots of timeout (no headers received) all of a sudden across various Heroku apps. Definitely something going wrong since 18:09 BST but no acknowledgement on @herokustatus

[Chris Jamieson @cj_iom]

@jamesmgillard @herokustatus Disappointing to see no acknowledgement from @heroku on @herokustatus but unfortunately this is now their standard approach it seems - radio silence

[Chris Jamieson @cj_iom]

@jamesmgillard @herokustatus Yes it seems to have started again a short time ago - approx 1930 BST or shortly thereafter.

We are on Router 2.0

[Chris Jamieson @cj_iom]

@jamesmgillard @herokustatus Seems @herokustatus did get updated in the end and this was resolved: https://t.co/Bz0DpQMEtF

[Chris Jamieson @cj_iom]

@jamesmgillard @herokustatus Good to know it's not just me! Apps are coming back online quickly after restarting all dynos, but dying again shortly after.

[Chris Jamieson @cj_iom]

@varunbhanot @magicdotfit BANG BANG! keep smashing it

[Chris Jamieson @cj_iom]

@jordnb @humanloop Congrats on the launch Jordan. Website looks great 😃

[Chris Jamieson @cj_iom]

@asksalesforce Thanks - that's helpful! Didn't see it because it was retroactive

[Chris Jamieson @cj_iom]

Seeing a lot of instability on our @heroku apps again today, for past 1-2 hours. No incident published the the @herokustatus page though. Is it just me? Seeing outages across about 5-6 apps, no obvious error messages.

[Chris Jamieson @cj_iom]

@asksalesforce The latest update now states it explicitly:

> Impact is minor and limited to the U.S. region. We are still investigating the root cause.

If that's the case, there must be a separate incident going on which has not been published, because our apps are in the EU region!

[Chris Jamieson @cj_iom]

@asksalesforce Thanks for that. The @herokustatus update (https://t.co/YASwnCWf3g) refers to US customers only. All of our affected apps are in the Europe (common) region...

[Chris Jamieson @cj_iom]

@asksalesforce @asksalesforce There is obviously an issue but there is no information on the @herokustatus page at https://t.co/Q0wJ82Yjgq - please use your own status page to communicate with customers!

[Chris Jamieson @cj_iom]

@asksalesforce Thanks for the update. I've just received the incident notification. Only 12 hours after the issue started! 🤦‍♀️

[Chris Jamieson @cj_iom]

Seeing some issues with multiple @heroku databases on the Essential tier over the past 30-40 mins. No incidents posted on @herokustatus - anyone else or just me? Connection error: out of memory

[Chris Jamieson @cj_iom]

@asksalesforce FYI @martinbean I've been able to complete a DB upgrade on the affected databases. Seems like this issue may be resolved?

[Chris Jamieson @cj_iom]

@martinbean @heroku @herokustatus Oof... just noticed a small change on our side. The pg:info CLI command was previously returning spurious info (e.g. Data Size: unknown usage / 10 GB). It's now returning the true data size of the database. So hopefully this is an indicator someone is working on it.

[Chris Jamieson @cj_iom]

@martinbean @heroku @herokustatus Good to know it's not just me. I had several apps lose DB connection at the same time.

[Chris Jamieson @cj_iom]

@asksalesforce OK thank you. We will wait for some further information from your team.