Delay in Live Messaging
Incident Report for ChatDaddy
Postmortem

This was the same issue as the one on Oct 31. We finally managed to track the source of the delay, that being the incorrect reading of “high memory usage” in our queuing system that prevented more events from being added. We’ve added more memory to counteract this as well as increase the threshold to high memory usage. Furthermore, the internal alarm setup on Oct 31 helped catch this incident early & resolve it much faster than the incident on the 31st.

Posted Dec 01, 2024 - 13:53 UTC

Resolved
This incident has been resolved.
Posted Nov 11, 2024 - 03:24 UTC
Investigating
We're investigating. Please refresh the page to review live data
Posted Nov 11, 2024 - 02:28 UTC
This incident affected: microservice (Instant Messaging Service, Keyword Reply Service, Flow service).