Disruptions in business rules execution

On July 24th, 2018 between 17.02 UTC and 20.16 UTC, an unusual increase in traffic caused SaleMove's business rule engine CPU utilization to exhaust the configured limit. The CPU resource exhaustion caused business rule processing to fail or be delayed. The incident impacted how Visitors are routed to Operators causing some Visitors to not be able to request to engage with an Operator. The Engineering team has identified issues with horizontal scaling and is taking steps to ensure the engine will work under increasing load. Meanwhile, safeguards, more robust monitoring and alerts have been put in place to avoid business rule disruptions.