Koyeb - Elevated rate of 404 and 502 on healthy services – Incident details

All systems operational

Elevated rate of 404 and 502 on healthy services

Resolved
Partial outage
Started 3 months agoLasted about 2 hours

Affected

North America

Partial outage from 10:28 AM to 12:18 PM, Operational from 12:18 PM to 12:20 PM

Washington, D.C. - WAS

Partial outage from 10:28 AM to 12:18 PM, Operational from 12:18 PM to 12:20 PM

San Francisco - SFO

Partial outage from 10:28 AM to 12:18 PM, Operational from 12:18 PM to 12:20 PM

Europe

Partial outage from 10:28 AM to 12:18 PM, Operational from 12:18 PM to 12:20 PM

Frankfurt - FRA

Partial outage from 10:28 AM to 12:18 PM, Operational from 12:18 PM to 12:20 PM

Paris - PAR

Partial outage from 10:28 AM to 12:18 PM, Operational from 12:18 PM to 12:20 PM

Updates
  • Resolved
    Resolved

    The incident is now resolved, we are sorry about the impact, and we are available to discuss the details if needed.

  • Monitoring
    Monitoring

    We implemented a fix and have re-scheduled most of the impacted failing service.

    The initial incident triggering 404 was handled very shortly after it was detected, we noticed that some users were still experiencing some 503 due to an automatic garbage collection of some network resources.

    Then we implemented a fix to recover the data that were garbage collected.

  • Identified
    Identified

    The recovery was done under 5m for most app, but a portion of user were still facing 503, we are currently trying to resolve this issue.

  • Investigating
    Investigating

    We are currently investigating this incident, it seems it was caused by a configuration error from our side.