We are experiencing failures in real-time exports
Incident Report for Lytics
Postmortem

A recent release included a bug fix intended to fix Lytics Jobs stuck in a bad state. That change introduced a side effect that impacted jobs that do real-time exports, which caused them to lose track of their profile state (a process we use to determine if a profile has been sent to external services).  

Once the Lytics team noticed the issue, we restored the profile state and began reevaluating all profiles that had been modified during the time period the state was missing. This reevaluation process should rebuild the missing state during the period it was missing and cause the Jobs to emit any missing changes. Lytics Engineering will be adding safety checks and alerts to ensure this issue does not happen again.

Posted Nov 21, 2022 - 11:58 PST

Resolved
This incident has been resolved.
Posted Nov 18, 2022 - 18:00 PST
Update
We are continuing to monitor for any further issues.
Posted Nov 17, 2022 - 14:06 PST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 17, 2022 - 13:55 PST
Identified
The issue has been identified and a fix is being implemented.
Posted Nov 17, 2022 - 13:46 PST
Investigating
We are currently investigating this issue.
Posted Nov 17, 2022 - 13:12 PST
This incident affected: API and Trigger Exports.