US Patent 11620157 Data ingestion pipeline anomaly detection

Systems and methods are described for processing ingested pipeline metrics and ingested logs in an asynchronous manner as the data is being ingested to explain anomalies detected in the pipeline metrics using the ingested logs. For example, one or more streaming data processors can convert data as the data is ingested into a comparable data structure, determine whether the comparable data structure should be assigned to an existing data pattern or a new data pattern, and determine whether the logs corresponding to the comparable data structure is anomalous. Separately, the streaming data processor(s) can perform an outlier detection on the pipeline metrics to detect outliers. The streaming data processor(s) can then window the anomalous logs and the pipeline metric outliers to surface explanations for the pipeline metric outliers using the anomalous logs.