Patent attributes
When a transformation job of flow logs generated for a cloud environment is triggered, a security service determines a parameterized template for batch data processing operations offered by the cloud service provider (CSP) to use based on the type of transformation job. The security service communicates an indication of the template and the corresponding parameter values to a data processing service/pipeline offered by the CSP. The provisioned processing resources retrieve the flow logs from a designated location in cloud storage, complete the transformation, and store the transformed flow logs in a new storage location. If the CSP does not provide a data processing service/pipeline which can perform bulk data transformation, the security service uses a generic parameterized template specifying a transformation job to be run on a cluster. Upon completion, the security service retrieves and analyzes the transformed flow logs as part of threat detection performed for securing the cloud environment.
