Patent attributes
A computer-implemented method, system and computer program product for optimizing a flow of data within transform, load (ETL) data processing pipelines. Database columns from a source database that are to be transformed as well as not transformed in the data processing stages of a processing segment of an ETL data processing pipeline are identified. Those database columns that are to be transformed are grouped into a processing schema. Transformations are then performed on the database columns of the processing schema. Those database columns that are not to be transformed are grouped into a non-processing schema. A large object data type (LOB) is then created to reference the non-processing schema. An identifier is created and inserted in the data processing stages to identify the LOB in replace of the database columns that are not to be transformed thereby avoiding the copying of the database columns that are not to be transformed.
