US Patent 11475036 Streamlined data engineering

A method for standardizing data input, data output and data manipulation at a data lake is provided. Methods include receiving a data transfer instruction comprising a seed file. Methods include parsing the seed file. Methods include validating the seed file. Methods include retrieving one or more data elements from one or more data sources as specified in the seed file. Methods include saving the retrieved data elements to a data lake. Methods include archiving the data elements at the data lake. Methods include receiving a schema from a network data mover client at an edge node at the data lake. Methods include creating a table to match the schema and validating the data elements using the schema. Methods include pushing the data elements into the table using the schema. Methods include saving the table that comprises the data elements in a shoreline edge node within the data lake.