aws-etl-orchestrator | serverless architecture for orchestrating ETL jobs | Cloud Storage library

aws-etl-orchestrator is a Python library typically used in Storage, Cloud Storage, Spark, Amazon S3 applications. aws-etl-orchestrator has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. However aws-etl-orchestrator build file is not available. You can download it from GitHub.

[the challenge of orchestrating an etl workflow] #the-challenge-of-orchestrating-an-etl-workflow). [example etl workflow requirements] #example-etl-workflow-requirements). [the etl orchestration architecture and events] #the-etl-orchestration-architecture-and-events). [modeling the etl orchestration workflow in aws step functions] #modeling-the-etl-orchestration-workflow-in-aws-step-functions). [handling failed etl jobs] #handling-failed-etl-jobs). [preparing your development environment] #preparing-your-development-environment). [the packagelambda build command] #the-packagelambda-build-command). [the deploylambda build command] #the-deploylambda-build-command). [the createstack build command] #the-createstack-build-command). [the updatestack build command] #the-updatestack-build-command). [the deletestack build command] #the-deletestack-build-command). [the deploygluescripts build command] #the-deploygluescripts-build-command). [putting it all together: example usage of build commands] #putting-it-all-together-example-usage-of-build-commands). # introduction extract, transform, and load (etl) operations collectively form the backbone of any modern enterprise data lake. it transforms raw data into useful datasets and, ultimately, into actionable insight. an etl job typically reads data from one or more data sources, applies various transformations to the data, and then writes the results to a target where data is ready for consumption. the sources and targets of an etl job could be relational databases in amazon relational database service (amazon rds) or on-premises, a data warehouse such as amazon redshift, or object storage such as amazon simple storage service (amazon s3) buckets. amazon s3 as a target is especially commonplace in the context of building a data lake in aws. aws offers [aws glue] which is a service that helps author and deploy etl