dockerflow | workflow runner that uses Dataflow | BPM library

by googlegenomics Java Version: Current License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions(1)Vulnerabilities Install Support

kandi X-RAY | dockerflow Summary

dockerflow is a Java library typically used in Automation, BPM, Docker applications. dockerflow has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. However dockerflow has 11 bugs. You can download it from GitHub.

Dockerflow makes it easy to run a multi-step workflow of Docker tasks using Google Cloud Dataflow for orchestration. Docker steps are run using the Pipelines API. You can run Dockerflow from a shell on your laptop, and the job will run in Google Cloud Platform using Dataflow's fully managed service and web UI. Dockerflow workflows can be defined in YAML files, or by writing Java code. Examples of workflows defined in YAML can be found in. Examples of workflows defined in Java can be found in. You can run a batch of workflows at once by providing a CSV file with one row per workflow to define the parameters.

Support

Quality

Security

License

Reuse

Support

dockerflow has a low active ecosystem.

It has 95 star(s) with 16 fork(s). There are 25 watchers for this library.

It had no major release in the last 6 months.

There are 7 open issues and 20 have been closed. On average issues are closed in 25 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of dockerflow is current.

Quality

dockerflow has 11 bugs (1 blocker, 0 critical, 9 major, 1 minor) and 316 code smells.

Security

dockerflow has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

dockerflow code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

dockerflow is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

dockerflow releases are not available. You will need to build from source code and install.

Build file is available. You can build the component from source.

Installation instructions, examples and code snippets are available.

dockerflow saves you 2925 person hours of effort in developing the same functionality from scratch.

It has 6316 lines of code, 502 functions and 40 files.

It has high code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed dockerflow and discovered the below as its top functions. This is intended to give you an instant insight into dockerflow implemented functionality, and help decide if they suit your requirements.

Create a new task
Declares an input parameter for a path in a GCS container
Declares an output parameter for execution
Declares an array of input files
Runs the tool with the given options
Resolves a local path
Load a workflow definition
Deletes an element
Performs an HTTPS DELETE request
Replaces the variables with the default value
Checks if a path exists
Gather task arguments
Adds parameters to the table
Set the parameters for a project
Creates a new arguments table
Sets a global variable
Set the task logging to use
Set the minimum CPU cores
Set whether this table is preemptible
Set the memory allocation
Run a single task
Run a task in Docker
Run a Dataflow on a Dataflow
Process the element
Create a dataflow pipeline
Creates a new task

Get all kandi verified functions for this library.

dockerflow Key Features

No Key Features are available at this moment for dockerflow.

dockerflow Examples and Code Snippets

No Code Snippets are available at this moment for dockerflow.

Community Discussions

Trending Discussions on dockerflow

Why are Google Pipeline VM instances hanging indefinitely?

QUESTION

Why are Google Pipeline VM instances hanging indefinitely?

Asked 2017-Feb-21 at 21:20

I am using Dockerflow to run parallel tasks through the Google Pipelines API on Google Cloud Platform. I started a single-step task running 1389 VMs in parallel and found that 233 of the VMs were apparently doing nothing and hanging indefinitely.

I did a spot check of the serial console output and repeatedly saw the VMs running into "Getting controller config failed" errors.

When I tried logging into the VMs I received the error: "Connection Failed. We are unable to connect to the VM on port 22".

I am wondering why my VM instances are hanging, and if there is something I can do to avoid running into these issues.

I've included a snippet of the serial console output below

...

ANSWER

Answered 2017-Feb-15 at 20:13

there may have been a temporary networking issue. Can you give me some failed operation ids (or failed VM names)?

Have you tried again since then; can you reproduce the problem?

Source https://stackoverflow.com/questions/42171101

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install dockerflow

Run the following steps on your laptop or local workstation:.
git clone this repository. git clone https://github.com/googlegenomics/dockerflow
Build it with Maven. cd dockerflow mvn package -DskipTests
Set up the DOCKERFLOW_HOME environment. export DOCKERFLOW_HOME="$(pwd)" export PATH="${PATH}":"${DOCKERFLOW_HOME}/bin" chmod +x bin/*
Run a sample workflow: dockerflow --project=MY-PROJECT \ --workflow-file=src/test/resources/linear-graph.yaml \ --workspace=gs://MY-BUCKET/MY-PATH \ --input BASE_DIR=gs://MY-BUCKET/MY-PATH/MY-INPUT-FILE.txt --runner=DirectPipelineRunner

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: