kandi background
Explore Kits

dolphinscheduler | Apache DolphinScheduler | Job Scheduling library

 by   apache Java Version: 3.0.0-alpha License: Apache-2.0

 by   apache Java Version: 3.0.0-alpha License: Apache-2.0

Download this library from

kandi X-RAY | dolphinscheduler Summary

dolphinscheduler is a Java library typically used in Data Processing, Job Scheduling applications. dolphinscheduler has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. You can download it from GitHub, Maven.
Dolphin Scheduler Official Website [dolphinscheduler.apache.org](https://dolphinscheduler.apache.org).
Support
Support
Quality
Quality
Security
Security
License
License
Reuse
Reuse

kandi-support Support

  • dolphinscheduler has a medium active ecosystem.
  • It has 7656 star(s) with 2901 fork(s). There are 310 watchers for this library.
  • There were 2 major release(s) in the last 6 months.
  • There are 660 open issues and 3668 have been closed. On average issues are closed in 71 days. There are 60 open pull requests and 0 closed requests.
  • It has a neutral sentiment in the developer community.
  • The latest version of dolphinscheduler is 3.0.0-alpha
dolphinscheduler Support
Best in #Job Scheduling
Average in #Job Scheduling
dolphinscheduler Support
Best in #Job Scheduling
Average in #Job Scheduling

quality kandi Quality

  • dolphinscheduler has no bugs reported.
dolphinscheduler Quality
Best in #Job Scheduling
Average in #Job Scheduling
dolphinscheduler Quality
Best in #Job Scheduling
Average in #Job Scheduling

securitySecurity

  • dolphinscheduler has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
dolphinscheduler Security
Best in #Job Scheduling
Average in #Job Scheduling
dolphinscheduler Security
Best in #Job Scheduling
Average in #Job Scheduling

license License

  • dolphinscheduler is licensed under the Apache-2.0 License. This license is Permissive.
  • Permissive licenses have the least restrictions, and you can use them in most projects.
dolphinscheduler License
Best in #Job Scheduling
Average in #Job Scheduling
dolphinscheduler License
Best in #Job Scheduling
Average in #Job Scheduling

buildReuse

  • dolphinscheduler releases are available to install and integrate.
  • Deployable package is available in Maven.
  • Build file is available. You can build the component from source.
  • Installation instructions are not available. Examples and code snippets are available.
dolphinscheduler Reuse
Best in #Job Scheduling
Average in #Job Scheduling
dolphinscheduler Reuse
Best in #Job Scheduling
Average in #Job Scheduling
Top functions reviewed by kandi - BETA

kandi has reviewed dolphinscheduler and discovered the below as its top functions. This is intended to give you an instant insight into dolphinscheduler implemented functionality, and help decide if they suit your requirements.

  • construct process instance
  • Update a user .
  • Check and import process definitions .
  • Run the command .
  • Splits a process definition into two tasks .
  • Create complement command list .
  • Add a job .
  • Build the content to JSON .
  • Update alert result handler .
  • Creates the command line arguments for Spark .

dolphinscheduler Key Features

Associate the tasks according to the dependencies of the tasks in a DAG graph, which can visualize the running state of the task in real-time.

Support various task types: Shell, MR, Spark, SQL (MySQL, PostgreSQL, hive, spark SQL), Python, Sub_Process, Procedure, etc.

Support scheduling of workflows and dependencies, manual scheduling to pause/stop/recover task, support failure task retry/alarm, recover specified nodes from failure, kill task, etc.

Support the priority of workflows & tasks, task failover, and task timeout alarm or failure.

Support workflow global parameters and node customized parameter settings.

Support online upload/download/management of resource files, etc. Support online file creation and editing.

Support task log online viewing and scrolling and downloading, etc.

Have implemented cluster HA, decentralize Master cluster and Worker cluster through Zookeeper.

Support the viewing of Master/Worker CPU load, memory, and CPU usage metrics.

Support displaying workflow history in tree/Gantt chart, as well as statistical analysis on the task status & process status in each workflow.

Support back-filling data.

Support multi-tenant.

Support internationalization.

More features waiting for partners to explore…​

Submit an [issue](https://github.com/apache/dolphinscheduler/issues/new/choose)

Subscribe to this mailing list: https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html, then email dev@dolphinscheduler.apache.org

default

copy iconCopydownload iconDownload
 Stability | Accessibility | Features | Scalability |
 --------- | ------------- | -------- | ------------|
Decentralized multi-master and multi-worker | Visualization of workflow key information, such as task status, task type, retry times, task operation machine information, visual variables, and so on at a glance.  |  Support pause, recover operation | Support customized task types
support HA | Visualization of all workflow operations, dragging tasks to draw DAGs, configuring data sources and resources. At the same time, for third-party systems, provide API mode operations. | Users on DolphinScheduler can achieve many-to-one or one-to-one mapping relationship through tenants and Hadoop users, which is very important for scheduling large data jobs.  | The scheduler supports distributed scheduling, and the overall scheduling capability will increase linearly with the scale of the cluster. Master and Worker support dynamic adjustment.
Overload processing: By using the task queue mechanism, the number of schedulable tasks on a single machine can be flexibly configured. Machine jam can be avoided with high tolerance to numbers of tasks cached in task queue. | One-click deployment | Support traditional shell tasks, and big data platform task scheduling: MR, Spark, SQL (MySQL, PostgreSQL, hive, spark SQL), Python, Procedure, Sub_Process |  |

Zookeeper management interface in monitoring center displays abnormal

copy iconCopydownload iconDownload
4lw.commands.whitelist=*

Community Discussions

Trending Discussions on dolphinscheduler
  • How to use apache-dolphinscheduler's queue?
  • Zookeeper management interface in monitoring center displays abnormal
Trending Discussions on dolphinscheduler

QUESTION

How to use apache-dolphinscheduler's queue?

Asked 2021-May-22 at 14:50

I am confused about the apache-dolphinscheduler's queue, as in user-guide, the queue is used for spark、mapreduce. But I want to use python code product seeds to queue and another python code in workers pull seeds from queue and run tasks. Can you tell me whether dolphinscheduler can handle it or I must use another tools, such as Redis? Thanks.

ANSWER

Answered 2021-May-22 at 14:50

as what you said, you need to use another tools. the queue designed to choose corresponding queue which exists in hadoop yarn cluster.

Source https://stackoverflow.com/questions/67604598

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install dolphinscheduler

You can download it from GitHub, Maven.
You can use dolphinscheduler like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the dolphinscheduler component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

DOWNLOAD this Library from

Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items
Find more libraries
Reuse Solution Kits and Libraries Curated by Popular Use Cases

Save this library and start creating your kit

Explore Related Topics

Share this Page

share link
Compare Job Scheduling Libraries with Highest Support
Compare Job Scheduling Libraries with Highest Reuse
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items
Find more libraries
Reuse Solution Kits and Libraries Curated by Popular Use Cases

Save this library and start creating your kit

  • © 2022 Open Weaver Inc.