spring-cloud-task | Short lived microservices with Spring Batch | Function As A Service library
kandi X-RAY | spring-cloud-task Summary
kandi X-RAY | spring-cloud-task Summary
Is a project centered around the idea of processing on demand. A user is able to develop a “task” that can be deployed, executed and removed on demand, yet the result of the process persists beyond the life of the task for future reporting.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Executes a job
- Merge job parameters
- Returns a copy of JobParameters with non identifiers
- Throw a TaskException
- Lock a task
- Creates the default lock registry
- Runs the execution
- Validate the request
- Get DAO
- Checks if is SQL server sequence available
- The thread pool executor
- Initializes this instance
- Compares two TaskLaunchRequest objects
- Registers job execution events
- DS batch item writer
- Returns the hashCode of this instance
- Retrieves all job execution ids for the given task execution id
- Convert a range into a Range
- Returns the platform transaction manager
- Bean batch job item reader
- Process a step
- Process a task launch request
- Create the partition handler
- The batch item writer
- Register the hints
- Initializes the platform
spring-cloud-task Key Features
spring-cloud-task Examples and Code Snippets
Community Discussions
Trending Discussions on spring-cloud-task
QUESTION
Latest Update (with an image to hope simplify the problem) (thanks for feedback from @Mahmoud)
Relate issue reports for other reference (after this original post created, it seem someone filed issues for Spring Cloud on similar issue, so also update there too):
https://github.com/spring-cloud/spring-cloud-task/issues/793 relate to approach #1
https://github.com/spring-cloud/spring-cloud-task/issues/792 relate to approach #2
Also find a workaround resolution for that issue and update on that github issue, will update this once it is confirmed good by developer https://github.com/spring-cloud/spring-cloud-task/issues/793#issuecomment-894617929
I am developing an application involved multi-steps using spring batch job but hit some roadblock. Did try to research doc and different attempts, but no success. So thought to check if community can shed light
Spring batch job 1 (received job parameter for setting for step 1/setting for step 2)
...ANSWER
Answered 2021-Aug-15 at 13:33
- Is above even possible setup?
yes, nothing prevents you from having two partitioned steps in a single Spring Batch job.
- Is it possible to use JobScope/StepScope to pass info to the partitionhandler
yes, it is possible for the partition handler to be declared as a job/step scoped bean if it needs the late-binding feature to be configured.
Updated on 08/14/2021 by @DanilKo
The original answer is correct in high - level. However, to actually achieve the partition handeler to be step scoped, a code modification is required
Below is the analyze + my proposed workaround/fix (maybe eventually code maintainer will have better way to make it work, but so far below fix is working for me)
Issue being continued to discuss at: https://github.com/spring-cloud/spring-cloud-task/issues/793 (multiple partitioner handler discussion) https://github.com/spring-cloud/spring-cloud-task/issues/792 (which this fix is based up to use partitionerhandler at step scope to configure different worker steps + resources + max worker)
Root cause analyze (hypothesis)The problem is DeployerPartitionHandler utilize annoation @BeforeTask to force task to pass in TaskExecution object as part of Task setup
But as this partionerHandler is now at @StepScope (instead of directly at @Bean level with @Enable Task) or there are two partitionHandler, that setup is no longer triggered, as @EnableTask seem not able to locate one partitionhandler during creation.
Resulted created DeployerHandler faced a null with taskExecution when trying to launch (as it is never setup)
Below is essentially a workaround to use the current job execution id to retrieve the associated task execution id From there, got that task execution and passed to deploy handler to fulfill its need of taskExecution reference It seem to work, but still not clear if there is other side effect (so far during test not found any)
Full code can be found in https://github.com/danilko/spring-batch-remote-k8s-paritition-example/tree/attempt_2_partitionhandler_with_stepscope_workaround_resolution
In the partitionHandler method
QUESTION
TL;DR
Spring Cloud Data Flow does not allow multiple executions of the same Task even though the documentation says that this is the default behavior. How can we allow SCDF to run multiple instances of the same task at the same time using the Java DSL to launch tasks? To make things more interesting, launching of the same task multiple times works fine when directly hitting the rest enpoints using curl for example.
Background :
I have a Spring Cloud Data Flow Task that I have pre-registered in the Spring Cloud Data Flow UI Dashboard
...ANSWER
Answered 2021-May-12 at 16:57In this case it looks like you are trying to recreate the task definition. You should only need to create the task definition once. From this definition you can launch multiple times. For example:
QUESTION
I'm trying out a simple Spring Cloud Task application. I'm currently trying to run the said application but somehow I'm ending up with a java.lang.ClassNotFoundException: com.example.sampletask.demotask.DemoTaskApplication.java. I'm trying to tweak the pom.xml file but I'm getting no where so far. Here's what I have as of now:
The application class:
...ANSWER
Answered 2020-Sep-25 at 19:18I think you need to drop the .java from this line in your pom.xml:
QUESTION
There is Data Flow deployed in Docker and a Spring Batch application is deployed as a 'Task' and turned into a task.
I am trying to provide a year job parameter for my task. I have tried using a properties class with @ConfigurationProperties annotation like in timestamp example. Then I turn this into job parameter via JobParametersIncrementer.
ANSWER
Answered 2020-Aug-16 at 02:27The job parameters are passed as Arguments where you don't need to specify the -- prefix. For instance, in this case you need to specify year=2011 as argument in Arguments section.
QUESTION
I'm developing a task to use Spring Cloud Data Flow, but I can't see the execution parameters of my task (which is a Spring Boot application) or how to execute the task.
I am importing my application as a base and as the project as a base: spring-cloud-starter-task-timestamp
This is my pom, I made this using Spring Initializr and then using the timestamp-task application for tweaking.
...ANSWER
Answered 2020-Jun-25 at 04:56You need to add your task application properties into an application metadata that Spring Cloud Data Flow knows how to extract and make them available.
You can refer this page that walks you through how to add metadata.
Also, you can check this sample configuration as a reference on how to add such metadata.
QUESTION
I went through the Introducing Spring Cloud Task, but things are not clear for the following questions.
I'm using Spring Batch
1) What's the use of Spring Cloud Task when we already have the metadata provided by Spring Batch ?
2) We're planning to use Spring Cloud Data Flow to monitor the Spring Batch. All the batch jobs can be imported into the SCDF as task and can be sceduled there, but don't see support for MongoDB. Hope MySQL works well.
What is the difference between Spring Cloud Task and Spring Batch?
...ANSWER
Answered 2020-May-04 at 10:40Spring Cloud Task has a broader scope than Spring Batch. It is designed for any short lived task, including but not limited to (Spring) Batch jobs. A short lived task could be a Java process, a shell script, a Docker image, etc. Spring Cloud Task has its own meta-data tables to track the progress/status/stats of tasks.
In the context of Spring Batch, Spring Cloud Task provides a number of additional features:
- Batch informational messages: ability to emit messages based on Spring Batch listeners events. Those messages can be consumed by streaming apps and makes it possible to bridge tasks and streaming apps.
- DeployerPartitionHandler: an additional partition handler that is suitable to cloud environments to dynamically deploy workers in a remote partitioning setup.
QUESTION
I am working on Spring Cloud Data Flow and Spring Batch by taking a reference from https://github.com/spring-cloud/spring-cloud-task/tree/master/spring-cloud-task-samples.
I'm executing the batch-job and when executed this example two times, on 2nd time I observed the error, however for the first time it worked fine.
I started the spring-cloud-dataflow-server-local server using below commands and it created all metadata for me- highlighted in yellow.
Error:
...ANSWER
Answered 2020-Apr-18 at 15:50To resolved this issue, I passed the unique key:value in the Task Argument and I so keeping passing the multiple key:value combinations, then I can run the same Task many times.
However I wanted to do this programmatically. Any quick solution ?
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install spring-cloud-task
You can use spring-cloud-task like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the spring-cloud-task component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
