spring-cloud-task | spring-cloud-task demo | Microservice library

I think you need to drop the .java from this line in your pom.xml:

The job parameters are passed as Arguments where you don't need to specify the -- prefix. For instance, in this case you need to specify year=2011 as argument in Arguments section.

You need to add your task application properties into an application metadata that Spring Cloud Data Flow knows how to extract and make them available.

You can refer this page that walks you through how to add metadata.

Also, you can check this sample configuration as a reference on how to add such metadata.

Spring Cloud Task has a broader scope than Spring Batch. It is designed for any short lived task, including but not limited to (Spring) Batch jobs. A short lived task could be a Java process, a shell script, a Docker image, etc. Spring Cloud Task has its own meta-data tables to track the progress/status/stats of tasks.

In the context of Spring Batch, Spring Cloud Task provides a number of additional features:

• Batch informational messages: ability to emit messages based on Spring Batch listeners events. Those messages can be consumed by streaming apps and makes it possible to bridge tasks and streaming apps.
• DeployerPartitionHandler: an additional partition handler that is suitable to cloud environments to dynamically deploy workers in a remote partitioning setup.

To resolved this issue, I passed the unique key:value in the Task Argument and I so keeping passing the multiple key:value combinations, then I can run the same Task many times.

However I wanted to do this programmatically. Any quick solution ?

The task instance restriction is a feature of Spring Cloud Task (your applications), not Spring Cloud Data Flow. Because of this, what you are seeing is expected behavior. You can open up an issue with the Spring Cloud Data Flow project for us to discuss adding that feature at the Spring Cloud Data Flow level.

Please see my response as below:

My current understanding is that both of these projects are under Spring Cloud Dataflow, and serve as components of the pipeline.

Both Spring Cloud Stream and Spring Cloud Task are not under Spring Cloud Data Flow, instead, they can be used as standalone projects and Spring Cloud Data Flow just uses them.

Spring Cloud Stream lets you bind your event-driven long-running applications into a messaging middleware or a streaming platform. As a developer, you have to choose your binder (the binder implementations for RabbitMQ, Apache Kafka etc.,) to stream your events or data from/to the messaging middleware you bind to.

Spring Cloud Task doesn't bind your application into a messaging middleware. Instead, it provides abstractions and lifecycle management to run your ephemeral or finite duration applications (tasks). It also provides the foundation for developing Spring Batch applications.

However, both can be made recurring (a stream is by definition recurring, where a task can run every certain time interval)

A task application can be triggered/scheduled to make it a recurring one whereas the streaming application is a long-running, not a recurring one.

In addition, both can be configured to communicate with the rest of the pipeline through the message broker.

Though a task application can be configured to communicate to a messaging middleware, the concept of pipeline is different when it comes to stream vs task (batch). For the streaming applications, the pipeline refers to the communication via the messaging middleware while for the task applications, the concept of composed tasks lets you create a conditional workflow of multiple task applications. For more information on composed tasks, you can refer to the documentation.

If you have the org.springframework.cloud:spring-cloud-task-batch dependency, and the annotation @EnableTask is present, then your application context contains a TaskBatchExecutionListener bean that you can inject into your class that dynamically creates the jobs and tasklets.

That might look similar to this:

Since you have @EnableTask, spring will close the context once everything is done running. From the code it looks like you are not running anything explicitly so spring is closing the context before your @Schedule() annotation kicks in.

The fix for that is to tell spring to not close the context at all with spring.cloud.task.closecontext_enable=false. This will keep the context open for your scheduled task. Some documentation here: https://docs.spring.io/spring-cloud-task/docs/1.2.2.RELEASE/reference/htmlsingle/#features-lifecycle

One more note about the property. In the documentation it says closecontext_enable but after inspecting the logs and the jar, that property has been deprecated and replaced by close_context_enabled.