batchconfig | Create custom Windows batch files from a configuration file

The BeanWrapperFieldSetMapper uses reflection to map fields, so their declaration order in your class should not matter.

The order of fields that you declare in the array parameter in .names() corresponds to the order of columns in the input file, not to the declaration order in the POJO.

EDIT: Add sample

persons.csv

For exactly this sort of reason, splitting the initialization of your database between Flyway and Spring Batch's scripts is not recommended:

If you are using a Higher-level Database Migration Tool, like Flyway or Liquibase, you should use them alone to create and initialize the schema.

There has never been a guarantee that Flyway will run first. In the case of your application it was running first by chance and something has changed in Spring Boot 2.5.3 that means that's no longer the case.

Rather than using multiple different database initializers, I would recommend setting spring.batch.jdbc.initialize-schema=never and adding a Flyway migration script to initialize Spring Batch's schema instead.

Alternatively, you can set spring.flyway.baseline-on-migrate=true. This will cause Flyway to call baseline() automatically when it detects the non-empty schema.

The problem is that the BeanWrapperFieldSetMapper does not know, by default, how to convert a String like 2021-06-22 to an object of type java.sql.Date (which is the field dateIntegration) in your domain object DAOUser. The way to tell this mapper how to deal with custom conversions is by registering a ConversionService. This conversion service should have a converter from String to java.sql.Date registered in it. Here is a quick example:

I do not understand the reason for making your step as a prototype-scoped bean, but I'm not able to reproduce the issue with a setup similar to what you shared (using Spring Batch v4.3.3). Here is a quick example:

This is caused by the fact that you declare in your config that you need a JobCompletionNotificationListener in following lines:

You did not share the entire stack trace to see when this error happens exactly, but it seems that the close method is called more than once. I think this is not due to a concurrency issue, as I see you are using one writer per thread in a partitioned step. So I would make this method "re-entrant" by checking if the output stream is already closed before closing it (there is no isClosed method on an output stream, so you can use a custom boolean around that).

That said, I would first confirm that the close method is called twice and if so, investigate why is that and fix the root cause.

I'm trying to share data between steps using JobExecutionContext, which works only if i keep the steps single threaded

Relying on the execution context to share data between multi-threaded steps is incorrect, because the keys will be overridden by concurrent threads. The reference documentation explicitly mentions to turn off state management in multi-threaded environment:

• Javadoc: remember to use saveState=false if used in a multi-threaded client
• Reference doc: it is not recommended to use job-scoped beans in multi-threaded or partitioned steps

That said, I don't see what key could be shared from a multi-threaded step to the next step (as threads are executed in parallel), but if you really need to do that, you should use another method like defining a shared bean that is thread safe.

The FlatFileItemWriter requires a org.springframework.core.io.Resource to write data. If the API you use does not implement this interface, it is not usable with the FlatFileItemWriter. You need to provide a Resource implementation for Azure or look for an library that implements it, like the Azure Spring Boot Starter Storage client library for Java.

You are creating a JdbcCursorItemReader instance in the read method of AllocationReader. This is not correct. The code of this method should be the implementation of the actual read operation, and not for creating an item reader.

I could not write Item reader as bean in batch config as i need to call before step in item reader to access step execution.

For this use case, you can define the reader as a step-scoped bean and inject attributes from the job execution context as needed. This is explained in the reference documentation here: Late Binding of Job and Step Attributes. In your case, the reader could be defined like this: