gs-batch-processing | Batch Service : : Learn how to create a basic batch | BPM library
kandi X-RAY | gs-batch-processing Summary
Support
Quality
Security
License
Reuse
- bean data reader .
- Verify that a BatchJob completes .
- Converts the person into a person object .
- Import user job .
- String representation .
- Entry point for the BatchProcessing application .
- Set the first name .
gs-batch-processing Key Features
gs-batch-processing Examples and Code Snippets
Trending Discussions on gs-batch-processing
Trending Discussions on gs-batch-processing
QUESTION
I've created small hello world project for spring batch:
build.gradle:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:2.1.6.RELEASE")
}
}
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
bootJar {
baseName = 'gs-batch-processing'
version = '0.1.0'
}
repositories {
mavenCentral()
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
dependencies {
compile("org.springframework.boot:spring-boot-starter-batch")
compile("org.postgresql:postgresql")
compile("org.springframework.boot:spring-boot-starter-data-jpa")
//to fix exception on startup
//compile('org.hibernate:hibernate-core:5.4.2.Final')
testCompile("junit:junit")
}
configuration:
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
private DbPersonWriter dbPersonWriter;
@Autowired
private ToLowerCasePersonProcessor toLowerCasePersonProcessor;
@Value("${app.users-location}")
Resource csvResource;
@Bean
public Job job() {
return jobBuilderFactory.get("myJob")
.incrementer(new RunIdIncrementer())
.flow(csvToDataBaseStep())
.end()
.build();
}
private Step csvToDataBaseStep() {
return stepBuilderFactory.get("csvToDatabaseStep")
.chunk(100)
.reader(csvPersonReader())
.processor(toLowerCasePersonProcessor)
.writer(dbPersonWriter)
.build();
}
on startup I see following exception:
java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111]
at org.hibernate.engine.jdbc.env.internal.LobCreatorBuilderImpl.useContextualLobCreation(LobCreatorBuilderImpl.java:113) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.engine.jdbc.env.internal.LobCreatorBuilderImpl.makeLobCreatorBuilder(LobCreatorBuilderImpl.java:54) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentImpl.(JdbcEnvironmentImpl.java:271) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:114) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:94) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.(InFlightMetadataCollectorImpl.java:179) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:119) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:904) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:935) [hibernate-core-5.3.10.Final.jar:5.3.10.Final]
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:57) [spring-orm-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365) [spring-orm-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:390) [spring-orm-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:377) [spring-orm-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341) [spring-orm-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1105) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at spring.boot.hello.world.MyApplication.main(MyApplication.java:9) ~[main/:na]
Caused by: java.sql.SQLFeatureNotSupportedException: Method org.postgresql.jdbc.PgConnection.createClob() is not yet implemented.
at org.postgresql.Driver.notImplemented(Driver.java:688) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.jdbc.PgConnection.createClob(PgConnection.java:1269) ~[postgresql-42.2.5.jar:42.2.5]
... 43 common frames omitted
I have to say that everything is working as expected and this trace just litter my logs. But I want to avoid it.
I've found workaround to fix it( thx this topic How to resolve org.postgresql.jdbc.PgConnection.createClob() is not yet implemented):
just added dependency and excption disappeared:
compile('org.hibernate:hibernate-core:5.4.2.Final')
Is there more elegant way? I don't think that workaround is a good solution
P.S.To play with source code you can follow my github repo
ANSWER
Answered 2019-Aug-07 at 16:15This was a Hibernate issue. If you are using Spring Boot latest version from 2.0.x
to 2.1.x
includes Hibernate 5.3.10.final
you can take a look here but this issue was fixed on Hibernate version
5.4.0.CR1 then you need to add that dependency or if it is possible the latest version:
For Gradle:
compile('org.hibernate:hibernate-core:5.4.2.Final')
For Maven:
org.hibernate
hibernate-core
5.4.2.Final
UPDATED "SPRING BOOT 2.2.0.M(1-4)"
In addition Spring boot v2.2.0.Mx
includes now Hibernate v5.4.x
then this issue was fixed for these versions.
QUESTION
I checkout this project from spring: https://github.com/spring-guides/gs-batch-processing
Source: https://spring.io/guides/gs/batch-processing/
I replace the ',' with 'tab' in 'sample-data.csv:
Jill Doe
Joe Doe
Justin Doe
Jane Doe
John Doe
Then I add the new Delimiter to the reader:
@Bean
public FlatFileItemReader reader() {
return new FlatFileItemReaderBuilder()
.name("personItemReader")
.resource(new ClassPathResource("sample-data.csv"))
.delimited()
.delimiter(DelimitedLineTokenizer.DELIMITER_TAB) // NEW DELIMITER
.names(new String[]{"firstName", "lastName"})
.fieldSetMapper(new BeanWrapperFieldSetMapper() {{
setTargetType(Person.class);
}})
.build();
}
When I launch I get this error:
Caused by: org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 2 actual 1
at org.springframework.batch.item.file.transform.AbstractLineTokenizer.tokenize(AbstractLineTokenizer.java:142) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:43) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:180) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
... 50 common frames omitted
I have tried with '@' delimiter -> it works. For some reason, I can't make it work with the tab delimiter...
Of course in my real project, I have an input file with 'tab' separators...
Any solution here?
ANSWER
Answered 2018-Aug-08 at 07:55You cant't set the tab delimiter that way. Since tab ('\t') doesnt contain any actual text it is ignored by the DelimitedLineTokenizer
in static DelimitedBuilder
class in FlatFileItemReaderBuilder.java
. Any non-whitespace delimiter can be set using above code that you have given in the question.
FlatFileItemReaderBuilder sourceCode
This is how the LineTokenizer
instance is built in FlatFileItemReaderBuilder.java
.
public DelimitedLineTokenizer build() {
Assert.notNull(this.fieldSetFactory, "A FieldSetFactory is required.");
Assert.notEmpty(this.names, "A list of field names is required");
DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
tokenizer.setNames(this.names.toArray(new String[this.names.size()]));
// the hasText ignores the tab delimiter.
if(StringUtils.hasText(this.delimiter)) {
tokenizer.setDelimiter(this.delimiter);
}
// more code
So to fix this issue, you need to provide bean of Type DelimitedLineTokenizer explicitly configured with tab delimiter.
use below code in your spring configuration file to set the tab delimiter:
@Bean
public FlatFileItemReader reader() {
return new FlatFileItemReaderBuilder().name("personItemReader")
.resource(new ClassPathResource("sample-data.csv"))
.lineMapper(lineMapper()).build();
}
@Bean
public DefaultLineMapper lineMapper(){
DefaultLineMapper lineMapper = new DefaultLineMapper<>();
lineMapper.setLineTokenizer(lineTokenizer());
lineMapper.setFieldSetMapper(new BeanWrapperFieldSetMapper() {
{
setTargetType(Person.class);
}
});
return lineMapper;
}
@Bean
public DelimitedLineTokenizer lineTokenizer() {
DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer(DelimitedLineTokenizer.DELIMITER_TAB);
tokenizer.setNames(new String[] { "firstName", "lastName" });
return tokenizer;
}
QUESTION
[New to SpringBatch] Using Spring Boot, I am trying to create a job which reads names from MongoDB, converts to lowercase, and outputs to CSV file. My reader and processor are working but the writer isn't.
My code is as follows.
Configuration file:
package bbye;
import java.util.HashMap;
import java.util.Map;
import javax.sql.DataSource;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.data.MongoItemReader;
import org.springframework.batch.item.data.builder.MongoItemReaderBuilder;
import org.springframework.batch.item.file.FlatFileItemWriter;
import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.DelimitedLineAggregator;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.batch.item.file.transform.LineAggregator;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.domain.Sort;
import org.springframework.data.mongodb.core.MongoTemplate;
import hello.Person;
@Configuration
@EnableBatchProcessing
public class BatchConfigProcessing {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
private MongoTemplate mongoTemplate;
private String readQuery = "{}";
// tag::readerwriterprocessor[]
@Bean
public MongoItemReader readMongo(DataSource dataSource) {
return new MongoItemReaderBuilder()
.name("mongoDocReader")
.jsonQuery(readQuery)
.targetType(Person.class)
.sorts(sort())
.template(mongoTemplate)
.collection("people")
.build();
}
@Bean
public PersonDocProcessor processor() {
return new PersonDocProcessor();
}
@Bean
public FlatFileItemWriter writer() {
/*FlatFileItemWriterBuilder writePerson = new FlatFileItemWriterBuilder();
writePerson.name("personDocWriter");
writePerson.resource(new ClassPathResource("PersonExtracted.csv"));
writePerson.lineAggregator(new DelimitedLineAggregator());
writePerson.shouldDeleteIfExists(true);
writePerson.build();*/
FlatFileItemWriter fileWriter = new FlatFileItemWriter<>();
fileWriter.setName("csvWriter");
fileWriter.setResource(new ClassPathResource("PersonExtracted.csv"));
fileWriter.setLineAggregator(lineAggregator());
fileWriter.setForceSync(true);
fileWriter.close();
return fileWriter;
}
// end::readerwriterprocessor[]
// tag::jobstep[]
@Bean
public Job exportUserJob(FileUploadNotificationListener listener, Step step1) {
return jobBuilderFactory.get("exportUserJob")
.incrementer(new RunIdIncrementer())
.listener(listener)
.flow(step1)
.end()
.build();
}
@Bean
public Step step2(MongoItemReader reader) {
return stepBuilderFactory.get("step2")
. chunk(10)
.reader(reader)
.processor(processor())
.writer(writer())
.build();
}
// end::jobstep[]
public FieldExtractor fieldExtractor()
{
BeanWrapperFieldExtractor extractor = new BeanWrapperFieldExtractor<>();
extractor.setNames( new String[] { "firstName",
"lastName"});
return extractor;
}
public LineAggregator lineAggregator() {
DelimitedLineAggregator la = new DelimitedLineAggregator();
la.setDelimiter(",");
la.setFieldExtractor(fieldExtractor());
return la;
}
public Map sort(){
String firstName = "firstName";
Map sortMap = new HashMap();
sortMap.put(firstName, Sort.DEFAULT_DIRECTION);
return sortMap;
}
}
Processor file
package bbye;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.stereotype.Component;
import hello.Person;
@Component
public class PersonDocProcessor implements ItemProcessor {
private static final Logger log = LoggerFactory.getLogger(PersonDocProcessor.class);
@Override
public Person process(final Person person) throws Exception {
final String firstName = person.getFirstName().toLowerCase();
final String lastName = person.getLastName().toLowerCase();
final Person transformedPerson = new Person(firstName, lastName);
log.info("Converting (" + person + ") into (" + transformedPerson + ")");
return transformedPerson;
}
}
Listener
package bbye;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.stereotype.Component;
@Component
public class FileUploadNotificationListener implements JobExecutionListener {
@Override
public void beforeJob(JobExecution jobExecution) {
System.out.println("===== listening for job - mongoReader - fileWriter ====");
}
@Override
public void afterJob(JobExecution jobExecution) {
System.out.println("==== file write job completed =====");
}
}
Here Person is a simple POJO. The stack trace with and without manual file creation is as follows:
- If the file is not present under src/main/resources FlatFileItemWriter does not create a new file and throws
org.springframework.batch.item.ItemStreamException: Could not convert resource to file: [class path resource [PersonExtracted.csv]]
at org.springframework.batch.item.file.FlatFileItemWriter.getOutputState(FlatFileItemWriter.java:399) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.file.FlatFileItemWriter.open(FlatFileItemWriter.java:337) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
........
Caused by: java.io.FileNotFoundException: class path resource [PersonExtracted.csv] cannot be resolved to URL because it does not exist
- If I create the PersonExtracted.csv file manually, the program runs without errors but does not write anything to the file. In fact, a blank file is returned. The stack trace is as below.
:: Spring Boot :: (v2.0.2.RELEASE)
2018-06-19 11:35:17.663 INFO 25136 --- [ main] hello.Application : Starting Application on MyPC with PID 25136 (C:\eclipse-workspace\gs-batch-processing-master\complete\target\classes started by shristi in C:\eclipse-workspace\gs-batch-processing-master\complete)
2018-06-19 11:35:17.666 INFO 25136 --- [ main] hello.Application : No active profile set, falling back to default profiles: default
2018-06-19 11:35:17.689 INFO 25136 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@15bb6bea: startup date [Tue Jun 19 11:35:17 EDT 2018]; root of context hierarchy
2018-06-19 11:35:18.135 INFO 25136 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2018-06-19 11:35:18.136 WARN 25136 --- [ main] com.zaxxer.hikari.util.DriverDataSource : Registered driver with driverClassName=org.hsqldb.jdbcDriver was not found, trying direct instantiation.
2018-06-19 11:35:18.282 INFO 25136 --- [ main] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Driver does not support get/set network timeout for connections. (feature not supported)
2018-06-19 11:35:18.284 INFO 25136 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2018-06-19 11:35:18.293 INFO 25136 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from URL [file:/C:/eclipse-workspace/gs-batch-processing-master/complete/target/classes/schema-all.sql]
2018-06-19 11:35:18.297 INFO 25136 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from URL [file:/C:/eclipse-workspace/gs-batch-processing-master/complete/target/classes/schema-all.sql] in 4 ms.
2018-06-19 11:35:18.518 INFO 25136 --- [ main] org.mongodb.driver.cluster : Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
2018-06-19 11:35:18.552 INFO 25136 --- [localhost:27017] org.mongodb.driver.connection : Opened connection [connectionId{localValue:1, serverValue:140}] to localhost:27017
2018-06-19 11:35:18.554 INFO 25136 --- [localhost:27017] org.mongodb.driver.cluster : Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 0]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=1438717}
2018-06-19 11:35:18.723 INFO 25136 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: HSQL
2018-06-19 11:35:18.770 INFO 25136 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2018-06-19 11:35:18.778 INFO 25136 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql]
2018-06-19 11:35:18.781 INFO 25136 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql] in 3 ms.
2018-06-19 11:35:18.870 INFO 25136 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2018-06-19 11:35:18.871 INFO 25136 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Bean with name 'dataSource' has been autodetected for JMX exposure
2018-06-19 11:35:18.873 INFO 25136 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Located MBean 'dataSource': registering with JMX server as MBean [com.zaxxer.hikari:name=dataSource,type=HikariDataSource]
2018-06-19 11:35:18.880 INFO 25136 --- [ main] hello.Application : Started Application in 1.357 seconds (JVM running for 2.284)
2018-06-19 11:35:18.881 INFO 25136 --- [ main] o.s.b.a.b.JobLauncherCommandLineRunner : Running default command line with: []
2018-06-19 11:35:18.908 INFO 25136 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportUserJob]] launched with the following parameters: [{run.id=1}]
===== listening for job - mongoReader - fileWriter ====
2018-06-19 11:35:18.917 INFO 25136 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
2018-06-19 11:35:18.995 INFO 25136 --- [ main] org.mongodb.driver.connection : Opened connection [connectionId{localValue:2, serverValue:141}] to localhost:27017
2018-06-19 11:35:19.022 INFO 25136 --- [ main] bbye.PersonDocProcessor : Converting (firstName: ALICE, lastName: WONDERLAND) into (firstName: alice, lastName: wonderland)
2018-06-19 11:35:19.022 INFO 25136 --- [ main] bbye.PersonDocProcessor : Converting (firstName: FIRSTNAME, lastName: LASTNAME) into (firstName: firstname, lastName: lastname)
2018-06-19 11:35:19.022 INFO 25136 --- [ main] bbye.PersonDocProcessor : Converting (firstName: JANE, lastName: DOE) into (firstName: jane, lastName: doe)
2018-06-19 11:35:19.022 INFO 25136 --- [ main] bbye.PersonDocProcessor : Converting (firstName: JOHN, lastName: DOE) into (firstName: john, lastName: doe)
2018-06-19 11:35:19.022 INFO 25136 --- [ main] bbye.PersonDocProcessor : Converting (firstName: MARK, lastName: WINN) into (firstName: mark, lastName: winn)
==== file write job completed =====
2018-06-19 11:35:19.031 INFO 25136 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportUserJob]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED]
2018-06-19 11:35:19.032 INFO 25136 --- [ Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@15bb6bea: startup date [Tue Jun 19 11:35:17 EDT 2018]; root of context hierarchy
2018-06-19 11:35:19.033 INFO 25136 --- [ Thread-2] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
2018-06-19 11:35:19.034 INFO 25136 --- [ Thread-2] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans
2018-06-19 11:35:19.035 INFO 25136 --- [ Thread-2] org.mongodb.driver.connection : Closed connection [connectionId{localValue:2, serverValue:141}] to localhost:27017 because the pool has been closed.
2018-06-19 11:35:19.036 INFO 25136 --- [ Thread-2] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2018-06-19 11:35:19.037 INFO 25136 --- [ Thread-2] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
ANSWER
Answered 2018-Jun-25 at 15:04I think we should use FileSystemResource instead of ClassPathResource. Could you please try and let us know.
QUESTION
I am using a batch process and i want it to run at a specific cron scheduled time. However, the job is getting triggered at the start up and again triggering at the scheduled time. I am trying to avoid the former however failing to do so. This is a sample repository which reproduces the same issue: https://github.com/ppanigrahi02/BatchJobWithScheduler. i am using the spring guide example https://github.com/spring-guides/gs-batch-processing and added a scheduler on top of it. I will really appreciate any leads.
ANSWER
Answered 2018-Feb-08 at 11:42The @Scheduled
annotation used here will start the job immediately at startup and every 600000 milliseconds. If you want to specify an initial delay to wait before the first execution, you can use the initialDelay
attribute of the annotation, something like:
@Scheduled(initialDelay = 600000, fixedRate = 600000)
Another option is to use the cron
attribute and provide a cron expression. For more details about this annotation, you can check the reference documentation here: https://docs.spring.io/spring/docs/5.0.3.RELEASE/spring-framework-reference/integration.html#scheduling-annotation-support-scheduled
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install gs-batch-processing
You can use gs-batch-processing like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the gs-batch-processing component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .
Support
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesExplore Kits - Develop, implement, customize Projects, Custom Functions and Applications with kandi kits
Save this library and start creating your kit
Share this Page