Part I of my Spring Batch blog ran through an example of a basic Spring Batch job. Now lets put together one that reads 200,000 rows from a flat file and inserts them into the database. The entire process took around 1 minute and 10 seconds to execute. That is pretty good time for Java-based batch processing. In all fairness I must point out that, this is relatively fast since, I am using a local HSQLDB database and there is no processing related logic being performed during the entire process.
The file is a comma separated file with format => receiptDate,memberName,checkNumber,checkDate,paymentType,depositAmount,paymentAmount,comments
The DDL for the database table:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
<span style="color: #008000; font-weight: bold;">create</span> <span style="color: #008000; font-weight: bold;">table</span> ledger ( id <span style="color: #007020;">int</span> <span style="color: #008000; font-weight: bold;">GENERATED</span> <span style="color: #008000; font-weight: bold;">BY</span> <span style="color: #008000; font-weight: bold;">DEFAULT</span> <span style="color: #008000; font-weight: bold;">AS</span> <span style="color: #008000; font-weight: bold;">IDENTITY</span> (<span style="color: #008000; font-weight: bold;">START</span> <span style="color: #008000; font-weight: bold;">WITH</span> <span style="color: #0000d0; font-weight: bold;">1</span>, <span style="color: #008000; font-weight: bold;">INCREMENT</span> <span style="color: #008000; font-weight: bold;">BY</span> <span style="color: #0000d0; font-weight: bold;">1</span>) <span style="color: #008000; font-weight: bold;">not</span> <span style="color: #008000; font-weight: bold;">null</span>, rcv_dt <span style="color: #007020;">date</span>, mbr_nm <span style="color: #007020;">VARCHAR</span>(<span style="color: #0000d0; font-weight: bold;">100</span>) <span style="color: #008000; font-weight: bold;">not</span> <span style="color: #008000; font-weight: bold;">null</span>, chk_nbr <span style="color: #007020;">VARCHAR</span>(<span style="color: #0000d0; font-weight: bold;">10</span>) <span style="color: #008000; font-weight: bold;">not</span> <span style="color: #008000; font-weight: bold;">null</span>, chk_dt <span style="color: #007020;">date</span>, pymt_typ <span style="color: #007020;">VARCHAR</span>(<span style="color: #0000d0; font-weight: bold;">50</span>) <span style="color: #008000; font-weight: bold;">not</span> <span style="color: #008000; font-weight: bold;">null</span>, dpst_amt double, pymt_amt double, comments <span style="color: #007020;">VARCHAR</span>(<span style="color: #0000d0; font-weight: bold;">100</span>), <span style="color: #008000; font-weight: bold;">constraint</span> PID <span style="color: #008000; font-weight: bold;">primary</span> <span style="color: #008000; font-weight: bold;">key</span> (id) ); |
Here is the spring application context xml file…
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
<span style="color: #507090;"><?xml version="1.0" encoding="UTF-8"?></span> <span style="color: #007000;"><beans</span> <span style="color: #0000c0;">xmlns=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/beans"</span> <span style="color: #0000c0;">xmlns:xsi=</span><span style="background-color: #fff0f0;">"http://www.w3.org/2001/XMLSchema-instance"</span> <span style="color: #0000c0;">xmlns:aop=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/aop"</span> <span style="color: #0000c0;">xmlns:tx=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/tx"</span> <span style="color: #0000c0;">xmlns:batch=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/batch"</span> <span style="color: #0000c0;">xmlns:jdbc=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/jdbc"</span> <span style="color: #0000c0;">xmlns:context=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/context"</span> <span style="color: #0000c0;">xsi:schemaLocation=</span><span style="background-color: #fff0f0;">"http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd</span> <span style="background-color: #fff0f0;"> http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd</span> <span style="background-color: #fff0f0;"> http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd</span> <span style="background-color: #fff0f0;"> http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.1.xsd </span> <span style="background-color: #fff0f0;"> http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd</span> <span style="background-color: #fff0f0;"> http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd"</span><span style="color: #007000;">></span> <span style="color: #808080;"><!-- 1) USE ANNOTATIONS TO CONFIGURE SPRING BEANS --></span> <span style="color: #007000;"><context:component-scan</span> <span style="color: #0000c0;">base-package=</span><span style="background-color: #fff0f0;">"com.batch"</span> <span style="color: #007000;">/></span> <span style="color: #808080;"><!-- 2) DATASOURCE, TRANSACTION MANAGER AND JDBC TEMPLATE --></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"dataSource"</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.jdbc.datasource.DriverManagerDataSource"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"driverClassName"</span> <span style="color: #0000c0;">value=</span><span style="background-color: #fff0f0;">"org.hsqldb.jdbcDriver"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"url"</span> <span style="color: #0000c0;">value=</span><span style="background-color: #fff0f0;">"jdbc:hsqldb:file:target/data/ledger"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"username"</span> <span style="color: #0000c0;">value=</span><span style="background-color: #fff0f0;">"SA"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"password"</span> <span style="color: #0000c0;">value=</span><span style="background-color: #fff0f0;">""</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #007000;"><jdbc:initialize-database</span> <span style="color: #0000c0;">data-source=</span><span style="background-color: #fff0f0;">"dataSource"</span><span style="color: #007000;">></span> <span style="color: #007000;"><jdbc:script</span> <span style="color: #0000c0;">location=</span><span style="background-color: #fff0f0;">"classpath:schema-ddl.sql"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></jdbc:initialize-database></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"transactionManager"</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.jdbc.datasource.DataSourceTransactionManager"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"dataSource"</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"dataSource"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #007000;"><tx:annotation-driven</span> <span style="color: #0000c0;">transaction-manager=</span><span style="background-color: #fff0f0;">"transactionManager"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"jdbcTemplate"</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.jdbc.core.JdbcTemplate"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"dataSource"</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"dataSource"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #808080;"><!-- 3) JOB REPOSITORY - WE USE IN-MEMORY REPOSITORY FOR OUR EXAMPLE --></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"jobRepository"</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"transactionManager"</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"transactionManager"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #808080;"><!-- 4) LAUNCH JOBS FROM A REPOSITORY --></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"jobLauncher"</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.batch.core.launch.support.SimpleJobLauncher"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"jobRepository"</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"jobRepository"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #808080;"><!-- 5) Define the job and its steps. In our case I use one step. Configure </span> <span style="color: #808080;"> its readers and writers --></span> <span style="color: #007000;"><batch:job</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"simpleJob"</span><span style="color: #007000;">></span> <span style="color: #007000;"><batch:listeners></span> <span style="color: #007000;"><batch:listener</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"appJobExecutionListener"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></batch:listeners></span> <span style="color: #007000;"><batch:step</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"step1"</span><span style="color: #007000;">></span> <span style="color: #007000;"><batch:tasklet></span> <span style="color: #007000;"><batch:listeners></span> <span style="color: #007000;"><batch:listener</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"itemFailureLoggerListener"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></batch:listeners></span> <span style="color: #007000;"><batch:chunk</span> <span style="color: #0000c0;">reader=</span><span style="background-color: #fff0f0;">"itemReader"</span> <span style="color: #0000c0;">writer=</span><span style="background-color: #fff0f0;">"itemWriter"</span> <span style="color: #0000c0;">commit-interval=</span><span style="background-color: #fff0f0;">"1000"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></batch:tasklet></span> <span style="color: #007000;"></batch:step></span> <span style="color: #007000;"></batch:job></span> <span style="color: #808080;"><!-- ======================================================= --></span> <span style="color: #808080;"><!-- 6) READER --></span> <span style="color: #808080;"><!-- ======================================================= --></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">id=</span><span style="background-color: #fff0f0;">"itemReader"</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.batch.item.file.FlatFileItemReader"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"resource"</span> <span style="color: #0000c0;">value=</span><span style="background-color: #fff0f0;">"classpath:ledger.txt"</span> <span style="color: #007000;">/></span> <span style="color: #808080;"><!-- property name="linesToSkip" value="1" / --></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"lineMapper"</span><span style="color: #007000;">></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.batch.item.file.mapping.DefaultLineMapper"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"lineTokenizer"</span><span style="color: #007000;">></span> <span style="color: #007000;"><bean</span> <span style="color: #0000c0;">class=</span><span style="background-color: #fff0f0;">"org.springframework.batch.item.file.transform.DelimitedLineTokenizer"</span><span style="color: #007000;">></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"names"</span> <span style="color: #0000c0;">value=</span><span style="background-color: #fff0f0;">"receiptDate,memberName,checkNumber,checkDate,paymentType,depositAmount,paymentAmount,comments"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #007000;"></property></span> <span style="color: #007000;"><property</span> <span style="color: #0000c0;">name=</span><span style="background-color: #fff0f0;">"fieldSetMapper"</span> <span style="color: #0000c0;">ref=</span><span style="background-color: #fff0f0;">"ledgerMapper"</span> <span style="color: #007000;">/></span> <span style="color: #007000;"></bean></span> <span style="color: #007000;"></property></span> <span style="color: #007000;"></bean></span> <span style="color: #007000;"></beans></span> |
- 1 through 4 are the same as the previous blog.
- 5 – Defines the job and its steps. Also registers a job listener and a step listener
- 6 – The reader used to read the flat file with comma separated columns. The FlatFileItemReader will read the rows from the flat file and pass it to the writer to persist to the database..
- 8 – Not shown here is the item writer. It is configured using annotations and the class is LedgerWriter.
Now for some of the Java code. Here is the DAO – LedgerDAOImpl.java
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
<span style="color: #008000; font-weight: bold;">package</span> com<span style="color: #303030;">.</span><span style="color: #0000c0;">batch</span><span style="color: #303030;">.</span><span style="color: #0000c0;">todb</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">java.sql.PreparedStatement</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">java.sql.SQLException</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.beans.factory.annotation.Autowired</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.jdbc.core.JdbcTemplate</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.jdbc.core.PreparedStatementSetter</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.stereotype.Repository</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.transaction.annotation.Propagation</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.transaction.annotation.Transactional</span><span style="color: #303030;">;</span> <span style="color: #505050; font-weight: bold;">@Repository</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #008000; font-weight: bold;">class</span> <span style="color: #b00060; font-weight: bold;">LedgerDAOImpl</span> <span style="color: #008000; font-weight: bold;">implements</span> LedgerDAO <span style="color: #303030;">{</span> <span style="color: #505050; font-weight: bold;">@Autowired</span> <span style="color: #008000; font-weight: bold;">private</span> JdbcTemplate jdbcTemplate<span style="color: #303030;">;</span> <span style="color: #505050; font-weight: bold;">@Transactional</span><span style="color: #303030;">(</span>propagation <span style="color: #303030;">=</span> Propagation<span style="color: #303030;">.</span><span style="color: #0000c0;">REQUIRED</span><span style="color: #303030;">)</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #303090; font-weight: bold;">void</span> <span style="color: #0060b0; font-weight: bold;">save</span><span style="color: #303030;">(</span><span style="color: #008000; font-weight: bold;">final</span> Ledger item<span style="color: #303030;">)</span> <span style="color: #303030;">{</span> jdbcTemplate <span style="color: #303030;">.</span><span style="color: #0000c0;">update</span><span style="color: #303030;">(</span><span style="background-color: #fff0f0;">"insert into ledger (rcv_dt, mbr_nm, chk_nbr, chk_dt, pymt_typ, dpst_amt, pymt_amt, comments) values(?,?,?,?,?,?,?,?)"</span><span style="color: #303030;">,</span> <span style="color: #008000; font-weight: bold;">new</span> <span style="color: #0060b0; font-weight: bold;">PreparedStatementSetter</span><span style="color: #303030;">()</span> <span style="color: #303030;">{</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #303090; font-weight: bold;">void</span> <span style="color: #0060b0; font-weight: bold;">setValues</span><span style="color: #303030;">(</span>PreparedStatement stmt<span style="color: #303030;">)</span> <span style="color: #008000; font-weight: bold;">throws</span> SQLException <span style="color: #303030;">{</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setDate</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">1</span><span style="color: #303030;">,</span> <span style="color: #008000; font-weight: bold;">new</span> java<span style="color: #303030;">.</span><span style="color: #0000c0;">sql</span><span style="color: #303030;">.</span><span style="color: #0000c0;">Date</span><span style="color: #303030;">(</span>item <span style="color: #303030;">.</span><span style="color: #0000c0;">getReceiptDate</span><span style="color: #303030;">().</span><span style="color: #0000c0;">getTime</span><span style="color: #303030;">()));</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setString</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">2</span><span style="color: #303030;">,</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">getMemberName</span><span style="color: #303030;">());</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setString</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">3</span><span style="color: #303030;">,</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">getCheckNumber</span><span style="color: #303030;">());</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setDate</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">4</span><span style="color: #303030;">,</span> <span style="color: #008000; font-weight: bold;">new</span> java<span style="color: #303030;">.</span><span style="color: #0000c0;">sql</span><span style="color: #303030;">.</span><span style="color: #0000c0;">Date</span><span style="color: #303030;">(</span>item <span style="color: #303030;">.</span><span style="color: #0000c0;">getCheckDate</span><span style="color: #303030;">().</span><span style="color: #0000c0;">getTime</span><span style="color: #303030;">()));</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setString</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">5</span><span style="color: #303030;">,</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">getPaymentType</span><span style="color: #303030;">());</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setDouble</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">6</span><span style="color: #303030;">,</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">getDepositAmount</span><span style="color: #303030;">());</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setDouble</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">7</span><span style="color: #303030;">,</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">getPaymentAmount</span><span style="color: #303030;">());</span> stmt<span style="color: #303030;">.</span><span style="color: #0000c0;">setString</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">8</span><span style="color: #303030;">,</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">getComments</span><span style="color: #303030;">());</span> <span style="color: #303030;">}</span> <span style="color: #303030;">});</span> <span style="color: #303030;">}</span> <span style="color: #303030;">}</span> |
Here is the mapper – LedgerMapper.java
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
<span style="color: #008000; font-weight: bold;">package</span> com<span style="color: #303030;">.</span><span style="color: #0000c0;">batch</span><span style="color: #303030;">.</span><span style="color: #0000c0;">todb</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">java.text.DecimalFormat</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">java.text.ParseException</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.batch.item.file.mapping.FieldSetMapper</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.batch.item.file.transform.FieldSet</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.stereotype.Component</span><span style="color: #303030;">;</span> <span style="color: #505050; font-weight: bold;">@Component</span><span style="color: #303030;">(</span><span style="background-color: #fff0f0;">"ledgerMapper"</span><span style="color: #303030;">)</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #008000; font-weight: bold;">class</span> <span style="color: #b00060; font-weight: bold;">LedgerMapper</span> <span style="color: #008000; font-weight: bold;">implements</span> FieldSetMapper <span style="color: #303030;">{</span> <span style="color: #008000; font-weight: bold;">private</span> <span style="color: #008000; font-weight: bold;">final</span> <span style="color: #008000; font-weight: bold;">static</span> String DATE_PATTERN <span style="color: #303030;">=</span> <span style="background-color: #fff0f0;">"mm/DD/yy"</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">private</span> <span style="color: #008000; font-weight: bold;">final</span> <span style="color: #008000; font-weight: bold;">static</span> String DOLLAR_PATTERN <span style="color: #303030;">=</span> <span style="background-color: #fff0f0;">"$###,###.###"</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">public</span> Object <span style="color: #0060b0; font-weight: bold;">mapFieldSet</span><span style="color: #303030;">(</span>FieldSet fs<span style="color: #303030;">)</span> <span style="color: #303030;">{</span> Ledger item <span style="color: #303030;">=</span> <span style="color: #008000; font-weight: bold;">new</span> Ledger<span style="color: #303030;">();</span> <span style="color: #303090; font-weight: bold;">int</span> idx <span style="color: #303030;">=</span> <span style="color: #0000d0; font-weight: bold;">0</span><span style="color: #303030;">;</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setReceiptDate</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readDate</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++,</span> DATE_PATTERN<span style="color: #303030;">));</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setMemberName</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readString</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++));</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setCheckNumber</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readString</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++));</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setCheckDate</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readDate</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++,</span> DATE_PATTERN<span style="color: #303030;">));</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setPaymentType</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readString</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++));</span> <span style="color: #808080;">// deposit amount</span> <span style="color: #008000; font-weight: bold;">try</span> <span style="color: #303030;">{</span> DecimalFormat fmttr <span style="color: #303030;">=</span> <span style="color: #008000; font-weight: bold;">new</span> DecimalFormat<span style="color: #303030;">(</span>DOLLAR_PATTERN<span style="color: #303030;">);</span> Number number <span style="color: #303030;">=</span> fmttr<span style="color: #303030;">.</span><span style="color: #0000c0;">parse</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readString</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++));</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setDepositAmount</span><span style="color: #303030;">(</span>number<span style="color: #303030;">.</span><span style="color: #0000c0;">doubleValue</span><span style="color: #303030;">());</span> <span style="color: #303030;">}</span> <span style="color: #008000; font-weight: bold;">catch</span> <span style="color: #303030;">(</span>ParseException e<span style="color: #303030;">)</span> <span style="color: #303030;">{</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setDepositAmount</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">0</span><span style="color: #303030;">);</span> <span style="color: #303030;">}</span> <span style="color: #808080;">// payment amount</span> <span style="color: #008000; font-weight: bold;">try</span> <span style="color: #303030;">{</span> DecimalFormat fmttr <span style="color: #303030;">=</span> <span style="color: #008000; font-weight: bold;">new</span> DecimalFormat<span style="color: #303030;">(</span>DOLLAR_PATTERN<span style="color: #303030;">);</span> Number number <span style="color: #303030;">=</span> fmttr<span style="color: #303030;">.</span><span style="color: #0000c0;">parse</span><span style="color: #303030;">(</span>fs<span style="color: #303030;">.</span><span style="color: #0000c0;">readString</span><span style="color: #303030;">(</span>idx<span style="color: #303030;">++));</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setPaymentAmount</span><span style="color: #303030;">(</span>number<span style="color: #303030;">.</span><span style="color: #0000c0;">doubleValue</span><span style="color: #303030;">());</span> <span style="color: #303030;">}</span> <span style="color: #008000; font-weight: bold;">catch</span> <span style="color: #303030;">(</span>ParseException e<span style="color: #303030;">)</span> <span style="color: #303030;">{</span> item<span style="color: #303030;">.</span><span style="color: #0000c0;">setPaymentAmount</span><span style="color: #303030;">(</span><span style="color: #0000d0; font-weight: bold;">0</span><span style="color: #303030;">);</span> <span style="color: #303030;">}</span> <span style="color: #808080;">//</span> <span style="color: #008000; font-weight: bold;">return</span> item<span style="color: #303030;">;</span> <span style="color: #303030;">}</span> <span style="color: #303030;">}</span> |
Here is the writer – LedgerWriter.java
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
<span style="color: #008000; font-weight: bold;">package</span> com<span style="color: #303030;">.</span><span style="color: #0000c0;">batch</span><span style="color: #303030;">.</span><span style="color: #0000c0;">todb</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">java.util.Iterator</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">java.util.List</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.batch.item.ItemWriter</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.beans.factory.annotation.Autowired</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.stereotype.Component</span><span style="color: #303030;">;</span> <span style="color: #505050; font-weight: bold;">@Component</span><span style="color: #303030;">(</span><span style="background-color: #fff0f0;">"itemWriter"</span><span style="color: #303030;">)</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #008000; font-weight: bold;">class</span> <span style="color: #b00060; font-weight: bold;">LedgerWriter</span> <span style="color: #008000; font-weight: bold;">implements</span> ItemWriter <span style="color: #303030;">{</span> <span style="color: #505050; font-weight: bold;">@Autowired</span> <span style="color: #008000; font-weight: bold;">private</span> LedgerDAO itemDAO<span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #303090; font-weight: bold;">void</span> <span style="color: #0060b0; font-weight: bold;">write</span><span style="color: #303030;">(</span>List items<span style="color: #303030;">)</span> <span style="color: #008000; font-weight: bold;">throws</span> Exception <span style="color: #303030;">{</span> <span style="color: #008000; font-weight: bold;">for</span> <span style="color: #303030;">(</span>Iterator<span style="color: #303030;"><</span>Ledger<span style="color: #303030;">></span> iterator <span style="color: #303030;">=</span> items<span style="color: #303030;">.</span><span style="color: #0000c0;">iterator</span><span style="color: #303030;">();</span> iterator<span style="color: #303030;">.</span><span style="color: #0000c0;">hasNext</span><span style="color: #303030;">();)</span> <span style="color: #303030;">{</span> Ledger item <span style="color: #303030;">=</span> iterator<span style="color: #303030;">.</span><span style="color: #0000c0;">next</span><span style="color: #303030;">();</span> itemDAO<span style="color: #303030;">.</span><span style="color: #0000c0;">save</span><span style="color: #303030;">(</span>item<span style="color: #303030;">);</span> <span style="color: #303030;">}</span> <span style="color: #303030;">}</span> <span style="color: #303030;">}</span> |
The test driver class is a JUnit class.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
<span style="color: #008000; font-weight: bold;">package</span> com<span style="color: #303030;">.</span><span style="color: #0000c0;">batch</span><span style="color: #303030;">.</span><span style="color: #0000c0;">todb</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.apache.log4j.Logger</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.junit.Test</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.junit.runner.RunWith</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.batch.core.Job</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.batch.core.JobParameters</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.batch.core.launch.JobLauncher</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.beans.factory.annotation.Autowired</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.test.context.ContextConfiguration</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.test.context.junit4.SpringJUnit4ClassRunner</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.test.context.transaction.TransactionConfiguration</span><span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">org.springframework.util.StopWatch</span><span style="color: #303030;">;</span> <span style="color: #505050; font-weight: bold;">@ContextConfiguration</span><span style="color: #303030;">(</span>locations <span style="color: #303030;">=</span> <span style="background-color: #fff0f0;">"classpath:com/batch/todb/contextToDB.xml"</span><span style="color: #303030;">)</span> <span style="color: #505050; font-weight: bold;">@RunWith</span><span style="color: #303030;">(</span>SpringJUnit4ClassRunner<span style="color: #303030;">.</span><span style="color: #0000c0;">class</span><span style="color: #303030;">)</span> <span style="color: #505050; font-weight: bold;">@TransactionConfiguration</span><span style="color: #303030;">(</span>transactionManager <span style="color: #303030;">=</span> <span style="background-color: #fff0f0;">"transactionManager"</span><span style="color: #303030;">,</span> defaultRollback <span style="color: #303030;">=</span> <span style="color: #008000; font-weight: bold;">false</span><span style="color: #303030;">)</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #008000; font-weight: bold;">class</span> <span style="color: #b00060; font-weight: bold;">ToDBBatchTestCase</span> <span style="color: #303030;">{</span> <span style="color: #008000; font-weight: bold;">private</span> <span style="color: #008000; font-weight: bold;">final</span> <span style="color: #008000; font-weight: bold;">static</span> Logger logger <span style="color: #303030;">=</span> Logger <span style="color: #303030;">.</span><span style="color: #0000c0;">getLogger</span><span style="color: #303030;">(</span>ToDBBatchTestCase<span style="color: #303030;">.</span><span style="color: #0000c0;">class</span><span style="color: #303030;">);</span> <span style="color: #505050; font-weight: bold;">@Autowired</span> <span style="color: #008000; font-weight: bold;">private</span> JobLauncher launcher<span style="color: #303030;">;</span> <span style="color: #505050; font-weight: bold;">@Autowired</span> <span style="color: #008000; font-weight: bold;">private</span> Job job<span style="color: #303030;">;</span> <span style="color: #008000; font-weight: bold;">private</span> JobParameters jobParameters <span style="color: #303030;">=</span> <span style="color: #008000; font-weight: bold;">new</span> JobParameters<span style="color: #303030;">();</span> <span style="color: #505050; font-weight: bold;">@Test</span> <span style="color: #008000; font-weight: bold;">public</span> <span style="color: #303090; font-weight: bold;">void</span> <span style="color: #0060b0; font-weight: bold;">testLaunchJob</span><span style="color: #303030;">()</span> <span style="color: #008000; font-weight: bold;">throws</span> Exception <span style="color: #303030;">{</span> StopWatch sw <span style="color: #303030;">=</span> <span style="color: #008000; font-weight: bold;">new</span> StopWatch<span style="color: #303030;">();</span> sw<span style="color: #303030;">.</span><span style="color: #0000c0;">start</span><span style="color: #303030;">();</span> launcher<span style="color: #303030;">.</span><span style="color: #0000c0;">run</span><span style="color: #303030;">(</span>job<span style="color: #303030;">,</span> jobParameters<span style="color: #303030;">);</span> sw<span style="color: #303030;">.</span><span style="color: #0000c0;">stop</span><span style="color: #303030;">();</span> logger<span style="color: #303030;">.</span><span style="color: #0000c0;">info</span><span style="color: #303030;">(</span><span style="background-color: #fff0f0;">">>> TIME ELAPSED:"</span> <span style="color: #303030;">+</span> sw<span style="color: #303030;">.</span><span style="color: #0000c0;">prettyPrint</span><span style="color: #303030;">());</span> <span style="color: #303030;">}</span> <span style="color: #303030;">}</span> |
Running the test case will insert approx 200k rows into the ledger table. It took roughly 1:12 seconds for the entire process to execute.
|
1 |
19:53:30,192 INFO ToDBBatchTestCase:36 - >>> TIME ELAPSED:StopWatch '': running time (millis) = 61125 |
Next move over to Spring Batch – Part III – From Database to Flat File. You can download the Maven project from GitHub – https://github.com/thomasma/springbatch3part.