Spring Batch Tutorial: Introduction

Most enterprise applications rely heavily on batch jobs. They run during the night and do all the time consuming tasks that cannot be done during business hours. These tasks are often critical to the business and errors can cause serious damage (i.e. cost a lot of money).

That’s why it is important to write robust batch jobs that provide the correct output, are fault tolerant, and are as fast as possible. Spring Batch can help you to achieve these goals.

This blog post is the first part of my Spring Batch tutorial, and it provides a quick introduction to Spring Batch. After you have read this blog post, you:

• Can specify the term batch job.
• Understand why you should use Spring Batch instead of writing your own batch jobs.
• Can identify the basic building blocks of a Spring Batch job.

Let’s start by defining the term batch job.

What Is a Batch Job?

A batch job is often defined as follows:

A batch job is a computer program or set of programs processed in batch mode. This means that a sequence of commands to be executed by the operating system is listed in a file (often called a batch file, command file, or shell script) and submitted for execution as a single unit.

However, this definition is not very pragmatic, and it doesn’t help you to understand what kind of batch jobs are required by a typical enterprise application. That’s why I will provide my own definition:

A batch job reads input data, processes the input data, and writes the processed data to the configured output.

The following figure illustrates a simple batch job that fulfills my definition:

As you can see, this batch job has only one step. This is perfectly fine if your batch job has only one logical task. For example, if you are implementing an import job that reads information from an input file and writes it to the database, your job has only one logical task.

However, some batch jobs have more than one logical task. For example, you might have to implement a batch job that imports information from an input file and creates an export file that’s exported to other applications. In other words, your batch job has two logical tasks. This means that it has two steps as well.

It seems that I have to rewrite my definition. The final version is:

A batch job consists of one or more steps. Each step is responsible of completing one logical task. Every step reads input data, processes the input data, and writes the processed data to the configured output. If the batch job has more than one step, the output of a step is often used as an input of the next step.

The following figure illustrates a batch job that has two steps:

I have now defined the term batch job. Let’s find out why you should implement your batch jobs by using Spring Batch.

How Can Spring Batch Help Us?

I have written a lot of batch jobs during my career and seen many batch jobs written by other developers. I have noticed that most non-trivial batch jobs (including mine), which don’t use any framework or library, suffer from these problems:

• The code that implements the batch job is a mess. Because it has only one huge step, no one cannot really understand how the batch job works.
• The batch job is slow because it does everything inside a HUGE transaction.
• The batch job doesn’t have a real error handling. If an error occurs during a batch job, the job simply fails. However, if you are lucky, the batch job might write an error message to a log file.
• The batch job doesn’t clean up the output data that’s written to the configured output if it fails. This is a problem because you cannot trust the data that’s produced by the batch job. In other words, you have to ensure (manually) that the output data of the batch job is correct. This is a waste of time.
• The batch job doesn’t report its final state. In other words, there is no easy way to figure out if the batch job was finished successfully.

You can (of course) fix every one of these problems. If you decide to follow this approach, you face two new problems:

• You have to essentially create an in-house batch job framework, and it is extremely hard to get everything right at the first time.
• Creating an in-house batch job framework is a big task and it takes time that you don’t often have. This means that you cannot fix the problems found from the first version of your batch job framework because you don’t have time to do it. That’s why all in-house frameworks have their own oddities.

Luckily, you don’t have to implement your own batch job framework because Spring Batch solves all of these problems. It provides the following features that helps you to solve these problems:

• It helps you to structure your code in a clean way by providing the infrastructure that’s used to implement, configure, and run batch jobs.
• It uses so called chunk oriented processing where items are processed one by one and the transaction is committed when the chunk size is met. In other words, it provides you an easy way to manage the size of your transactions.
• It provides proper error handling. For example, you can skip items if an exception is thrown and configure retry logic that’s used to determine whether your batch job should retry the failed operation. You can also configure the logic that’s used to decide if your transaction should be rolled back.
• It writes comprehensive log to the used database. This log contains the metadata of each job and step execution, and it’s extremely useful if you have to troubleshoot a failed batch job. Because the log is written to a database, you can access it by using a database client.

You should now understand that Spring Batch solves the problems caused by handwritten batch jobs. Let’s move on and take a quick look at the key components of a Spring Batch job.

The Key Components of a Spring Batch Job

A Spring Batch job consists of the following components:

• The Job represents a single Spring Batch job. Each job can have one or more steps.
• The Step represents an independent logical task (i.e. import information from an input file). Each step belongs to one job.
• The ItemReader reads the input data and provides the found items one by one. An ItemReader belongs to one step and each step must have one ItemReader.
• The ItemProcessor transforms items into a form that’s understood by the ItemWriter one item at a time. An ItemProcessor belongs to one step and each step can have one ItemProcessor.
• The ItemWriter writes an information of an item to the output one item at a time. An ItemWriter belongs to one step and each step must have one ItemWriter

The following figure illustrates the relationships of these components:

You can now define the term batch job, you understand why you should use Spring Batch, and you can identify the key components of a Spring Batch job. Let’s summarize what you learned from this blog post.

Summary

This blog post has taught you five things:

• A batch job consists of one or more steps. Each step is responsible of completing one logical task. Every step reads input data, processes the input data, and writes the processed data to the configured output. If the batch job has more than one step, the output of a step is often used as an input of the next step.
• You should use Spring Batch because it solves the problems caused by handwritten batch jobs.
• A Spring batch Job can have one or more steps.
• A Step must have one ItemReader and ItemWriter.
• A Step can have one ItemProcessor.

The next part of this tutorial describes how you can get the required dependencies with Maven.