iComposer & AI
iComposer
iComposer Batch
Sidebar On this page

iComposer Batch is a batch job orchestration platform built on Spring Batch. It supports dynamically defining, configuring, and executing batch jobs through Groovy DSL. This document covers the core concepts, operational workflows, and code templates.

Table of Contents


Core Concepts

ConceptDescription
Batch JobA batch processing task containing metadata (name, module, group, status) and an ordered set of Steps
StepA processing unit within a Job, supporting Chunk (batch processing) and Tasklet (single task) modes
Step ItemA component inside a Step, including Reader, Processor, Writer, Listener, etc.
ChunkSpring Batch’s chunk-oriented processing pattern: Reader reads a batch → Processor processes items one by one → Writer writes the batch

Execution Flow:

Batch Job
└─ Step 1 (Chunk mode)
├─ ItemReader / ItemStreamReader / ItemEnhancedPagingReader
├─ ItemProcessor
├─ ItemWriter
└─ Listeners (Step / Read / Process / Write / Skip)
└─ Step 2 (Tasklet mode)
├─ Tasklet
└─ Listeners

Data Model

Batch Job

FieldDescriptionConstraints
BatchNameBatch nameMust match [A-Z][0-9A-Za-z_]*
JobNameSpring Batch Job nameRuntime identifier
BatchTypeBatch type
StatusStatus1 = Active
ModuleIdOwning moduleLinks to a business module
GroupIdGroupUsed for grouped management and import/export
DescriptionDescription

Batch Step

FieldDescriptionConstraints
StepNameStep nameMust follow naming conventions
StepTypeStep type1 = Chunk, 2 = Tasklet
SeqExecution orderNumeric value determining step execution order
ChunkSizeChunk sizeNumber of records per chunk in Chunk mode
ThrottleLimitConcurrency limitNumber of concurrent processing threads
SkipLimitSkip limitMaximum allowed errors before failing
StatusStatus

Batch Step Item

FieldDescriptionConstraints
ItemNameComponent nameMust follow naming conventions
ItemTypeComponent typeReader / Processor / Writer / Tasklet, etc.
ImplementTypeImplementation typeSpecific implementation variant (includes Listener subtypes)
ContentGroovy codeScript content compiled and executed at runtime
StaticCompileStatic compilationY / N; enabling improves performance
FoundationDisabledFoundation whitelistY / N

Operational Workflow

1. Create a Batch Job

  1. Navigate to iComposer → Batch page
  2. Click New, fill in BatchName, JobName, Module, Group, etc.
  3. Save

2. Configure Steps

  1. Open the Job detail page and add a Step
  2. Select StepType (Chunk or Tasklet)
  3. Configure ChunkSize (for Chunk mode), SkipLimit, and ThrottleLimit
  4. Set Seq to define the step execution order

3. Write Item Code

  1. Add an Item (Reader / Processor / Writer / Tasklet) to the Step
  2. Select the ImplementType — the system auto-loads the corresponding template
  3. Write business logic at the //TODO markers
  4. Save and compile

4. Execute & Monitor

  1. Submit the job (call the Submit Job API)
  2. Query execution status via JobExecutionId
  3. View Step execution details and error logs

5. Import & Export

  • Export Batch configurations in bulk by GroupId
  • Import exported configurations into other environments

Step Types

Chunk Mode (StepType = 1)

Processes data in chunks, suitable for large-volume ETL scenarios:

┌─────────────┐ ┌──────────────┐ ┌─────────────┐
ItemReader ──→ ItemProcessor ──→ ItemWriter
Read N items Process each Write N items│
└─────────────┘ └──────────────┘ └─────────────┘

└──────── Repeat until exhausted ─────┘
  • ChunkSize determines the number of records per read → process → write cycle
  • Configure SkipLimit to tolerate error records
  • Configure ThrottleLimit to control concurrency

Tasklet Mode (StepType = 2)

Executes a single task, suitable for initialization, cleanup, notification, and other non-data-flow scenarios:

┌─────────────┐
Tasklet Execute a custom operation once
└─────────────┘

Item Types & Code Templates

ItemReader — Data Reading

Reads from a data source and returns a list of objects. Each invocation returns a List; the framework passes items one by one to the Processor.

/*
* Implement Type : ItemReader
* return: object list
* Quick Method:
* 1. Map getParameters()
* 2. StepExecution getStepExecution()
*/
import org.springframework.batch.core.StepExecution;

StepExecution stepExecution = getStepExecution();
def params = getParameters();

// TODO: Query data and return a list of records
def data = ["1", "2"]

return data;

Use case: Small data volumes that can be loaded into memory at once.


ItemStreamReader — Streaming Paged Reading

Streams data with paged queries and cursor-based positioning. Each invocation returns a single object.

/*
* Implement Type : ItemStreamReader
* return: single object (return null to signal end of stream)
* Quick Method:
* 1. Map getParameters()
* 2. StepExecution getStepExecution()
* 3. long getPreDetailId() / setPreDetailId(long)
* 4. int getPos() / setPos(int)
* 5. List getObjectList() / setObjectList(List)
*/
import org.apache.commons.collections4.CollectionUtils;
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();

def objectList = getObjectList(); // Currently queried records
def preId = getPreDetailId(); // Last ID from the previous query
def pos = getPos(); // Current traversal position

// First query or current batch exhausted — fetch the next batch
if (CollectionUtils.isEmpty(objectList) || pos >= objectList.size() - 1) {
// TODO: Replace with actual paged query logic, using preId as cursor
objectList = ["111", "222", "333"];

setObjectList(objectList); // Save the latest query results
if (CollectionUtils.isEmpty(objectList)) {
return null; // Empty result means all data has been read
}
pos = -1; // Reset position
}
pos++;
Object object = objectList.get(pos);
preId = 0L; // TODO: Replace with object.getXXXId()
setPos(pos);
setPreDetailId(preId);
return object;

Use case: Large data volumes — paged queries prevent OOM; uses ID-based cursor pagination.


ItemEnhancedPagingReader — Enhanced Paged Reading

The framework automatically manages pagination parameters (startIdNotIncluded / maxRecords). Each invocation returns a batch of data.

/*
* Implement Type : ItemEnhancedPagingReader
* return: object list (return null to signal end of data)
* Quick Method:
* 1. Map getParameters()
* 2. StepExecution getStepExecution()
* 3. String getIdField() / setIdField(String)
* 4. Long startIdNotIncluded / int maxRecords
*/
import org.apache.commons.lang3.StringUtils;
import org.springframework.batch.core.StepExecution;

info("params: startIdNotIncluded={},maxRecords={}", startIdNotIncluded, maxRecords)

// TODO: Replace termination condition with actual logic
if (startIdNotIncluded >= 2) return null;

def parameters = getParameters();

def idField = getIdField();
if (StringUtils.isEmpty(idField)) {
setIdField("XXXX"); // TODO: Replace with actual primary key field name, e.g. "id"
}

StepExecution stepExecution = getStepExecution();

// TODO: Use startIdNotIncluded and maxRecords for paged query
// SQL example: SELECT * FROM table WHERE id > startIdNotIncluded ORDER BY id LIMIT maxRecords
def dataList = [["id": 1], ["id": 2]]

return dataList;

Use case: Large data volumes with auto-increment primary key. The framework auto-increments startIdNotIncluded; maxRecords is derived from ChunkSize.


ItemProcessor — Data Processing

Processes items read by the Reader one by one. Supports transformation, validation, enrichment, etc.

/*
* Implement Type : ItemProcessor
* return: processed object (return null to filter out the item)
* Quick Method:
* 1. Map getParameters()
* 2. StepExecution getStepExecution()
* 3. Object getItemObject()
*/
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
def item = getItemObject();

// TODO: Write processing logic

return item;

Note: Returning null filters out the item — it will not be passed to the Writer.


ItemWriter — Data Writing

Receives a batch of processed items from the Processor and performs write operations (DB insert, API call, file output, etc.).

/*
* Implement Type : ItemWriter
* return: none
* Quick Method:
* 1. Map getParameters()
* 2. StepExecution getStepExecution()
* 3. List getItemObjectList()
*/
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
def items = getItemObjectList();

info("Write data list, record count: " + items.size());

// TODO: Write batch insert/update logic

Note: The list size returned by getItemObjectList() equals ChunkSize (the last batch may be smaller).


Tasklet — Custom Task

Executes a one-shot operation not involving data flow, such as sending notifications, initializing environments, or cleaning up temporary data.

/*
* Implement Type : Tasklet
* return: none
* Quick Method:
* 1. Map getParameters()
* 2. StepExecution getStepExecution()
*/
import org.springframework.batch.core.StepExecution;

def parameter = getParameters();
StepExecution stepExecution = getStepExecution();

// TODO: Write business logic

Listener Types

Listeners inject callback logic at various stages of batch execution (logging, monitoring, error handling, etc.). A single Listener Item can contain multiple method implementations, separated by //==========ICOMPOSER SPLIT LINE==========.

StepExecutionListener — Step Listener

Triggers callbacks before and after Step execution.

/*--- beforeStep ---*/
/*
* Method: beforeStep
* return: object
* Quick Method: getParameters(), getStepExecution()
*/
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
info("Before Step: " + stepExecution.getStepName());

//==========ICOMPOSER SPLIT LINE==========
/*--- afterStep ---*/
/*
* Method: afterStep
* return: ExitStatus
* Quick Method: getParameters(), getStepExecution()
*/
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.ExitStatus;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
info("After Step: " + stepExecution.getStepName());

return stepExecution.getExitStatus();

JobExecutionListener — Job Listener

Triggers callbacks before and after the entire Job execution.

/*--- beforeJob ---*/
/*
* Method: beforeJob
* return: object
* Quick Method: getParameters(), getJobExecution()
*/
import org.springframework.batch.core.JobExecution;

def parameters = getParameters();
JobExecution jobExecution = getJobExecution();
info("Before Job: " + jobExecution.getJobId());

//==========ICOMPOSER SPLIT LINE==========
/*--- afterJob ---*/
/*
* Method: afterJob
* return: object
* Quick Method: getParameters(), getJobExecution()
*/
import org.springframework.batch.core.JobExecution;

def parameters = getParameters();
JobExecution jobExecution = getJobExecution();
info("After Job: " + jobExecution.getJobId());

ReadListener — Read Listener

Triggers before/after ItemReader reads data, and on read errors.

MethodTriggerAvailable Quick Methods
beforeReadBefore each readgetParameters(), getStepExecution()
afterReadAfter each readgetParameters(), getStepExecution(), getItemObject()
onReadErrorOn read exceptiongetParameters(), getStepExecution(), getReadException()

ProcessListener — Process Listener

Triggers before/after ItemProcessor processes data, and on processing errors.

MethodTriggerAvailable Quick Methods
beforeProcessBefore processinggetParameters(), getStepExecution(), getItemObject()
afterProcessAfter processinggetParameters(), getStepExecution(), getItemObject(), getProcessResult()
onProcessErrorOn process exceptiongetParameters(), getStepExecution(), getItemObject(), getProcessException()

WriteListener — Write Listener

Triggers before/after ItemWriter writes data, and on write errors.

MethodTriggerAvailable Quick Methods
beforeWriteBefore writinggetParameters(), getStepExecution(), getItemObjectList()
afterWriteAfter writinggetParameters(), getStepExecution(), getItemObjectList()
onWriteErrorOn write exceptiongetParameters(), getStepExecution(), getItemObjectList(), getWriteException()

SkipListener — Skip Listener

Triggers when a data item is skipped due to an exception (requires SkipLimit > 0).

/*--- skipInRead ---*/
/*
* Method: skipInRead
* Quick Method: getParameters(), getStepExecution(), getSkipInReadThrowable()
*/
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
info("Skip In Read: " + stepExecution.getStepName());

//==========ICOMPOSER SPLIT LINE==========
/*--- skipInProcess ---*/
/*
* Method: skipInProcess
* Quick Method: getParameters(), getStepExecution(),
* getSkipInProcessItemObject(), getSkipInProcessThrowable()
*/
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
info("Skip In Process: " + stepExecution.getStepName());

//==========ICOMPOSER SPLIT LINE==========
/*--- skipInWrite ---*/
/*
* Method: skipInWrite
* Quick Method: getParameters(), getStepExecution(),
* getSkipInWriteItemObject(), getSkipInWriteThrowable()
*/
import org.springframework.batch.core.StepExecution;

def parameters = getParameters();
StepExecution stepExecution = getStepExecution();
info("Skip In Write: " + stepExecution.getStepName());

Common Quick Methods

All Item types can use the following built-in methods:

MethodReturn TypeDescription
getParameters()MapReturns the parameters passed at job startup
getStepExecution()StepExecutionReturns the current Step’s execution context
info(String msg)voidLogs at INFO level
info(String fmt, Object... args)voidLogs at INFO level with format placeholders

Item-specific Quick Methods:

Item TypeQuick MethodDescription
ItemProcessorgetItemObject()Returns the current item to be processed
ItemWritergetItemObjectList()Returns the current list of items to write
ItemStreamReadergetPos() / setPos(int)Get/set current traversal position
getPreDetailId() / setPreDetailId(long)Get/set last ID from previous query
getObjectList() / setObjectList(List)Get/set current batch of data
ItemEnhancedPagingReadergetIdField() / setIdField(String)Get/set primary key field name
startIdNotIncludedStarting ID (exclusive), auto-incremented by framework
maxRecordsMax records per batch, derived from ChunkSize
Job ListenergetJobExecution()Returns the Job execution context

Best Practices

Choosing a Reader

ScenarioRecommended ReaderReason
Small data volume (< 1000 records)ItemReaderSimple and direct — one-time load
Large data volume, cursor pagination neededItemStreamReaderManual control over paging logic and cursor
Large data volume, auto-increment PK availableItemEnhancedPagingReaderFramework manages paging automatically — minimal code

ChunkSize Configuration

  • General scenarios: 50 ~ 200
  • External API calls involved: 10 ~ 50 (reduces timeout risk)
  • Pure database read/write: 200 ~ 1000

Error Handling

  • Set a reasonable SkipLimit to allow minor data errors without aborting the job
  • Use SkipListener to log skipped records for post-mortem analysis
  • Use try-catch in the Processor for predictable exceptions to prevent overall failure

Logging Guidelines

  • Use the info() method — do not use println
  • Include key business identifiers in log messages (e.g., record ID, batch number)
  • Listener logs should include Step/Job names for traceability

Naming Conventions

  • BatchName: UPPER_SNAKE_CASE, e.g., POLICY_MIGRATION_JOB
  • StepName: Descriptive, e.g., READ_POLICY_DATA, PROCESS_CLAIM
  • ItemName: Reflect type and responsibility, e.g., PolicyItemReader, ClaimItemProcessor

Performance Optimization

  • Avoid full table scans in Readers — use WHERE conditions and pagination
  • Use batch operations in Writers (e.g., batch insert) — do not write records one by one
  • Enable StaticCompile = Y to improve Groovy execution performance
  • Configure ThrottleLimit appropriately for parallel processing

Multi-Step Coordination

  • Use StepExecution.getExecutionContext() to pass data between steps
  • Strictly control step execution order via the Seq field
  • Tasklet steps are ideal for pre-validation or post-notification


Feedback
Was this page helpful?
|
Provide feedback