Google

Jun 29, 2012

Reviewing a given Java code in job interviews and code review sessions

In job interviews, it is common to give a smippet of bad code, and ask you to describe what is wrong with the code and how you would go about improving it. A thorough review of the code will be beneficial not only in job interviews to impress your potential employers, but also to impress your peers and superiors during code review sessions.

Q. How will you go about improving on the following code snippet that calculates the new balance based on the current balance, total debits, and total credits?

 Note: All amounts need to be positive values.

import java.math.BigDecimal;

public class CashCalculatorBasic {

 public BigDecimal getCalculatedAvailableBalance(BigDecimal currentBalance, BigDecimal totalDebits, 
                                           BigDecimal totalCredits) {
  
  BigDecimal result =  currentBalance
     .subtract(totalDebits)
     .add(totalCredits);
  
  System.out.println("The calculated result is " + result);
  
  return result;

 }

 public static void main(String[] args) {
    
    new CashCalculatorBasic().getCalculatedAvailableBalance(
    new BigDecimal("1250.00"), new BigDecimal("250.00"), new BigDecimal("500.00"));
 }

}

A. Firstly, a good thing about the above code is that it uses the BigDecimal class instead of a floating point type like float or double. Here are a number of things that can be improved.


1. Don't use System.out.println(.....) and replace it with the log4j and slf4j framewowrks. It is a bad practice to use System.out.println because you cannot easily change log levels, turn it off, customize it, etc. A proper logging system, like Log4J, adds all sorts of features that are very useful in real applications. You can direct the output to different places (console, file, network, etc.). You can tell it to output only a subset of the messages, without having to recompile. You can get timestamp on each message, etc.




2. Don't use static void main method to test your class. Write unit tests using a unit testing framework like JUnit or TestNG.

3. Don't test only the happy path. Include negative test scenarios as well. By writing proper test cases, the code can be refactored with confidence.

4. The above code does not fail fast. A fail fast code is the one that performs input validation and throws any validation errors if the pre-conditions or post conditions are not met. For example, testing for null, empty String, negative values, etc.

5. Your code should favour code to interface. What if you want to write an improved CashCalculator? You will have to go and change all your references where CashCalculatorBasic is used.

6. The readability can be improved by abstracting out the calculation to a separate util class or an inner class as shown below. This will be really useful when you have more methods to calculate different values, which would be the case in real life scenario. This will also improve readability and reusability of the code. Here is the improved code. Define an interface, which is the contract for the invokers or callers.


import java.math.BigDecimal;


public interface CashCalculator {
 
  BigDecimal getCalculatedAvailableBalance(BigDecimal currentBalance, BigDecimal totalDebits, 
                                     BigDecimal totalCredits) ;

}

Now, define the implementation class that takes into consideration the best practices listed above. even though it has more code, as the class takes on more methods, it will improve its reusability and readability.
import java.math.BigDecimal;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;


/**
 * 
 * coding to an interface
 *
 */
public class CashCalculatorImproved implements  CashCalculator {
 
  //using a logging framework 
 private static final Logger log = LoggerFactory.getLogger(CashCalculatorImproved.class);

 public BigDecimal getCalculatedAvailableBalance(BigDecimal currentBalance, BigDecimal totalDebits, 
                                           BigDecimal totalCredits) {
      
  //improved readability
  BigDecimal result = new Balance(currentBalance)
                             .less(totalDebits)
                             .more(totalCredits)
                             .getBalance();
      
   //using a logging framework 
      if(log.isDebugEnabled()) {
       log.debug("The calculated result is {0}" , result);
      }
      
      return result;
 }

 //an inner class that abstracts out the calculation logic
 private class Balance {

  private BigDecimal balance;

  public Balance(BigDecimal balance) {
   //check for pre condition
   if (balance == null) {
      throw new IllegalArgumentException("Invalid balance " + balance);//fail fast
   }
   this.balance = balance;
  }

  public BigDecimal getBalance() {
   return balance;
  }

  public Balance less(BigDecimal amount) {
   //check for pre condition
   if (!isPositiveOrZero(amount)) {
      throw new IllegalArgumentException("Invalid amount " + amount);//fail fast
   }

   balance = balance.subtract(amount);
   
   //check for post condition
   if(!isPositiveOrZero(balance)){
      //should throw a real business exception
      throw new RuntimeException("The balance is negative " + balance);//fail fast
   }

   return this;
  }

  public Balance more(BigDecimal amount) {
   //check for pre condition
   if (!isPositiveOrZero(amount)) {
    throw new IllegalArgumentException("Invalid amount " + amount);//fail fast
   }

   balance = balance.add(amount);
 
   return this;
  }

  //a method with a meaningful name
  private boolean isPositiveOrZero(BigDecimal amount) {
     return amount != null && amount.signum() != -1;
  }

 }

}

The above code also demonstrates a good use of an inner class. Finally the unit tests that test for both positive (i.e. happy path) and negative scenarios.

import java.math.BigDecimal;

import junit.framework.Assert;

import org.junit.Before;
import org.junit.Test;

public class CashCalculatorImprovedTest {
 
 CashCalculator calculator = null;

 @Before
 public void init() {
  // prefer Spring/Guice to inject the dependency
  calculator = new CashCalculatorImproved();
 }

 @Test
 public void testCalculatedAvailableBalance(){
  BigDecimal result = calculator.getCalculatedAvailableBalance(
    new BigDecimal("1250.00"), new BigDecimal("250.00"), new BigDecimal("500.00"));
  Assert.assertEquals(new BigDecimal("1500.00"), result);
 }
 
 @Test
 public void testForZeroCalculatedAvailableBalance(){
  BigDecimal result = calculator.getCalculatedAvailableBalance(
    new BigDecimal("1250.00"), new BigDecimal("0.00"), new BigDecimal("0.00"));
  Assert.assertEquals(new BigDecimal("1250.00"), result);
 }
 
 
 @Test(expected=IllegalArgumentException.class)
 public void negativeTest1CalculatedAvailableBalance(){
   calculator.getCalculatedAvailableBalance( null, new BigDecimal("250.00"), new BigDecimal("500.00"));
 }
 
 @Test(expected=IllegalArgumentException.class)
 public void negativeTest2CalculatedAvailableBalance(){
   calculator.getCalculatedAvailableBalance( new BigDecimal("1250.00"), new BigDecimal("-25.00"), new BigDecimal("500.00"));
 }
 
 @Test(expected=IllegalArgumentException.class)
 public void negativeTest3CalculatedAvailableBalance(){
   calculator.getCalculatedAvailableBalance( new BigDecimal("1250.00"), new BigDecimal("250.00"), new BigDecimal("-500.00"));
 }
}  



Top 17 Java Coding Tips for job interviews and pre-interview coding tests


More similar examples are covered in my Java career Essentials and companion books. For example, you will be expected to know what design pattern to apply to a given piece of code and design patterns are discussed with examples in "Java/J2EE Job Interview Companion".

Labels:

Jun 27, 2012

Core Java coding questions frequently asked in written tests and interviews - part 3: method overloading Vs overriding

Core Java Coding Questions and Answers for beginner to intermediate level

Q1 Q2 Q3 Q4 Q5 - Q8 Q9 Q10 Q11 Q12 - Q14 Q15



This part of the Java coding interview question is on the popular method overloading versus overriding concept.

Q. What will be the output of the following code snippet?

public class MethodOverrideVsOverload {
  
 public boolean equals( MethodOverrideVsOverload other ) {
     System.out.println("MethodOverrideVsOverload equals method reached" );
     return true;
   }
 
 public static void main(String[] args) {
  Object o1 = new MethodOverrideVsOverload();
  Object o2 = new MethodOverrideVsOverload();
  
  MethodOverrideVsOverload o3 = new MethodOverrideVsOverload();
  MethodOverrideVsOverload o4 = new MethodOverrideVsOverload();
  
  if(o1.equals(o2)){
   System.out.println("objects o1 and o2 are equal");
  }
  
  if(o3.equals(o4)){
   System.out.println("objects o3 and o4 are equal");
  }
 }
}

A. The output will be

MethodOverrideVsOverload equals method reached
objects o3 and o4 are equal

What concepts does this question try to test?
  • In Java, a class can only extend a single class (i.e. single inheritance), and when it does not explicitly extend a class, it implicitly extends the class Object. So, MethodOverrideVsOverload implicitly extends the class Object. 
  • The majority of the non final Object class methods are meant to be overridden by the sub classes.

    public boolean equals(Object obj); // make note of this method
    public int hashCode();
    public String toString();
    
  • The method overloading takes place at compile time (i.e. static binding) and method overriding takes place at runtime (i.e. dynamic binding). Static binding means the JVM decides, which class or method to call during compile time. Dynamic binding means, the JVM decides, which class or method to call during runtime. The polymorphism is possible because of dynamic binding. Learn more about compile-time vs runtime
  • The method overriding must adhere to the following rules 

    Arguments Must not change.
    Return type Can't change except for covariant (subtype) returns.
    Exceptions The extending class can eliminate or call fewer exceptions than its parent, but must not throw new or broader checked exceptions.
    Access Must not be more restrictive than the class it extends. Can be less restrictive.
    Invocation Which method to call is based on object type, at runtime time (i.e. dynamic binding).

Now, if you look at the above code

The "equals(MethodOverrideVsOverload other)" method in class MethodOverrideVsOverload does not actually override the Object class's "public boolean equals(Object obj)" method. This is because it fails to adhere to the "Arguments" rule as both methods have different arguments as one is of  type "MethodOverrideVsOverload " and the other of type "Object" . So, the two methods are overloaded (happens at compile time) and not overridden.

So, when o1.equals(o2) is invoked, the public boolean equals(Object obj) method of the object class is invoked because during compile time, the o1 and o2 are of type Object.The Object class's equals( ... ) method returns false as it compares the memory address (e.g. Object@235f56 and Object@653af32) of both objects.

When o3.equals(o4) is invoked, the "equals( MethodOverrideVsOverload other )" of the class MethodOverrideVsOverload is invoked as during compile time o3 and o4 are of type MethodOverrideVsOverload, hence you get the above output.


What follow on questions can you expect?

Q. How will you fix the above issue?
A. In Java 5, annotations were introduced and one of the handy compile time annotations is the @override, which will ensure that the methods are overridden correctly. If you had this annotation, when you override it incorrectly as in the above example, a compile time error will be thrown. 

So, to fix it, add the @override annotation to the "boolean equals( MethodOverrideVsOverload other )" of the MethodOverrideVsOverload  class. This will give you a compile time error indicating that the method is not properly overridden. Now, fix the method signature by changing the  argument type in the method signature from "MethodOverrideVsOverload" to "Object" as shown below.


public class MethodOverrideVsOverload {
 
 @Override
 public boolean equals( Object other ) {
     System.out.println("MethodOverrideVsOverload equals method reached" );
     return true;
 }
 
 
 public static void main(String[] args) {
  Object o1 = new MethodOverrideVsOverload(); //during compile time o1 is of type Object
                                              //during runtime o1 is of type MethodOverrideVsOverload
  Object o2 = new MethodOverrideVsOverload(); //during compile time o2 is of type Object
                                              //during runtime o2 is of type MethodOverrideVsOverload
  
  MethodOverrideVsOverload o3 = new MethodOverrideVsOverload(); //o3 is of type MethodOverrideVsOverload
                                                                // during both compile time and runtime
  MethodOverrideVsOverload o4 = new MethodOverrideVsOverload(); //o4 is of type MethodOverrideVsOverload
                                                                // during both compile time and runtime
  
  if(o1.equals(o2)){
   System.out.println("objects o1 and o2 are equal");
  }
  
  if(o3.equals(o4)){
   System.out.println("objects o3 and o4 are equal");
  }
 
 }

}

The output will be

MethodOverrideVsOverload equals method reached
objects o1 and o2 are equal
MethodOverrideVsOverload equals method reached
objects o3 and o4 are equal

This is because now the methods are overridden and this happens at runtime. This is a bit tricky question, and think out loud at the interview to show that you understand the fundamentals.

Labels:

Jun 26, 2012

Core Java coding questions frequently asked in written tests and interviews - part 2: equals Vs ==

Core Java Coding Questions and Answers for beginner to intermediate level

Q1 Q2 Q3 Q4 Q5 - Q8 Q9 Q10 Q11 Q12 - Q14 Q15


Q2. What will be the output of the following code snippet?

   Object s1 = new String("Hello");
   Object s2 = new String("Hello");

   if(s1 == s2) {
     System.out.println("s1 and s2 are ==");
   }else if (s1.equals(s2)) {
     System.out.println("s1 and s2 are equals()");
   }

A2. The answer is:


 s1 and s2 are equals()


Here is the explanation with a diagram.




So, the above question tests your understanding of "==" versus "equals( )" applied to objects in Java. One compares the references and the other compares the actual values.

Here are some follow on questions:

Q. What will be the output for the following code snippet?

 Object s1 = "Hello";
 Object s2 = "Hello";

 if (s1 == s2) {
  System.out.println("s1 and s2 are ==");
 } else if (s1.equals(s2)) {
  System.out.println("s1 and s2 are equals()");
 }
 
A. The answer is

s1 and s2 are ==

Now the above answer violates the rule we saw before. This is a special case (or rule) with the String objects and a typical example of the flyweight design pattern in action. This design pattern is used to conserve memory by reducing the number of objects created. The String object creates a pool of string and reuse an existing String object from the pool if the values are same instead of creating a new object. The flyweight pattern is all about object reuse. This rule applies only when you declare the String object as Object s = "Hello"; without using the keyword "new".

-->


This is a very popular Java  interview question.

More Java coding questions and answers


Labels:

Jun 25, 2012

Core Java coding questions frequently asked in written tests and interviews - part 1: Immutability and object references

Core Java Coding Questions and Answers for beginner to intermediate level

Q1 Q2 Q3 Q4 Q5 - Q8 Q9 Q10 Q11 Q12 - Q14 Q15

Some core java questions are very frequently asked in job interviews and written tests to ascertain if you know the Java fundamentals. I have covered a few questions that are not covered in my books. These coding questions will also clear up the fundamentals.

Q1. What will be the output of the following code snippet?

String s = " Hello ";
s += " World ";
s.trim( );
System.out.println(s);

A1. The output will be

" Hello  World "

with the leading and trailing spaces. Some would expect a trimmed "Hello World".
So, what concepts does this question try to test?
  1. String objects are immutable and there is a trick in s.trim( ) line.
  2. Understanding object references and unreachable objects that are eligible for garbage collection.
What follow on questions can you expect?
  1. You might get a follow on question on how many string objects are created in the above example and when will it become an unreachable object to be garbage collected.
  2. You might also be asked a follow on question as to if the above code snippet is efficient.
The best way to explain this is via a self-explanatory diagram as shown below.






If you want the above code to output "Hello World" with leading and trailing spaces trimmed then assign the s.trim( ) to the variable "s". This will make the reference "s" to now point to the newly created trimmed String object.

The above code can be rewritten as shown below


StringBuilder sb = new StringBuilder(" Hello ");
sb.append(" World ");
System.out.println(sb.toString().trim( ));


The StringBuilder is not a thread-safe class. It is fine when you are using it as a local variable. If you want to use it as an instance variable then use the StringBuffer class which is thread-safe. If you are curious to know what happens under the covers during String manipulation -- String concatenation

Relevant must get it right coding questions and answers

Labels: ,

Jun 20, 2012

Batch Processing in Java with Spring batch -- part 3

This assumes that you have read the spring batch part 1 & part 2. This is the final part.

Step 1: The annotated Java classes are referenced directly due to following line in the batch-context.xml
  
    <!-- Component-Scan automatically detects annotations in the Java classes.  -->
 <context:component-scan base-package="com.myapp.batch" />

Step 2: You define the batch job as shown below. It can have a number of steps. Some details are omitted to keep it simple.
 

  <batch:job id="availableBalanceJob">
        <batch:listeners>
      <!-- annotated listener using context:component-scan--> 
            <batch:listener ref="appJobExecutionListener" />
            <batch:listener ref="itemFailureLoggerListener" />
        </batch:listeners>
        <batch:step id="step1" parent="batchControlStep">
            <batch:end on="FAILED" />
            <batch:next on="*" to="step2" />
        </batch:step>
        <batch:step id="step2">
            <batch:tasklet>
                <batch:chunk
                    reader="accountReader"
                    processor="availableCashProcessor"
                    writer="availableCashWriter"
                    commit-interval="100"
                    retry-limit="1000000">
     <!-- deadlock retry -->
                    <batch:retryable-exception-classes>
                        <batch:include class="org.springframework.dao.DeadlockLoserDataAccessException"/>
                    </batch:retryable-exception-classes>
                </batch:chunk>
                <batch:listeners>
                    <batch:listener ref="batchItemListener" />
                </batch:listeners>
            </batch:tasklet>
        </batch:step>
    </batch:job>

Step 3: Provide the relevant annotated and referenced class implementations. The listener "appJobExecutionListener" is defined as shown below. As you can see the class is annotated with @Component("appJobExecutionListener").
 

package com.myapp.batch.listener;

import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.myapp.batch.dao.BatchControlDao;
import com.myapp.batch.domain.BatchControl;

@Component("appJobExecutionListener")
public class AppJobExecutionListener implements JobExecutionListener {

    @Autowired
    private BatchControlDao batchControlDao;

    public void beforeJob(JobExecution arg0) {
    }

    public void afterJob(JobExecution jobExecution) {
        Long batchControlId = (Long) jobExecution.getExecutionContext().get(BatchControl.JOB_KEY_BATCH_CONTROL_ID);
        if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
            batchControlDao.saveJobComplete(batchControlId, BatchStatus.COMPLETED);
            System.out.println("Job completed: " + jobExecution.getJobInstance().getJobName());
        } else if (jobExecution.getStatus() == BatchStatus.FAILED) {
            batchControlDao.saveJobComplete(batchControlId, BatchStatus.FAILED);
            System.out.println("Job failed: " + jobExecution.getJobInstance().getJobName() + " "
                    + jobExecution.getFailureExceptions());
        }
    }

}

The "itemFailureLoggerListener" can be defined in a similar fashion.
 
package com.myapp.batch.listener;

import org.springframework.batch.core.listener.ItemListenerSupport;
import org.springframework.stereotype.Component;

@Component("itemFailureLoggerListener")
public class ItemFailureLoggerListener extends ItemListenerSupport {

  
    public void onReadError(Exception ex) {
        System.out.println("Encountered error on read", ex);
    }

    public void onWriteError(Exception ex, Object item) {
        System.out.println("Encountered error on write", ex);
    }
}

The "accountReader" is the ItemReader that reads the relevant account numbers from the accounts table. It basically gets a list of items (i.e. Accounts) to be processed by the processor.
 

    <bean id="accountReader" class="org.springframework.batch.item.database.JdbcCursorItemReader">
        <property name="dataSource" ref="dataSourceMyDs" />
        <property name="sql">
            <value>
                <![CDATA[
                select a.account_no, a.account_name, a.debit, a.credit
                from accounts a  
                where a.account_no > ?  and
                      a.account_no >= ? and  
                      a.account_no <= ?         
                order by a.account_no
                ]]>
            </value>
        </property>
        <property name="rowMapper" ref="accountRowMapper" />
        <property name="preparedStatementSetter" ref="accountReaderStatementSetter" />
    </bean>

The AccountRowMapper is defined as follows to map the results
 

package com.myapp.batch.rowmapper;

import java.sql.ResultSet;
import java.sql.SQLException;

import org.springframework.jdbc.core.RowMapper;
import org.springframework.stereotype.Component;

import com.myapp.batch..domain.Account;

@Component("accountRowMapper")
public class AccountRowMapper implements RowMapper {
    
    public static final String ACC_NO = "account_no";
    //...
    
    public Account mapRow(ResultSet rs, int rowNum) throws SQLException {
        Account account = new Account();
        account.setAccNo(rs.getInteger(ACC_NO));
        //.....
        return account;
    }

}

The where clause arguments can be supplied as shown below by reading from the "jobExecutionContext"
 

     <bean id="accountReaderStatementSetter" scope="step" class="org.springframework.batch.core.resource.ListPreparedStatementSetter">
        <property name="parameters">
            <list>
                <value>#{jobExecutionContext['lastProcessedAccNo']}</value>
                <value>#{jobExecutionContext['accFrom']}</value>
                <value>#{jobExecutionContext['accTo']}</value>
            </list>
        </property>
    </bean>

 
The availableCashProcessor is the ItemProcessor that calculates the available cash balance. You can delegate this work to a number of processor classes as shown below. It loops through each item (i.e Account)
 

    <bean id="availableCashProcessor" class="org.springframework.batch.item.support.CompositeItemProcessor">
        <property name="delegates">
            <list>
                <bean class="com.myapp.batch.processor.CalculateCashValueProcessor" />
            </list>
        </property>
    </bean>
 
The processor class is shown below
 

package com.myapp.batch.processor;

import java.util.ArrayList;
import java.util.List;

import org.springframework.batch.item.ItemProcessor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;


@Component("availableCashProcessor")
public class CalculateValueProcessor implements ItemProcessor<Account, Account> {

 
    //the account read by the "accountReader" is passed in
    public Account process(Account account) {
        //simple and dirty without using BigDecimal
        account.setAvailableCash(account.getCredit() - account.getDebit());
        return account;
    }

}

The "availaavailableCashWriter" is the ItemWriter that writes the results.
 

package com.myapp.batch.writer;

import java.util.List;

import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.annotation.BeforeStep;
import org.springframework.batch.item.ExecutionContext;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

//...

@Component("availableCashWriter")
public class AvailableCashWriter implements ItemWriter<Account> {

    private StepExecution stepExecution;

    @Autowired
    private AccountDao accountDao; //injected via spring confih

    @Autowired
    private ModelAccountPortfolioValueDao modelAccountPortValueDao;

    public void write(List<? extends Account> items) throws Exception {
        for (Account account : items) {
            // Save portfolio value for account
            Account readAcc = accountDao.getAccount(account.getAccNo());
            readAcc.setAvalableCash(account.getAvailableCash());
            accountDao.updateAccount(readAcc);

            // Save to step execution context so that it can be promoted to job execution context
            ExecutionContext stepContext = this.stepExecution.getExecutionContext();
            stepContext.put(BatchControl.JOB_KEY_LAST_PROCESSED_ACC_NO, account.getAccNo());
        }
    }


    @BeforeStep
    public void setStepExecution(StepExecution stepExecution) {
        this.stepExecution = stepExecution;
    }
}

Finally, the batchItemListener that updates the lastProcessedAccNo.
 

package com.myapp.batch.listener;

import java.util.List;

import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.annotation.BeforeStep;
import org.springframework.batch.core.listener.ItemListenerSupport;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.myapp.batch.dao.BatchControlDao;
import com.myapp.batch.domain.BatchControl;

@Component("batchItemListener")
public class ItemBatchListener extends ItemListenerSupport {

    private StepExecution stepExecution;

    @Autowired
    private BatchControlDao batchControlDao;

    public void afterWrite(List items) {
        Long batchId =
                (Long) this.stepExecution.getJobExecution().getExecutionContext().get(
                        BatchControl.JOB_KEY_BATCH_ID);
        String lastProcessedAccNo =
                (String) this.stepExecution.getExecutionContext().get(BatchControl.JOB_KEY_LAST_PROCESSED_ACC_NO);
        batchControlDao.saveLastProcessedAccNo(lastProcessedAccNo, batchId);
    }

    @BeforeStep
    public void saveStepExecution(StepExecution stepExecution) {
        this.stepExecution = stepExecution;
    }
}

Labels:

Jun 19, 2012

Batch processing in Java with Spring batch - part 2

This post assumes that you have read the part-1 spring batch overview questions and answers. The data access objects, datasource configuration, and few other classes are left out for brevity as these were covered on other spring topics.

Step 1: Define the batch-context.xml with the JobLauncher and the JobRepository. JobLaunchers are responsible for starting a Job with a given job parameters. The spring provided implementation, SimpleJobLauncher, relies on a TaskExecutor to launch the jobs. If no specific TaskExecutor is set then the spring provided default SyncTaskExecutor is used for testing purpose. A JobRepository implementation requires a set of execution Daos to store its information. The MapJobRepositoryFactoryBean is a FactoryBean that automates the creation of a SimpleJobRepository using non-persistent in-memory DAO implementations. This repository is only really intended for use in testing and rapid prototyping. The JobRegistryBeanPostProcessor registers Job beans with a JobRegistry.


<!-- define the job repository -->
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
    <property name="transactionManager" ref="transactionManager" />
</bean>

<!-- define the launcher and pass the jobRepository as setter injection -->
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
    <property name="jobRepository" ref="jobRepository" />
</bean>


<bean id="jobRegistry" class="org.springframework.batch.core.configuration.support.MapJobRegistry" />

<bean id="jobRegistryBeanPostProcessor" class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
    <property name="jobRegistry" ref="jobRegistry" />
</bean>

Step 2: Define the steps necessary for the batch job. Firstly, define the batch control step that will update the batch_control table accordingly.
  
    <!-- Component-Scan automatically detects annotations in the Java classes.  -->
 <context:component-scan base-package="com.myapp.batch" />

    <batch:step id="batchControlStep">
        <batch:tasklet>
            <batch:chunk reader="batchControlReader" writer="jobStartBatchControlWriter" commit-interval="1" />
            <batch:listeners>
                <batch:listener ref="promotionListener" />
                <batch:listener ref="batchReaderListener" />
            </batch:listeners>
        </batch:tasklet>
    </batch:step>
    
 
    <bean id="batchControlReader" class="org.springframework.batch.item.database.JdbcCursorItemReader">
        <property name="dataSource" ref="dataSourceMyDs" />
        <property name="sql">
            <util:constant static-field="com.myapp.batch.dao.impl.BatchControlDaoImpl.SELECT_BY_JOB_NAME_SQL" />
        </property>
        <property name="rowMapper" ref="batchControlRowMapper" />
        <property name="preparedStatementSetter" ref="batchControlReaderStatementSetter" />
    </bean>

    <bean id="batchControlReaderStatementSetter" scope="step" class="org.springframework.batch.core.resource.ListPreparedStatementSetter">
        <!-- The parameter that is passed via command line: my_job_run.sh  accountValueUpdateJob1 accountValueUpdateJob1.log -->
  <!-- $JAVA_HOME/bin/java -classpath ${CLASSPATH} ${JOB_CLASS} batch-context.xml availableBalanceJob jobName=accountValueUpdateJob1" -->
  <property name="parameters">
            <list>
                <value>#{jobParameters['jobName']}</value>
            </list>
        </property>
    </bean>

 <!-- promotes the values read via static constants from one step to another step via the StepExecutionContext -->
    <bean id="promotionListener" class="org.springframework.batch.core.listener.ExecutionContextPromotionListener">
        <property name="keys">
            <list>
                <util:constant static-field="com.myapp.batch.domain.BatchControl.JOB_KEY_LAST_PROCESSED_ACC_NO" />
                <util:constant static-field="com.myapp.batch.domain.BatchControl.JOB_KEY_BATCH_ID" />
                <util:constant static-field="com.myapp.batch.domain.BatchControl.ACC_FROM" />
                <util:constant static-field="com.myapp.batch.domain.BatchControl.ACC_TO" />
            </list>
        </property>
    </bean>

 <!-- The listener class that has the  afterStep() method to print -- >
    <bean id="batchReaderListener" class="com.myapp.batch.listener.BatchReaderListener" />
 
Step 3: Define the relevant Java classes that are rereferenced in the above configuration file. Capture the batch control meta data as shown below.

public class BatchControl {

    public static final String JOB_KEY_LAST_PROCESSED_ACC_NO = "lastProcessedAccNo";
    public static final String JOB_KEY_BATCH_ID = "batchId";
    public static final String ACC_FROM = "accFrom";
    public static final String ACC_TO = "accTo";

    public static final String BATCH_USER = "batch";

    private Long jobId;
    private String jobName;
    private DateTime startDateTime;
    private DateTime endDateTime;
    private String status;
    private Integer accFrom;
    private Integer accTo;
    private Integer lastProcessedAccNo;
 
    //getters and setters are omitted
}

The listener class that prints the job status -- "COMPLETED" or "FAILED" after the step execution.
package com.myapp.batch.listener;

import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.step.NoWorkFoundStepExecutionListener;

public class BatchReaderListener extends NoWorkFoundStepExecutionListener {

    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
        ExitStatus exitStatus = super.afterStep(stepExecution);
        if (exitStatus != null && exitStatus.equals(ExitStatus.FAILED)) {
            System.out.println("Could not load batch control record with job name "
                    + stepExecution.getJobExecution().getJobInstance().getJobParameters().getString("jobName"));
        }
        return exitStatus;
    }
}
 
The Dao object that reads and updates the batch_control table.

package com.myapp.batch.dao.impl;

import org.joda.time.DateTime;
import org.springframework.batch.core.BatchStatus;
import org.springframework.jdbc.core.simple.ParameterizedBeanPropertyRowMapper;
import org.springframework.jdbc.core.simple.SimpleJdbcTemplate;
import org.springframework.orm.hibernate3.HibernateTemplate;
import org.springframework.orm.hibernate3.SessionFactoryUtils;
import org.springframework.orm.hibernate3.support.HibernateDaoSupport;

import com.myapp.batch.dao.BatchControlDao;
import com.myapp.batch.domain.BatchControl;

public class BatchControlDaoImpl extends HibernateDaoSupport implements BatchControlDao {
    
    public static final String SELECT_BY_JOB_NAME_SQL =
            "select job_id, job_name, start_timestamp, end_timestamp, status,"
                    + " account_no_from, account_no_to, last_account_no from batch_control where job_name = ?";
            
    private static final String JOB_START_UPDATE_SQL =
            "update batch_control set start_timestamp = ?, end_timestamp = ?, status = ?, last_account_no=?"
                    + " where batch_id = ?";

    private static final String JOB_COMPLETE_UPDATE_SQL =
            "update batch_control set end_timestamp = ?, status = ? where batch_id = ?";

    private static final String LAST_PROCESSED_ACC_UPDATE_SQL =
            "update batch_control set last_account_no = ? where batch_id = ?";

    private SimpleJdbcTemplate simpleJdbcTemplate;

    public BatchControlDaoImpl(HibernateTemplate hibernateTemplate) {
        setHibernateTemplate(hibernateTemplate);
        this.simpleJdbcTemplate =
                new SimpleJdbcTemplate(SessionFactoryUtils.getDataSource(hibernateTemplate.getSessionFactory()));
    }

    public void init(Long Id, Integer last_account_no) {
        simpleJdbcTemplate.update(JOB_START_UPDATE_SQL, new DateTime().toDate(), null, BatchStatus.STARTED.name(),
                last_account_no, batchId);
    }

    public void saveJobComplete(Long batchId, BatchStatus status) {
        simpleJdbcTemplate.update(JOB_COMPLETE_UPDATE_SQL, new DateTime().toDate(), status.name(), batchId);
    }

    public void saveLastProcessedId(String last_account_no, Long batchId) {
        simpleJdbcTemplate.update(LAST_PROCESSED_ACC_UPDATE_SQL, last_account_no, batchId);
    }

    public BatchControl getBatchControl(String jobName) {
        BatchControl batchControl =
                simpleJdbcTemplate.queryForObject(SELECT_BY_JOB_NAME_SQL, ParameterizedBeanPropertyRowMapper
                        .newInstance(BatchControl.class), jobName);
        return batchControl;
    }

}

This will be concluded in part-3.


Labels:

Jun 12, 2012

Server side versus client side templating

The modern Rich Internet Applications (RIA) use the design concept of "single page web design", where a single rich page makes ajax based service calls to render different sections of a page instead of the traditional approach of loading a new page of each user action.

The "single page web design" can make use of  client side and server side technologies as discussed below. Each approach has its use. It also shows the power of JavaScript.


Q. What do you understand by client side and server side templating?
A. The modern rich single page web applications built today harness the power of dynamically typed and interpreted languages like JavaScript for faster prototyping and increased developer productivity and statically typed Java based frameworks for maintainability, robustness, easier refactoring, and scalability. The modern browsers using faster JavaScript engines and efficient minification of JavaScript files have made client side templating a reality.

  • The server side templates are such as JSPs, Facelets for JSF, Apache Velocity, tiles, Sitemesh, etc. 
  • The client side templating libraries based on JavaScript include dust.js, underscore.js, mustache.js, etc 
  • The client side and server side templates like Google closure. 

 Server side templating and data composition 

Generates the markup on the server. For example, returns generated HTML code.



Client  side templating and data composition

Generates the markup on the client. More suited for applications that load more data from different back end service providers via ajax. The services could return data in JSON or XML format  and then add the HTML snippets for the new data via the client side templates.




Javascript templating is a technique to render templates on client-side (i.e. within the browser) with Javascript using a JSON data retrieved via RESTful web service calls. The template is basically HTML markup, sprayed with tags and variables that will insert values from the JSON data or execute a programming logic. For example,

1. Import jQuery template library


<script src="http://code.jquery.com/jquery.min.js" type="text/javascript"></script>
<script src="jquery.tmpl.js" type="text/javascript"></script>

2. Define JSON data to be used to fill the template values. This can be returned from the server via a RESTFul service call.

var carsData = [
    { make: "Toyota", model: "Camry" },
    { make: "BMW", model: "3 Series" },
    { make: "Mazda", model: "3" }
];

3. Define your client side template like

<script id="carsTemplate" type="text/html">
    <li><a href="cars/${make}">${model}</a></li>
</script>

or the complete HTML code may look like:

<html lang="en">
<head>
  ...
  <script src="http://code.jquery.com/jquery.min.js" type="text/javascript"></script>
  <script src="jquery.tmpl.js" type="text/javascript"></script>
  
  <script id="clientTemplate" type="text/html">
    <li>make: ${make}, model: ${model}</li>
  </script>
 
</head>
 
<body>
 
<ul></ul>
 
</body>
</html>

4. Invoking the template enginge (e.g jQuery) plugin to render the template with data

$("#carsTemplate").tmpl(carsData).appendTo( "ul" );

The  <li> tags will be appended to the <ul> tag with the retrieved JSON data values.


Q. What are the pros and cons of server side versus client side templating?
A. 

 Server Side Templating

Pros:

  • More search engine friendly.
  • Entire response message can be cached.
  • Can work without JavaScript

Cons:

  • Harder to mix and match different server side technologies. For example, retrieve data from both Java and Ruby based services
  • Harder to send content for multiple devices. E.g. browser, mobile devices, etc.


Client Side Templating


Pros:

  •  Faster development and prototyping
  •  Serves JSON data to multiple devices.
  •  The responses are smaller as JSON data is less verbose.
  •  Templates and JSON data can be cached.
  •  Can work with multiple server side technologies like .Net, JEE, Ruby, etc


Cons:

  •  Less search engine friendly
  •  Older browsers may not work properly
  •  Cross browser compatbility testing is required




Labels:

Jun 5, 2012

Batch processing in Java with Spring batch - part 1


Make sure that you do this beginner tutorial first

Beginner Spring batch tutorial with simple reader and writer

Q. What do you understand by batch processing and why do you need them?
A. The idea behind batch processing is to allow a program to run without the need for human intervention, usually scheduled to run periodically at a certain time or every x minutes.The batch process solves
  • execution that takes a lengthy amount of time (e.g. consolidated reports, re-balancing the financial portfolio, etc)
  • processing a large input set of data via ETL (Extract Transform and Load) operation.
  • automating some business process that doesn’t necessarily need to be triggered by a human.
In general, a batch process involve

  • Reading data from data sources like databases, files, etc
  • Processing the data through a number of steps. This can involve transforming the data, performing calculation, generating length reports, etc.
  • Finally, writing the processed data to databases or file systems.

Q. What libraries do you need to get started with spring batch
A. The pom.xml file will be the start. Fill in the appropriate versions and the additional dependencies required.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.myapp</groupId>
  <artifactId>mybatchapp</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>mybatchapp</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
        <groupId>org.springframework.batch</groupId>
        <artifactId>spring-batch-core</artifactId>
        <version>....</version>
    </dependency>
    <dependency>
        <groupId>commons-lang</groupId>
        <artifactId>commons-lang</artifactId>
        <version>....</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring</artifactId>
        <version>...</version>
    </dependency>
 
 ...other dependencies like hibernate
 
  </dependencies>
  ...
</project> 

Q. What are the key terms of Spring batch?  
A.


  • JobLauncher: It helps you to launch a job. It uses a JobRepository to obtain a valid JobExecution.
  • JobRepository: A persistent store for all the job meta-data information. Stores JobInstances, JobExecutions, and StepExecutions information to a database or a file system. The repository is required because a job could be a rerun of a previously failed job. 
  • Job: Represents a job. For example, an portfolio processing job that calculates the portfolio value. JobInstance: A running instance of a job. If a job runs every week night, then there will 5 instances of that job in a week. 
  • JobParameters: Parameters that are used by the JobInstance. For example, the portfolio processing job requires the list of account numbers as the input parameters. The job parameters are passed as command line arguments.
  • JobExecution: Every attempt to run a JobInstance results in a JobExecution. If a job fails then it will be rerun and we end up having a single JobInstance with 2 JobExecutions (1 failed execution and 1 successful execution). 
  • Step: Each job is made up of one or more steps. 
  • StepExecution: Similar to JobExecution, there are StepExecutions. This represents an attempt to run a Step in a Job.  

Q. Can you describe the key steps involved in a typical batch job scenario you had worked on?
A. A simplified batch process scenario is explained below.

The scenario is basically a batch job that runs overnight to go through all the accounts from the accounts table and calculates the available_cash by subtracting debit from the credit.

Step 1: Define a batch control table that keeps metdata about the batch job. The example below shows a batch_control table that process accounts in chunks. The account numbers 000 - 999 ara processed by a job and account_no 1000 - 1999 by another job. This table also holds information about when the job started, when the job finished, status (i.e. COMPLETED or FAILED), last processed account_no, etc.

job_id job_name start_timestamp end_timestamp status account_ no_from account_ no_to last_ account_no
1 accountValueUpdateJob1 21/04/2012 3:49:11.053 AM 21/04/2012 3:57:55.480 AM COMPLETED 000 999 845
2 accountValueUpdateJob2 21/04/2012 3:49:11.053 AM 21/04/2012 3:57:55.480 AM FAILED 1000 1999 1200

Step 2: To keep it simple, a single data table is used. You need to define a job and its steps. A job can have more than one steps. Each step can have a reader, processor, and a writer. An ItemReader will read data from the accounts table shown below for account numbers between account_from and account_to read from the batch_control table shown above. An ItemProcessor will calculate the availbale_cash and an ItemWriter will update the available_cash on the accounts table.

account_no account_name debit credit available_cash
001 John Smith 200.00 4000.0 0.0
1199 Peter Smith 55000.50 787.25 0.0

Step 3: Once the batch job is completed, the batch_contol table will be updated accordingly.

Step 4: Listeners can be used to process errors (e.g. onProcessError(....), onReadError(....), etc) and other pre and post item events like beforeRead(...), afterRead(...), etc. The spring-batch framework make use of the configuration xml file, for example batch-context.xml and the Java classes annotated with @Component to wire up the components and implement the logic.

Step 5: The batch job can be executed via a shell or batch script that invokes the spring-batch framework as shown below. The CommandLineJobRunner is the Spring class that initiates the job by wiring up the relevant components, listeners, daos, etc via the configuration file batch-context.xml. The job parameter that is passed is "accountValueUpdateJob1", which is used to retrieve the relevant job metatdata from the job control table.

my_job_run.sh  accountValueUpdateJob1 accountValueUpdateJob1.log
   $0                 $1                    $2

The my_job_run.sh looks something like
...
JOB_CLASS=org.springframework.batch.core.launch.support.CommandLineJobRunner
APPCONTEXT=batch-context.xml
SPRING_JOB=availableBalanceJob
CLASSPATH=$JAVA_HOME/bin:....................... 
JOB_TO_RUN=$1
...

# the jobParameter is jobName. It is passed via job script argument <jobNameToRun>.
$JAVA_HOME/bin/java -classpath ${CLASSPATH} ${JOB_CLASS} ${APPCONTEXT} ${SPRING_JOB} jobName=${JOB_TO_RUN}"

Step 6: The shell script (e.g. my_job_run.sh) or batch file will be invoked by a job scheduler like quartz or
Unix cron job at a particular time without any human intervention.

This is basically the big picture. The wiring up of spring-batch will be explained in a different post.




Labels: