FIX tutorial in Java with QuickFIX/j simple example

FIX and Java

FIX tutorial in Java with QuickFIX/j simple example

QuickFix Java example sending and receiving FIX messages

Finding your way around the FIX specification

If you like to work on trading applications, you need to know the  FIX protocol and a FIX engine like QuickFIX/J.

Q. What is FIX Protocol?
A. FIX stands for Financial Information eXchange, which is an open protocol intended to streamline electronic communications in the financial securities industry. Most of the exchanges use this standard for communication like sending Order, Executions, MarketData, etc. There are many versions of the specifications like 4.2, 4.4, etc. Have a look at https://fixspec.com/FIX44.

Let's have a look at QuickFIX/J and Java example for sending and recieving FIX messages.

Step 1: Required dependency JAR files

Step 2: The quickfixj-msg-fix44-1.5.3.jar contains the data dictionary for FIX4.4 with domain objects like and XML "NewSingleOrder". Extract FIX44.xml and copy it to c:\\temp.

Step 3: Define the configuration for the receiver or message acceptor. receiver.cfg file in /conf. Needs to be in the classpath.

[DEFAULT]
ConnectionType=acceptor
SocketAcceptPort=5001
SocketReuseAddress=Y
StartTime=00:00:00
EndTime=00:00:00
FileLogPath=log
FileStorePath=c:\\temp

[SESSION]
BeginString=FIX.4.4
SenderCompID=FixServer
TargetCompID=CLIENT1
DataDictionary=c:\\temp\\FIX44.xml

Step 4: Define the FIXReceiver class. It extends MessageCracker implements Application. Relevant method gets invoked in this event driven architecture used in MINA.

package com.example;

import quickfix.Application;
import quickfix.DoNotSend;
import quickfix.FieldNotFound;
import quickfix.IncorrectDataFormat;
import quickfix.IncorrectTagValue;
import quickfix.Message;
import quickfix.RejectLogon;
import quickfix.SessionID;
import quickfix.UnsupportedMessageType;
import quickfix.fix44.MessageCracker;
import quickfix.fix44.NewOrderSingle;

public class FIXReceiver extends MessageCracker implements Application {

 @Override
 public void onMessage(NewOrderSingle order, SessionID sessionID) {
  System.out.println("Receiver onMessage..  " + order); 
 }
 
 @Override
 public void fromAdmin(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat,
  IncorrectTagValue, RejectLogon {
 }

 @Override
 public void fromApp(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, 
 IncorrectTagValue, UnsupportedMessageType {
  System.out.println("Receiver fromApp..  " + arg0);
  crack(arg0, arg1); // calls onMessage(..,..)
 }

 @Override
 public void onCreate(SessionID arg0) {
  System.out.println("Receiver onCreate.. " + arg0);
 }

 @Override
 public void onLogon(SessionID arg0) {
  System.out.println("Receiver onLogon.." + arg0);
 }

 @Override
 public void onLogout(SessionID arg0) {}

 @Override
 public void toAdmin(Message arg0, SessionID arg1) {}

 @Override
 public void toApp(Message arg0, SessionID arg1) throws DoNotSend {}
}

Step 5: The server socket code to receive messages. ReceiverApp.Java

package com.example;

import java.util.Scanner;

import quickfix.Application;
import quickfix.ConfigError;
import quickfix.DefaultMessageFactory;
import quickfix.FileStoreFactory;
import quickfix.ScreenLogFactory;
import quickfix.SessionSettings;
import quickfix.SocketAcceptor;

public class RecieverApp {
 public static void main(String[] args) throws ConfigError {
  
  SessionSettings settings = new SessionSettings("receiver.cfg");
  Application myApp = new FIXReceiver();
  FileStoreFactory fileStoreFactory = new FileStoreFactory(settings);
  ScreenLogFactory screenLogFactory = new ScreenLogFactory(settings);
  DefaultMessageFactory msgFactory = new DefaultMessageFactory();
  SocketAcceptor acceptor = new SocketAcceptor(myApp, fileStoreFactory,
    settings, screenLogFactory, msgFactory);
  
  acceptor.start();
  
  Scanner reader = new Scanner(System.in);
  System.out.println("press <enter> to quit");
  
  //get user input for a
  reader.nextLine();
  
  acceptor.stop();
 }

}

Step 5: The sender or initiator configuration file sender.cfg.

[DEFAULT]
ConnectionType=initiator
HeartBtInt=30
ReconnectInterval=0
FileStorePath=c:\\temp
FileLogPath=log
StartTime=00:00:00
EndTime=00:00:00
UseDataDictionary=N
SocketConnectHost=localhost
ContinueInitializationOnError=Y

[SESSION]
BeginString=FIX.4.4
SenderCompID=CLIENT1
TargetCompID=FixServer
SocketConnectPort=5001

Step 6: Define the FixSender class that implements Application. Relevant method gets invoked in this event driven architecture used in MINA.

package com.example;

import quickfix.Application;
import quickfix.DoNotSend;
import quickfix.FieldNotFound;
import quickfix.IncorrectDataFormat;
import quickfix.IncorrectTagValue;
import quickfix.Message;
import quickfix.RejectLogon;
import quickfix.SessionID;
import quickfix.UnsupportedMessageType;

public class FIXSender implements Application {

 @Override
 public void fromAdmin(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, 
 IncorrectTagValue, RejectLogon {
 }

 @Override
 public void fromApp(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, 
         IncorrectTagValue, UnsupportedMessageType { }

 @Override
 public void onCreate(SessionID arg0) {}

 @Override
 public void onLogon(SessionID arg0) {}

 @Override
 public void onLogout(SessionID arg0) {}

 @Override
 public void toAdmin(Message arg0, SessionID arg1) {}

 @Override
 public void toApp(Message msg, SessionID sessionId) throws DoNotSend {
  System.out.println("Sender toApp: " + msg.toString());
 }
}

Step 7: The client socket initiator to send FIX messages. SenderApp.java.

package com.example;

import java.util.Date;

import quickfix.Application;
import quickfix.ConfigError;
import quickfix.DefaultMessageFactory;
import quickfix.FileStoreFactory;
import quickfix.ScreenLogFactory;
import quickfix.Session;
import quickfix.SessionID;
import quickfix.SessionNotFound;
import quickfix.SessionSettings;
import quickfix.SocketInitiator;
import quickfix.field.ClOrdID;
import quickfix.field.OrdType;
import quickfix.field.OrderQty;
import quickfix.field.Price;
import quickfix.field.Side;
import quickfix.field.Symbol;
import quickfix.field.TransactTime;
import quickfix.fix44.NewOrderSingle;

public class SenderApp {

 public static void main(String[] args) throws ConfigError, InterruptedException, SessionNotFound {

  SessionSettings settings = new SessionSettings("sender.cfg");
  Application myApp = new FIXSender();
  FileStoreFactory fileStoreFactory = new FileStoreFactory(settings);
  ScreenLogFactory screenLogFactory = new ScreenLogFactory(settings);
  DefaultMessageFactory msgFactory = new DefaultMessageFactory();
  SocketInitiator initiator = new SocketInitiator(myApp, fileStoreFactory, settings, 
                                       screenLogFactory, msgFactory);

  initiator.start();

  Thread.sleep(3000);

  // matching values from sender.cfg
  SessionID sessionID = new SessionID("FIX.4.4", "CLIENT1", "FixServer");
  NewOrderSingle order = new NewOrderSingle(new ClOrdID("DLF"), new Side(Side.BUY), 
    new TransactTime(new Date()), new OrdType(OrdType.LIMIT));

  order.set(new OrderQty(45.00));
  order.set(new Price(25.40));
  order.set(new Symbol("BHP"));
  Session.sendToTarget(order, sessionID);
  
  Thread.sleep(60000);

  initiator.stop();
 }
}

Step 8: Run the ReceiverApp.

log4j:WARN No appenders could be found for logger (quickfix.SessionSchedule).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
<20140731-01:59:36, FIX.4.4:FixServer->CLIENT1, event> (Session FIX.4.4:FixServer->CLIENT1 schedule is daily, 00:00:00-UTC - 00:00:00-UTC)
<20140731-01:59:36, FIX.4.4:FixServer->CLIENT1, event> (Created session: FIX.4.4:FixServer->CLIENT1)
Receiver onCreate.. FIX.4.4:FixServer->CLIENT1
press <enter> to quit

Step 9: Run the SenderApp.

log4j:WARN No appenders could be found for logger (quickfix.SessionSchedule).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
<20140731-01:59:44, FIX.4.4:CLIENT1->FixServer, event> (Session FIX.4.4:CLIENT1->FixServer schedule is daily, 00:00:00-UTC - 00:00:00-UTC)
<20140731-01:59:44, FIX.4.4:CLIENT1->FixServer, event> (Created session: FIX.4.4:CLIENT1->FixServer)
<20140731-01:59:45, FIX.4.4:CLIENT1->FixServer, outgoing> (8=FIX.4.4 9=72 35=A 34=52 49=CLIENT1 52=20140731-01:59:45.635 56=FixServer 98=0 108=30 10=203 )
<20140731-01:59:45, FIX.4.4:CLIENT1->FixServer, event> (Initiated logon request)
<20140731-01:59:45, FIX.4.4:CLIENT1->FixServer, incoming> (8=FIX.4.4 9=72 35=A 34=41 49=FixServer 52=20140731-01:59:45.672 56=CLIENT1 98=0 108=30 10=202 )
<20140731-01:59:45, FIX.4.4:CLIENT1->FixServer, event> (Received logon)
Sender toApp: 8=FIX.4.4 9=123 35=D 34=53 49=CLIENT1 52=20140731-01:59:47.671 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140731-01:59:47.669 10=238 
<20140731-01:59:47, FIX.4.4:CLIENT1->FixServer, outgoing> (8=FIX.4.4 9=123 35=D 34=53 49=CLIENT1 52=20140731-01:59:47.671 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140731-01:59:47.669 10=238 )
<20140731-02:00:15, FIX.4.4:CLIENT1->FixServer, incoming> (8=FIX.4.4 9=60 35=0 34=42 49=FixServer 52=20140731-02:00:15.918 56=CLIENT1 10=145 )
<20140731-02:00:18, FIX.4.4:CLIENT1->FixServer, outgoing> (8=FIX.4.4 9=60 35=0 34=54 49=CLIENT1 52=20140731-02:00:18.604 56=FixServer 10=143 )
<20140731-02:00:46, FIX.4.4:CLIENT1->FixServer, incoming> (8=FIX.4.4 9=60 35=0 34=43 49=FixServer 52=20140731-02:00:46.916 56=CLIENT1 10=148 )
<20140731-02:00:48, FIX.4.4:CLIENT1->FixServer, event> (Initiated logout request)
<20140731-02:00:48, FIX.4.4:CLIENT1->FixServer, outgoing> (8=FIX.4.4 9=60 35=5 34=55 49=CLIENT1 52=20140731-02:00:48.603 56=FixServer 10=151 )
<20140731-02:00:48, FIX.4.4:CLIENT1->FixServer, incoming> (8=FIX.4.4 9=60 35=5 34=44 49=FixServer 52=20140731-02:00:48.607 56=CLIENT1 10=153 )
<20140731-02:00:48, FIX.4.4:CLIENT1->FixServer, error> (quickfix.SessionException Logon state is not valid for message (MsgType=5))
<20140731-02:00:48, FIX.4.4:CLIENT1->FixServer, event> (Already disconnected: Verifying message failed: quickfix.SessionException: Logon state is not valid for message (MsgType=5))

Step 10: The receiver or acceptor logs grow further to

<20140731-01:59:45, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=72 35=A 34=52 49=CLIENT1 52=20140731-01:59:45.635 56=FixServer 98=0 108=30 10=203 )
<20140731-01:59:45, FIX.4.4:FixServer->CLIENT1, event> (Accepting session FIX.4.4:FixServer->CLIENT1 from /127.0.0.1:60832)
<20140731-01:59:45, FIX.4.4:FixServer->CLIENT1, event> (Acceptor heartbeat set to 30 seconds)
<20140731-01:59:45, FIX.4.4:FixServer->CLIENT1, event> (Received logon)
<20140731-01:59:45, FIX.4.4:FixServer->CLIENT1, event> (Responding to Logon request)
<20140731-01:59:45, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=72 35=A 34=41 49=FixServer 52=20140731-01:59:45.672 56=CLIENT1 98=0 108=30 10=202 )
Receiver onLogon..FIX.4.4:FixServer->CLIENT1
<20140731-01:59:47, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=123 35=D 34=53 49=CLIENT1 52=20140731-01:59:47.671 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140731-01:59:47.669 10=238 )
Receiver fromApp..  8=FIX.4.4 9=123 35=D 34=53 49=CLIENT1 52=20140731-01:59:47.671 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140731-01:59:47.669 10=238 
Receiver onMessage..  8=FIX.4.4 9=123 35=D 34=53 49=CLIENT1 52=20140731-01:59:47.671 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140731-01:59:47.669 10=238 
<20140731-02:00:15, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=60 35=0 34=42 49=FixServer 52=20140731-02:00:15.918 56=CLIENT1 10=145 )
<20140731-02:00:18, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=60 35=0 34=54 49=CLIENT1 52=20140731-02:00:18.604 56=FixServer 10=143 )
<20140731-02:00:46, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=60 35=0 34=43 49=FixServer 52=20140731-02:00:46.916 56=CLIENT1 10=148 )
<20140731-02:00:48, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=60 35=5 34=55 49=CLIENT1 52=20140731-02:00:48.603 56=FixServer 10=151 )
<20140731-02:00:48, FIX.4.4:FixServer->CLIENT1, event> (Received logout request)
<20140731-02:00:48, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=60 35=5 34=44 49=FixServer 52=20140731-02:00:48.607 56=CLIENT1 10=153 )
<20140731-02:00:48, FIX.4.4:FixServer->CLIENT1, event> (Sent logout response)

Java and XML tutorial with SAX parser - reading only

Q. What is a SAX Parser and when will you use it?
A. SAX  (Simple API for XML) is a stream oriented API like StAX, but

• Unlike StAX, which can read and write, but SAX can only read.
• Unlike StAX, which can't validate schema, SAX can perform schema validation.

SAX works by iterating over the XML and call certain methods on a "handler" object , which listens to events like startDocument, startElement, characters, comments, processing instructions, endElement, and endDocument.

StAX your handler "pulls" the XML events out of the parser.

SAX  your parser pushes events into the handler

One major advantage of StAX over SAX is that the pull model allows subparsing of the XML input. You can extract out the element name, then the attributes, and then the characters (i.e. content)


Here is the sample code to read the employee XML.

package com.xml;

import java.io.ByteArrayInputStream;
import java.io.FileInputStream;
import java.io.InputStream;

import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.helpers.DefaultHandler;

public class SaxProcessing {

 public static void main(String[] args) {
  SAXParserFactory factory = SAXParserFactory.newInstance();
  try {
   String xml = "<Employee><name type=\"first\">Peter</name><age>25</age></Employee>";

   SAXParser saxParser = factory.newSAXParser();
   DefaultHandler handler = new SaxHandler();
   saxParser.parse(new ByteArrayInputStream(xml.getBytes()), handler);

  } catch (Throwable err) {
   err.printStackTrace();
  }
 }

}

Now the handler class that gets pushed with the events

package com.xml;

import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

public class SaxHandler extends DefaultHandler {
 public void startDocument() throws SAXException {
  System.out.println("Start document   : ");
 }

 public void endDocument() throws SAXException {
  System.out.println("End document     : ");
 }

 public void startElement(String uri, String localName, String qName,
   Attributes attributes) throws SAXException {
  System.out.println("Start element    : " + qName);
  if (qName.equalsIgnoreCase("name")) {
   String attrName = attributes.getQName(0);
   String attrValue = attributes.getValue(0);
   System.out.println("Attribute        : " + attrName + "=" + attrValue);
  }
 }

 public void endElement(String uri, String localName, String qName) 
     throws SAXException {
  System.out.println("End element      : " + qName);
 }

 public void characters(char ch[], int start, int length) throws SAXException {
  System.out.println("Start characters : " + new String(ch, start, length));
 }
}

Output:

Start document   : 
Start element    : Employee
Start element    : name
Attribute        : type=first
Start characters : Peter
End element      : name
Start element    : age
Start characters : 25
End element      : age
End element      : Employee
End document     : 

Java and XML tutorial with StAX parser - writing cursor based and iterator based

package com.xml;

import java.math.BigDecimal;

import org.springframework.core.style.ToStringStyler;

public class Employee {
 private String name;
 private String type;
 private int age;
 private BigDecimal salary;
 public String getName() {
  return name;
 }
 public void setName(String name) {
  this.name = name;
 }
 public String getType() {
  return type;
 }
 public void setType(String type) {
  this.type = type;
 }
 public int getAge() {
  return age;
 }
 public void setAge(int age) {
  this.age = age;
 }
 public BigDecimal getSalary() {
  return salary;
 }
 public void setSalary(BigDecimal salary) {
  this.salary = salary;
 }
 
 @Override
 public String toString() {
  StringBuilder sb = new StringBuilder();
  sb.append("name=" + name);
  sb.append("\n");
  if(type != null) {
   sb.append("type=" + type + "\n");
  }
  sb.append("age=" + age);
  return sb.toString();
 }
}

Here is a simple example to write employee object to XML using simpler cursor based approach.

package com.xml;

import java.io.StringWriter;
import java.math.BigDecimal;

import javax.xml.stream.XMLOutputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;

public class StaxProcessing {

 public static void main(String[] args) {

  XMLOutputFactory factory = XMLOutputFactory.newInstance();

  // construct employee object
  Employee emp = new Employee();
  emp.setName("Peter");
  emp.setType("first");
  emp.setAge(25);
  emp.setSalary(BigDecimal.valueOf(35000.00));

  StringWriter sw = new StringWriter();

  try {
   XMLStreamWriter writer = factory.createXMLStreamWriter(sw);
   writer.writeStartDocument();
   writer.writeStartElement("Employee");
   writer.writeStartElement("name");
   writer.writeAttribute("type", "first");
   writer.writeCharacters(emp.getName());
   writer.writeEndElement();
   writer.writeStartElement("age");
   writer.writeCharacters(new Integer(emp.getAge()).toString());
   writer.writeEndElement();
   writer.writeStartElement("salary");
   writer.writeCharacters(emp.getSalary().toString());
   writer.writeEndElement();
   writer.writeEndElement();

   System.out.println(sw.getBuffer().toString());

  } catch (XMLStreamException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }

 }
}

Output:

<?xml version="1.0" ?>
<Employee><name type="first">Peter</name><age>25</age><salary>35000.0</salary></Employee>

Here is a simple example to write employee object to XML using simpler iterator based approach.

package com.xml;

import java.io.StringWriter;
import java.math.BigDecimal;

import javax.xml.stream.XMLEventFactory;
import javax.xml.stream.XMLEventWriter;
import javax.xml.stream.XMLOutputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.events.XMLEvent;

public class StaxProcessing {

 public static void main(String[] args) {

  XMLOutputFactory factory = XMLOutputFactory.newInstance();
  XMLEventFactory eventFactory = XMLEventFactory.newInstance();

  // construct employee object
  Employee emp = new Employee();
  emp.setName("Peter");
  emp.setType("first");
  emp.setAge(25);
  emp.setSalary(BigDecimal.valueOf(35000.00));

  StringWriter sw = new StringWriter();

  try {
   XMLEventWriter writer = factory.createXMLEventWriter(sw);
   XMLEvent event = eventFactory.createStartDocument();
   writer.add(event);

   // employee
   event = eventFactory.createStartElement("", "", "Employee");
   writer.add(event);

   // name
   event = eventFactory.createStartElement("", "", "name");
   writer.add(event);
   event = eventFactory.createAttribute("type", "first");
   writer.add(event);
   event = eventFactory.createCharacters(emp.getName());
   writer.add(event);
   event = eventFactory.createEndElement("", "", "name");
   writer.add(event);

   // age
   event = eventFactory.createStartElement("", "", "age");
   writer.add(event);
   event = eventFactory.createCharacters(new Integer(emp.getAge()).toString());
   writer.add(event);
   event = eventFactory.createEndElement("", "", "age");
   writer.add(event);

   // salary
   event = eventFactory.createStartElement("", "", "salary");
   writer.add(event);
   event = eventFactory.createCharacters(emp.getSalary().toString());
   writer.add(event);
   event = eventFactory.createEndElement("", "", "salary");
   writer.add(event);

   // close employee
   event = eventFactory.createEndElement("", "", "Employee");
   writer.add(event);

   writer.flush();
   writer.close();

   System.out.println(sw.getBuffer().toString());

  } catch (XMLStreamException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }

 }
}

Output:

<?xml version="1.0" ?>
<Employee><name type="first">Peter</name><age>25</age><salary>35000.0</salary></Employee>

Why this Java blog? Are you feeling stagnated or finding it hard to get good Java jobs?

I can think of 8 reasons

1. As a journal of my valuable experience for future reference, which otherwise would be lost as Java/JEE technologies, frameworks, libraries, and tools are very vast.
2. As a motivator to proactively learn, apply, and experience new concepts, frameworks, and tools without feeling stagnated. Just knowing Java alone is never enough in this fiercely competitive market. You need to have Java + FIX engine + Spring batch, Java + Drools + JasperReports + Angular JS, Java + Hadoop, and so on. The prospective employers are very picky.
3. As a confidence booster through pre interview, pre group sharing, and pre problem solving refresher. Preparation breeds confidence, and confidence can bring success.
4. As a job security provider by empowering me to be in a position to find new contracts as a freelancer through interview preparation and online presence. Real job security is to be able to find job in any market with sought-after and marketable skills.
5. As a passive income generator. Can I bring in some income while I am not actively working? Can I bring in some income while I am in between jobs or contracts?
6. As a good will to share my experience. You can learn and understand more by helping others. Especially, proactively solving others' technical problems.
7. As a future door opener to new ventures, collaborations, and opportunities. You never know who is going to stumble across your blog. Your regular readers may become your future employers or customers.
8. As a passion☺. Prospective employers look for your passion in the job interviews. So, do you have a blog? Do you have a self-taught project in GitHub? Can you proudly explain how you went about fixing performance issues, thread-safety issues, memory issues, etc? Can you pick potential errors during peer code reviews?,  Can you pass the coding job interviews that given to you by some of the top-notch companies that take hiring quality developers very seriously? etc

In short, your online presence can tell a lot about you.


How will you benefit from this Java blog?

If the answer is "Yes" to any one of the following questions, then you will benefit from this Java blog

• Do you want to stand out from other Java developers who are as qualified as you are or even more qualified than you are by presenting yourself in a better light?

• Do you feel frustrated that a fellow Java professional who discharges similar duties as you earning 1.5 to 2.0 time as you are?


• Do you feel nervous at the job interviews or about an impending code review?


• Do you feel that your job interview performance is not the true reflection of what you are really capable of?


• Are you feeling stagnated or not having much luck trying to break into trading systems, low latency systems, or financial applications?


• Do you feel that every other developer is certified, and you want to get an edge by learning a new framework or tool like FIX engines, Apache Camel, Drools, etc or getting a good handle on the 16 key areas?


• Do you want to impress your peers and superiors at work during code review sessions and solving complex issues to earn a reputation as go to person?


• Are you entrusted with hiring quality Java candidates and want to prepare a list of well rounded Java interview questions to ask?

If your answered  "Yes" to any of the above questions, then preparation and pro-active learning via this blog will help you. This blog is full of Java interview questions and Answers, Java career tips, Java know hows, How to use Java tools, and Java tutorials. Preparation breeds confidence, and confidence bring Java-Success.



It is no easy to take your career to the next level. It requires the proper "know hows" and perseverance. You will benefit from not only technical know hows, but also non technical know hows including resume writing, job hunting, blogging, interviewing,  and marketing.

Feel free to request for any topics that I have missed out or any interview question, how to do in Java or tutorial not covered here. Always looking for more blog posts to write in addition to 450+ posts.

Also, thanks for your support as this blog post has reached 4 million+ page views with 1200+ followers, and 30,000+ books sold.

Java and XML tutorial with StAX parser - reading

Q. What is a StAX Parser and when will you use it?
A. The StAX Java API for XML processing is designed for parsing XML streams, just like the SAX API's, but

• StAX is a "pull" API whereas SAX is a "push" API.
• StAX can do both XML reading and writing whereas SAX can only do XML reading.

Q. Why use StAX when there is DOM?

A. The DOM parsing involves creating in-memory objects representing an entire document tree for an XML document. Once in memory, DOM trees can be navigated and parsed arbitrarily, providing the maximum flexibility for developers. However this flexibility comes at a cost of large memory footprint and significant processor requirements, as the entire representation of the document must be held in memory as objects for the duration of the document processing. This may not be an issue when working with small documents, but memory and processor requirements can escalate quickly for larger size documents.

StAX (Streaming Api for XML) involves streaming where streaming refers to a programming model where XML data sets are transmitted and parsed serially at application runtime as events like StartElementEvent, CharacterEvent, EndElementEvent, etc, often in real time and dynamically where contents are not precisely known beforehand. These stream-based parsers can start generating output immediately, and data set elements can be discarded and garbage collected immediately after they are used. This ensures smaller memory footprint, reduced processor requirements, and higher performance in certain scenarios. This memory and processing efficiency comes at the cost where streaming can only see the  data set state at one location at a time in the document.

Here is how the events are sequentially processed:

Q. What is the major advantage of StAX over SAX?
A. The major advantage of StAX over SAX is that the pull model allows sub parsing of the XML input. You can extract out the element name, then the attributes, and then the characters (i.e. content)

Here is a simple code example to read the Employee from XML:

package com.xml;

import java.io.StringReader;

import javax.xml.namespace.QName;
import javax.xml.stream.XMLEventReader;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.events.Attribute;
import javax.xml.stream.events.EndElement;
import javax.xml.stream.events.StartElement;
import javax.xml.stream.events.XMLEvent;

public class StaxProcessing {

 public static void main(String[] args) {
  XMLInputFactory factory = XMLInputFactory.newInstance();
  String xml = "<Employee><name type=\"first\">Peter</name><age>25</age></Employee>";

  XMLEventReader eventReader;

  try {
   eventReader = factory.createXMLEventReader(new StringReader(xml));
   Employee emp = null;

   while (eventReader.hasNext()) {
    // read node
    XMLEvent event = eventReader.nextEvent();

    if (event.isStartElement()) {
     StartElement startElement = event.asStartElement();
     // root node
     if (startElement.getName().getLocalPart().equalsIgnoreCase("Employee")) {
      System.out.println(startElement.getName().getLocalPart());
      System.out.println("Before adding a new node ----------------");
      emp = new Employee();
     }
     if (startElement.getName().getLocalPart().equalsIgnoreCase("name")) {
      // get attribut values
      Attribute attr = startElement.getAttributeByName(new QName("type"));
      if(attr != null && attr.getName().toString().equals("type")) {
          emp.setType(attr.getValue());
      }
      // element data
      event = eventReader.nextEvent();
      emp.setName(event.asCharacters().getData());

     } else if (startElement.getName().getLocalPart().equalsIgnoreCase("age")) {
      // element data
      event = eventReader.nextEvent();
      emp.setAge(new Integer(event.asCharacters().getData()));
     }

    }

    if (event.isEndElement()) {
     EndElement endElement = event.asEndElement();
     if (endElement.getName().getLocalPart().equalsIgnoreCase("Employee")) {
      System.out.println(emp);
     }
    }

   }

  } catch (XMLStreamException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }

 }
}

package com.xml;

import java.math.BigDecimal;

public class Employee {
 private String name;
 private String type;
 private int age;
 private BigDecimal salary;
 public String getName() {
  return name;
 }
 public void setName(String name) {
  this.name = name;
 }
 public String getType() {
  return type;
 }
 public void setType(String type) {
  this.type = type;
 }
 public int getAge() {
  return age;
 }
 public void setAge(int age) {
  this.age = age;
 }
 public BigDecimal getSalary() {
  return salary;
 }
 public void setSalary(BigDecimal salary) {
  this.salary = salary;
 }
 
 @Override
 public String toString() {
  StringBuilder sb = new StringBuilder();
  sb.append("name=" + name);
  sb.append("\n");
  if(type != null) {
   sb.append("type=" + type + "\n");
  }
  sb.append("age=" + age);
  return sb.toString();
 }
}

Output

Employee
Before adding a new node ----------------
name=Peter
type=first
age=25

In the next post will cover writing employee object back to XML.

You have two approaches: cursor based and iterator based.

Java and JAXB with MOXy implementation tutorial to convert XML to object and vice versa

Q. Why favor MOXy JAXB implementation?
A. JAXB is a specification for which there are many implementations like JDK default, MOXy, Metro, and JaxMeAPI. MOXy is from EclipseLink  enhances mapping with

1. @XmlPath extension which is inspired from XPath
2. The Jersey JAX-RS reference implementation provides JSON binding via MOXy.
3. MOXy is already the default JAXB implementation in WebLogic 12.1.1


Q. How to enable MOXy?
A.

Step 1: You need eclipselink-xxx.jar in your classpath. So, include in pom.xml dependency.

<dependency>
    <groupid>org.eclipse.persistence</groupId>
    <artifactid>eclipselink</artifactId>
    <version>2.5.1</version>
</dependency>

Step 2: You need  jaxb.properties in your classpath with the following entry.

javax.xml.bind.context.factory=org.eclipse.persistence.jaxb.JAXBContextFactory

Verify?

JAXBContext context = JAXBContext.newInstance(Employee.class);
System.out.println("JAXB Context is=" + context.toString());

Output:

JAXB Context is=org.eclipse.persistence.jaxb.JAXBContext@4501b7af

Now, remap Employee covered in tutorial without the nested Name object. The @XmlPath annotation from the MOXy to the rescue.


Step 1: The employee XML file

<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

Step 2: The Employee  object with annotations to map to the above XML. Note the @XmlPath annotation with XPath syntax.

package com.xml;

import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlTransient;
import javax.xml.bind.annotation.XmlType;

import org.eclipse.persistence.oxm.annotations.XmlPath;

@XmlRootElement(name = "Employee")
@XmlType(propOrder = { "name", "type", "age" })
public class Employee {

 private int id;
 private String name;
 private String type;
 private int age;

 @XmlTransient
 public int getId() {
  return id;
 }

 public void setId(int id) {
  this.id = id;
 }

 @XmlElement
 public String getName() {
  return this.name;
 }

 public void setName(String name) {
  this.name = name;
 }

 @XmlAttribute
 @XmlPath("name/@type")
 public String getType() {
  return type;
 }

 public void setType(String type) {
  this.type = type;
 }

 public int getAge() {
  return age;
 }

 public void setAge(int age) {
  this.age = age;
 }

 @Override
 public String toString() {
  StringBuilder sb = new StringBuilder();
  sb.append("name=" + this.name);
  if (type != null) {
   sb.append("\ntype=" + type);
  }
  sb.append("\n");
  sb.append("age=" + age);
  return sb.toString();
 }
}

Step 3: Marshalling and unmarshalling with MOXy.

package com.xml;

import java.io.ByteArrayInputStream;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;
import javax.xml.bind.Unmarshaller;

public class JaxbMarshallUnMarshallWithMoxy {

 public static void main(String[] args) {

  String xml = "<Employee><name type=\"first\">Peter</name><age>25</age></Employee>";

  try {

   JAXBContext context = JAXBContext.newInstance(Employee.class);
   
   //Ensuring that we are using MOXy JAXB implementation instead of JDK default
   System.out.println("JAXB Context is=" + context.toString());

   // unmarshalling - XML to Java object
   Unmarshaller un = context.createUnmarshaller();
   Employee emp = (Employee) un.unmarshal(new ByteArrayInputStream(xml.getBytes()));

   System.out.println("Object: " + emp);

   // Marshalling - Java object to XML
   Marshaller m = context.createMarshaller();
   // for pretty-print XML in JAXB
   m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
   m.marshal(emp, System.out);

  } catch (JAXBException e) {
   e.printStackTrace();
  }

 }
}

Output:

JAXB Context is=org.eclipse.persistence.jaxb.JAXBContext@4501b7af
Object: name=Peter
type=first
age=25
<?xml version="1.0" encoding="UTF-8"?>
<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

Top 10 most common Core Java beginner mistakes

Mistake #1: Using floating point data types like float or double for monetary calculations. This can lead to rounding issues.

package com.monetary;
import java.math.BigDecimal;

public class MonetaryCalc {
 
    public static void main(String[] args) {
      System.out.println(1.05 - 0.42); //1:  $0.6300000000000001
       //2: $0.63
      System.out.println(new BigDecimal("1.05") .subtract(new BigDecimal("0.42")));
      //3: $0.630000000000000059952043329758453182876110076904296875
      System.out.println(new BigDecimal(1.05) .subtract(new BigDecimal(0.42)));
      //4: $0.63
      System.out.println(BigDecimal.valueOf(1.05) .subtract(BigDecimal.valueOf(0.42)));
      System.out.println(105 - 42); //5: 63 cents
   }
}

In the above code, 2, 4, and 5 are correct and 1 and 3 are incorrect usage leading to rounding issues. So, either use BigDecimal properly as shown in 2 and 4, or use the smallest monetary value like cents with data type long. The cents approach performs better, and handy in applications that have heavy monetary calculations.

Mistake #2: Using floating point variables like float or double in loops to compare for equality. This can lead to infinite loops.

 public static void main(String[] args) {
         float sum = 0;
         while (sum != 1.0) { //causes infinite loop.
             sum += 0.1;
         }
         System.out.print("The sum is: "+sum);
 }

Fix is to change while (sum != 1.0) to while (sum < 1.0). Even then, due to mistake #1, you will get rounding issues (e.g. The sum is: 1.0000001). So, the better fix is

   public static void main(String[] args) {
         BigDecimal sum = BigDecimal.ZERO;
         while (sum.compareTo(BigDecimal.ONE)  != 0) { 
             sum =  sum.add(BigDecimal.valueOf(0.1)); 
         }
         System.out.print("The sum is: "+sum); //The sum is: 1.0
 }

Mistake #3: Not properly implementing the equals(..) and hashCode( ) methods. Can you pick if anything wrong with the following Person class.

public class Person  {

 private String name;
 private Integer age;

 public Person(String name, Integer age) {
  this.name = name;
  this.age = age;
 }

 //getters and setters

 @Override
 public String toString() {
  return "Person [name=" + name + ", age=" + age + "]";
 } 
}

Now, the main class that creates a collection of Person, and then searches for a person.

import java.util.Collections;
import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.CopyOnWriteArraySet;

public class PersonTest {
 
 public static void main(String[] args) {
  
  Set<Person> people = new CopyOnWriteArraySet<>();
  people.add(new Person("John", 35));
  people.add(new Person("John", 32));
  people.add(new Person("Simon", 30));
  people.add(new Person("Shawn", 30));
  
  System.out.println( people);
  
  Person search = new Person("John", 35);
  
  if(people.contains(search)){
   System.out.println("found: " + search);
  }
  else {
   System.out.println("not found: " + search);
  }
 
 }

}

The output

[Person [name=John, age=35], Person [name=John, age=32], Person [name=Simon, age=30], Person [name=Shawn, age=30]]
not found: Person [name=John, age=35]

Q. Why is the person not found even though there in the collection?
A. Every Java object implicitly extends the Object class, which has the default implementation of equals(...) and hashCode( ) methods. The equals(...) method is  implicitly invoked by the Set class's contains(...) method. In this case the default implementation in the Object class performs a shallow comparison of the references, and not the actual values like name and age. The equals and hashCode methods in Java are meant for overridden for POJOs. So, this can be fixed as shown below by overriding the equals and hashCode methods in Person.

import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;

public class Person  {

 private String name;
 private Integer age;

 public Person(String name, Integer age) {
  this.name = name;
  this.age = age;
 }

 //getters and setters

 @Override
 public int hashCode() {
  return new HashCodeBuilder().append(name).append(age).hashCode();
 }

 @Override
 public boolean equals(Object obj) {
  Person rhs = (Person) obj;
  return new EqualsBuilder().append(this.name, rhs.name).append(this.age, rhs.age).isEquals();
 }

 @Override
 public String toString() {
  return "Person [name=" + name + ", age=" + age + "]";
 }
 
}

Now, output will be

[Person [name=John, age=35], Person [name=John, age=32], Person [name=Simon, age=30], Person [name=Shawn, age=30]]
found: Person [name=John, age=35]

There are other subtle issues you can face if the hashCode and equals contracts are not properly adhered to. Refer to the Object class API for the contract details.

When sorting objects in a collection with compareTo, it is strongly recommended, but not strictly required that (x.compareTo(y)==0) == (x.equals(y)). Generally speaking, any class that implements the Comparable interface and violates this condition should clearly indicate this fact.

Mistake #4: Getting a ConcurrentModificationException  when trying to modify (i.e. adding or removing an item) a collection while iterating.

The following code throws a ConcurrentModificationException.

List<T> list = getListOfItems();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
  T obj = iter.next();
  if (obj.someCondition()) {
    list.remove(0); // ConcurrentModificationException
  }
}

To avoid ConcurrentModificationException in a single-threaded environment, you can remove the object that you are working on with the iterator.

List<T> list = getListOfItems();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
  T obj = iter.next();
  if (obj.someCondition()) {
    iter.remove(); //OK to use the iterator
  }
}

To Avoid ConcurrentModificationException in a multi-threaded environment:

If you are using JDK1.5 or higher then you can use ConcurrentHashMap and CopyOnWriteArrayList classes. This is the recommended approach compared to other approaches like locking the list with synchronized(list) while iterating, but this approach defeats the purpose of using multi-threading.

Q. Difference between fail-fast and fail-safe iterators
A. Iterators returned by most of pre JDK1.5 collection classes like Vector, ArrayList, HashSet, etc are fail-fast iterators. Iterators returned by JDK 1.5+ ConcurrentHashMap and CopyOnWriteArrayList classes are fail-safe iterators.


Mistake #5: Common exception handling mistakes.

a) Sweeping exceptions under the carpet by doing nothing.

try{
 
}catch(SQLException sqe){
    // do nothing
}

In few rare scenarios, it is desired to do nothing with an exception, e.g. in finally block, we try to close database connection, and some exception occurs. In this case, exception can be ignored.

try{
 
}catch(SQLException sqe){
    ...
    ...
}finally{
    try{
        conn.close();
    }catch(Exception e){
        //leave it.
    }
}

But in general, this is a very bad practice and can hide issues.

b) Inconsistent use of checked and unchecked (i.e. Runtime) exceptions.

Document a consistent exception handling strategy. In general favor unchecked (i.e. Runtime) exceptions, which you don't have to handle with catch and throw clauses. Use checked exceptions in a rare scenarios where you can recover from the exception like deadlock or service retries. In this scenario, you will catch a checked exception like java.io.IOException and wait a few seconds as configured with the retry interval, and retry the service configured by the retry counts.

c) Exceptions are polymorphic in nature and more specific exceptions need to be caught before the generic exceptions. 

So, it is wrong to catch Exception before IOException.

try{
   //....
} catch(Exception ex){
    log.error("Error:" + ex)
} catch(IOException ex){
    log.error("Connectivity issue:" + ex); //never reached as Exception catch block catches everything
}

Fix this by catching the more specific IOException first

try{
   //....
} catch(IOException ex){
    log.error("Connectivity issue:" + ex);
} catch(Exception ex){
    log.error("Error:" + ex)
} 

In Java 7 on wards, you can catch multiple exceptions like

try{
   //....
} 
catch (ParseException | IOException exception) {
    // handle I/O problems.
} catch (Exception ex) {
    //handle all other exceptions
}

d) Wiping out the stack trace

try{
    //....
}catch(IOException ioe){
    throw new MyException("Problem in data reading."); //ioe stack is lost
}

e) Unnecessary Exception Transformation. 

In a layered application, many times each layer catches exception and throws new type of exception. Sometimes it is absolutely unnecessary to transform an exception. An unchecked exception can be automatically bubbled all the way upto the GUI Layer, and then handled at the GUI layer with a log.error(ex) for the log file and a generic info like "An expected error has occurred, and please contact support on xxxxx" to the user. Internal details like stack trace with host names, database table names, etc should not be shown to the user as it can be exploited to cause security threats.

Data Access Layer --> Business Service Layer --> GUI Layer

Mistake #6: Using System.out.println(... ) statements for debugging without making use of the debugging capabilities provided in IDE tools. If you need to log, use log4j library instead.

if(log.isDebugEnabled()) {
    log.debug("..................");
}
//....
log.info("................");
//...
log.error(ex);

The log4j library will not only perform better than System.out.println statements, but also provides lots of additional features like writing to a file, console, queue, etc, archiving and rolling the log files, controlling the log levels with debug, warn, info, error, etc and many more.

Mistake #7: Reinventing the wheel by writing your own logic when there are already well written and proven APIs and libraries are available. When coding, always have Core Java APIs, Apache APIs, Spring framework APIs, Google Gauva library APIs, and relevant reference documentations handy to reuse them instead of writing your own half baked solutions.

Instead of:

if(str != null && str.trim().length() > 0) {
   //..............
}

You can use the StringUtils library from the Apache commons API to simplify your code

if(!StringUtils.isEmpty(str){
    //...........
}

EqualsBuilder, HashCodeBuilder, StringBuilder, etc are Apache commons utility methods.


Mistake #8: Resource leak issues are reported when resources are allocated but not properly disposed (i.e. closed) after use.

The system resources like file handles, sockets, database connections, etc are limited resources that need to be closed once done with them otherwise your applications run the risk of leaking resources and then running out of sockets, file handles, or database connections.

Bad code: if an exception is thrown before in.close() is reached, the Scanner that is holding on to the System.in resource will never get closed.

public void readShapeData() throws IOException {
    Scanner in = new Scanner(System.in);
    log.info("Enter salary: ");
    salary = in.nextDouble();
    in.close();
}

Good code: The finally block is reached even if an exception is thrown. So, the scanner will be closed.

public void readShapeData() throws IOException {
    Scanner in = new Scanner(System.in);
    try {
        log.info("Enter salary: ");
        salary = in.nextDouble();
    } finally {
        in.close();  //reached even when an exception is thrown
    }
}

Mistake #9: Comparing two objects ( == instead of .equals)

When you use the == operator, you are actually comparing two object references, to see if they point to the same object. You cannot compare, for example, two strings or objects for equality, using the == operator. You must instead use the .equals method, which is a method inherited by all classes from java.lang.Object.


Mistake #10: Confusion over passing by value, and passing by reference as Java has both primitives like int, float, etc and objects.

When you pass a primitive data type, such as a char, int, float, or double, to a function then you are passing by value, which means a copy of the data type is duplicated, and passed to the function.If the function chooses to modify that value, it will be modifying the copy only.

When you pass a Java object, such as an array or an Employee object, to a function then you are passing by reference, which means the reference is copied, but both the original and copied references point to the same object. Any changes you make to the object's member variables will be permanent - which can be either good or bad, depending on whether this was what you intended.


5 bonus mistakes:

1. Forgetting that Java is zero-indexed.
2. Not writing thread-safe code with proper synchronization or thread-local objects. 
3. Not favoring immutable objects. Immutable objects are inherently thread-safe.
4. Not properly handling null references, and causing the ubiquitous NullPointerException. This is a run time exception and the compiler can't warn you. 
5. Capitalization errors where the class names and Java file names should start with capital letters and method and variable names should start with lowercase letters.

Java and JAXB tutorial to map and convert XML to Java object and vice versa in a nested scenario

Q. What is JAXB, and where is it used?
A. JAXB is an Object o XML Mapping (i.e. OXM) library for Java. Hibernate is an Object to Relational Mapping (i.e. ORM) tool for relational databases, likewise JAXB is an ORM for XML documents. Most often used alongside of web services (JAX-WS and JAX-RS ) when XML is used as a payload to convert XML to Java objects and vice versa. Web services predominantly use XML or JSON (JavaScript Object Notation) as the payloads.

Step 1: The XML document

<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

Step 2: The Java object with the relevant JAXB annotations to map fields to XML. Some elements are not annotated with @XmlElement and they are implicitly mapped. Employee and Name are nested objects.

package com.xml;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlTransient;
import javax.xml.bind.annotation.XmlType;

@XmlRootElement(name = "Employee")
@XmlType(propOrder = { "name", "age" })
public class Employee {

 private int id;
 private Name name;
 private int age;

 @XmlTransient
 public int getId() {
  return id;
 }

 public void setId(int id) {
  this.id = id;
 }

 @XmlElement
 public Name getName() {
  return this.name;
 }

 public void setName(Name name) {
  this.name = name;
 }

 public int getAge() {
  return age;
 }

 public void setAge(int age) {
  this.age = age;
 }

 @Override
 public String toString() {
  StringBuilder sb = new StringBuilder();
  sb.append(name);
  sb.append("\n");
  sb.append("age=" + age);
  return sb.toString();
 }
}

Name is a nested class as the "type" attribute belongs to name XML element and not "Employee" element.

package com.xml;

import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlValue;

public class Name {

 private String name;
 private String type;

 @XmlValue
 public String getName() {
  return name;
 }

 public void setName(String name) {
  this.name = name;
 }

 @XmlAttribute
 public String getType() {
  return type;
 }

 public void setType(String type) {
  this.type = type;
 }

 @Override
 public String toString() {
  StringBuilder sb = new StringBuilder();
  sb.append("name=" + this.name);
  if (type != null) {
   sb.append("\ntype=" + type);
  }
  return sb.toString();
 }
}

Step 3: The conversion from and and to object.

package com.xml;

import java.io.ByteArrayInputStream;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;
import javax.xml.bind.Unmarshaller;

public class JaxbMarshallUnMarshall {

 public static void main(String[] args) {

  String xml = "<Employee><name type=\"first\">Peter</name><age>25</age></Employee>";

  try {

   JAXBContext context = JAXBContext.newInstance(Employee.class);

   // unmarshalling - XML to Java object
   Unmarshaller un = context.createUnmarshaller();
   Employee emp = (Employee) un.unmarshal(new ByteArrayInputStream(xml.getBytes()));

   System.out.println("Object: " + emp);

   // Marshalling - Java object to XML
   Marshaller m = context.createMarshaller();
   // for pretty-print XML in JAXB
   m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
   m.marshal(emp, System.out);

  } catch (JAXBException e) {
   e.printStackTrace();
  }

 }
}

Output:

Object: name=Peter
type=first
age=25
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Employee>
    <name type="first">Peter</name>
    <age>25</age>
</Employee>

JAXB is a specification like JDBC, JNDI, JPA, etc, and the above example uses the default JDK implementation of JAXB. In the next post, I will  cover MOXy implementation, which has @XmlPath annotation to simplify your Object to XML mapping.

Java with SVNKit to programmatically checkout and update files

Q. What is SVNKit?
A. SVNKit is an Open Source, pure Java software library for working with the Subversion version control system.

Q. What libraries are required?
A.

Use the latest version from the SVNKit website or Maven repository.

Here is the sample code:

package test;

import java.io.File;

import org.tmatesoft.svn.core.SVNDepth;
import org.tmatesoft.svn.core.SVNURL;
import org.tmatesoft.svn.core.auth.BasicAuthenticationManager;
import org.tmatesoft.svn.core.auth.ISVNAuthenticationManager;
import org.tmatesoft.svn.core.wc.SVNClientManager;
import org.tmatesoft.svn.core.wc.SVNRevision;
import org.tmatesoft.svn.core.wc.SVNUpdateClient;
import org.tmatesoft.svn.core.wc.SVNWCUtil;

public class SVNTest {
 
 private static SVNClientManager ourClientManager;

 public static void main(String[] args) throws Exception {
  SVNURL url = SVNURL.parseURIDecoded("http://svn-host/svn/myapp/artifacts/trunk/XSD/");
  ISVNAuthenticationManager authManager = new BasicAuthenticationManager("NTADMIN\\name", "pwd");
  
  ourClientManager = SVNClientManager.newInstance(SVNWCUtil.createDefaultOptions(true));
  SVNUpdateClient updateClient = ourClientManager.getUpdateClient();
  
  
  File dstPath = new File("c:\\Temp\\myapp");
  SVNRevision pegRevision = SVNRevision.HEAD;
  SVNRevision revision = SVNRevision.HEAD;
  SVNDepth depth = SVNDepth.IMMEDIATES;
  boolean allowUnversionedObstructions = true;
  
  long res = 0;
  if (new File("c:\\Temp\\wts").list().length == 0) {
   res = updateClient.doCheckout(url , dstPath , pegRevision , revision, depth , allowUnversionedObstructions );
  } else { 
   res = updateClient.doUpdate(dstPath, revision, depth, allowUnversionedObstructions, true);
  } 
 }

}

The above code is quick and dirty for familiarization with SVNKit.

Q. Where to use SVNKit?
A.

• SVNKit is widely used in different development tools like IDEs (IntelliJ IDEA, Eclipse Subversion integrations, SmartSVN, JDeveloper, etc) and SDLC tools like Atlassian JIRA and FishEye to name a few. 
• You can use it on your own home grown  automation activities relating SDLC, one off data migration tasks and batch jobs. 

Java and XML tutorial with DOM Parser

Q. What is a DOM parser, and when will you use it?
A. The Java DOM API for XML parsing is intended for working with small to medium XML documents as an object graph in memory. The DOM parser traverses the XML file and creates the corresponding DOM objects linked together in a tree structure. Once the parser is done, you get this DOM object structure back from it. Then you can traverse the DOM structure back and forth, and also insert new nodes. If you have large XML files, favor SAX parser if only read only or a StAX parser if you want to read and write.

Step 1: Sample XML to read and insert new element into.

<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

Step 2: The Java DOM code to traverse and write to above XML.

package com.xml;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.Iterator;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;

import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class DOMProcessing {

 public static void main(String[] args) {
  DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
  DocumentBuilder builder = null;
  try {
   builder = builderFactory.newDocumentBuilder();
   String xml = "<Employee><name type=
      \"first\">Peter</name><age>25</age></Employee>";
   Document document = builder.parse(new ByteArrayInputStream(xml.getBytes()));

   // read root node
   Element rootElement = document.getDocumentElement();
   System.out.println(rootElement.getNodeName());

   // read child nodes
   NodeList childNodes = rootElement.getChildNodes();
   System.out.println("Before adding a new node ----------------");
   print(childNodes);

   //add a new node
   if (rootElement.getNodeName().equalsIgnoreCase("Employee")) {
    Node nodeNew = document.createElement("salary");
    nodeNew.setTextContent("35000.00");
    rootElement.appendChild(nodeNew);
   }
   
   System.out.println("After adding a new node ----------------");
   print(childNodes);

  } catch (ParserConfigurationException e) {
   e.printStackTrace();
  } catch (SAXException e) {
   e.printStackTrace();
  } catch (IOException e) {
   e.printStackTrace();
  }
 }

 private static void print(NodeList childNodes) {
  for (int i = 0; i < childNodes.getLength(); i++) {
   Node node = childNodes.item(i);
   System.out.println(node.getNodeName() + "=" + node.getTextContent());
   if (node instanceof Element && node.getNodeName().equalsIgnoreCase("name")) {
    // a child element to process
    Element child = (Element) node;
    String attribute = child.getAttribute("type");
    System.out.println("type=" + attribute);
   }
  }
 }
}

Output:

Employee
Before adding a new node ----------------
name=Peter
type=first
age=25
After adding a new node ----------------
name=Peter
type=first
age=25
salary=35000.00

As you can see, the "salary" is the new node that has been added.

11 tips to writing low latency real-time applications in Java

Q. Have you seen job advertisements requiring Java candidates to work in real-time or high volume transaction processing systems? Wondering what questions you will be asked?

Real-time and low-latency are distinctly separate subjects although often related. Real-time is about being more predictable than fast. Low latency systems need to be fast to meet SLAs (Service Level Acceptances) in sub milliseconds (e.g. micro seconds).

Tip #1: Use a RTSJ (Real Time Specification for Java ) JVM.  IBM, Oracle, and other smaller vendors have implemented this, but it comes at a cost. Oracle's JavaRT, IBM's real-time WebSpere, and aicas JamaicaVM to name a few popular ones. In real time JVM, instead of writing java.lang.Thread you just have to write, javax.realtime.RealtimeThread.

Tip #2: Big O notation for algorithms: Ensure all your data structures related algorithms are O(1) or at least O(log n). This is probably the biggest cause of performance issues. Make sure that you have performance tests with real size data. Also, make sure that your algorithms are cache friendly. It is imperative to use proper cache strategies to minimize garbage collection pauses by having proper cache expiry strategy, using weak references for cache, reducing cache by carefully deciding what to cache, increasing the cache size along with the heap memory to reduce object eviction from cache, etc.  Understanding Big O notations through Java examples

Tip #3: Lock free:  Use lock free algorithms and I/O.  Even the most well designed concurrent application that uses locks is at risk of blocking. For example, the java.util.concurrent package that allows concurrent reads and the Java NIO (New I/O using non-blocking multiplexers) respectively. Blocking is not good for low latency applications. Minimize context switching among threads by having threads not more than the number of CPU cores in your machine.

Tip #4: Reduce memory size: Reduce the number of objects you create. Apply the flyweight design pattern where applicable. Favor stateless objects. Where applicable write immutable objects that can be shared between threads. Fewer objects mean lesser GC.

Tip #5: Tune your JVM:  Tune your JVM with appropriate heap sizes and GC configuration. Before tuning profile your application with real life data. Basically you want to avoid GC pauses and increase GC throughput. GC throughput is a measure of % of time not spent on GC over a long period of time.  Specialist GC collectors like the Azul collector can in many cases solve this problem for you out of the box, but for many you who use the Oracle's GC, you need to understand how GC works and tune it to minimize the pauses. The default JVM options optimize for throughput, and latencies could be improved by switching to the Concurrent Garbarge Collector.

GC tuning is very application specific. It is imperative to understand following topics

-- You need to first understand how your application uses the garbage collection. Memory is cheap and abundant on modern servers, but garbage collector pauses is a serious obstacle for using larger memory sizes.  You should configure GC so that

• Enable diagnostic options (-XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintGCTimestamps).
• Decide the total amount of memory you can afford for the JVM by graphing your own performance metric against young generation sizes to find the best setting.
• Make plenty of memory available to the younger (i.e eden) generation. The default is calculated from NewRatio and the -Xmx setting.
• Make the survival space to be same size as Eden (-XX:SurvivorRatio=1) and increase new space to account for growth of the survivor spaces  (-XX:MaxNewSize= -XX:NewSize=) 
• Larger younger generation spaces increase the spacing between full GCs. But young space collections could take a proportionally longer time. In general, keep the eden size between one fourth and one third the maximum heap size. The old generation must be larger than the new generation.

Tip #6: Favor primitives to wrapper classes to eliminate auto-boxing and un-boxing: In situations where getter and setter methods are called very frequently for the wrapper classes like Integer, Float, Double, etc the performance is going to be adversely impacted due to auto boxing and unboxing. The operations like x++ will also provide poor performance if x is an Integer and not an int. So, avoid using wrappers in performance critical loops.


Tip #7: Good caching strategy and applying the short-circuit pattern

Short-circuit pattern is handy for I/O related patterns like socket or URL based, database operations, and complex File I/O operations. I/O operations need to complete within a short amount of time, but with low latency Web sites, the short-circuit pattern can be applied to time-out long running I/O tasks, and then can either display an error message or show cached results.


Tip #8: Coding best practices to avoid performance issues due to death by 1000 cuts.

• When using arrays it is always efficient to copy arrays using System.arraycopy( ) than using a loop. The following example shows the difference.
• When using short circuit operators place the expression which is likely to evaluate to false on extreme left if the expression contains &&.
• Do not use exception handling inside loops.
• Avoid using method calls to check for termination condition in a loop.
• Short-circuit equals( ) in large object graphs where it compares for identity first

@Override
public boolean equals(Object other) {
    if (this == other) return true;
    if (other == null) return false;
 
    // Rest of equality logic...
}

Tip #9: Experience and knowledge with some of  the libraries like

These libraries are aimed at providing reduced memory size, less impact on GC, lock free concurrent processing, data structure algorithmic efficiency, etc.

• NIO-based scalable server applications by directly using java.nio package or framework like Apache MINA.
• FIX protocol and commercial FIX libraries like Cameron FIX.
• Use  Java 5 concurrency utilities, and locks.
• Lock free Java disruptor library for high throughput.
• Chronicle Java library for low latency and high throughput, which almost uses no heap, hence has trivial impact on GC.
• Trove collection libraries for primitives. Alternative for the JDK wrapper classes like java.lang.Integer for primitives requiring less space and providing better performance.
• Javolution library with real-time classes. For example, Javolution XML provides real-time marshaling and unmarshaling.

Tip #10: How is your data stored? Are you using a SQL database? How will that scale? Can you use a NoSQL data tore instead.  Transactional systems need SQL for transaction demarcation.

Relational and NoSQL data models are very different.

SQL Model:

The relational model takes data and  store them in many normalized interrelated tables that contain rows and columns. Tables relate with each other through foreign keys.  When looking up data, the desired information needs to be collected by joining many related tables and combined before it can be provided to the application.

NoSQL Model 

NoSQL databases have a very different model. NoSQL databases have been built from the ground up to be distributed, scale-out technologies and therefore fit better with the highly distributed nature of the three-tier Internet architecture. A document-oriented NoSQL database takes the data you want to store and aggregates it into documents using the JSON format. Each JSON document can be thought of as an object to be used by your application. This might relate to data agregated from 10+ tables in an SQL model.

Tip #11: Pay attention to network round trips, payload sizes and type, protocols used, service timeouts and retries.

6 tips to writing loosely coupled Java applications with examples

Q. What is tight coupling?
A. If class OrderServiceImpl relies on parts of class PaymentServiceImpl that are not part of class PaymentServiceImpl's interface PaymentService, then the OrderServiceImpl and PaymentServiceImpl are said to be tightly coupled. In other words, OrderServiceImpl knows more than what it should about the way in which PaymentServiceImpl was implemented. If you want to change PaymentServiceImpl with a separate implementations BasicPaymentServiceImpl, then you need to modify the  OrderServiceImpl class as well by changing PaymentServiceImpl to BasicPaymentServiceImpl

Tip #1: Coding to interface will loosely couple classes. 

Q. What is loose coupling?
A. If the only knowledge that class OrderServiceImpl has about class PaymentServiceImpl, is what class PaymentServiceImpl has exposed through its interface PaymentService, then class OrderServiceImpl and class PaymentServiceImpl are said to be loosely coupled. If you want to change PaymentServiceImpl with a separate implementations BasicPaymentServiceImpl, then you don't need to modify OrderServiceImpl. Change only OrderServiceMain from

PaymentService payService  = new PaymentServiceImpl();

to

PaymentService payService  = new BasicPaymentServiceImpl();

This is what the Dependency Inversion Principle (DIP) states.

Q. Can the above classes be further improved in terms of coupling?
A. Yes. Change the PaymentService method signature from

   public abstract void handlePay(int accountNumber, BigDecimal amount);

to

     public abstract void handlePay(PaymentDetail paymentDetail);

Tip #2: Design method signatures carefully by avoiding long parameter lists. As a rule, three parameters should be viewed as a practical maximum, and fewer is better (as recommended by Mr. Joshua Bloch.). This is not only from coupling perspective, but also in terms of readability and maintainability of your code.

It is likely that the PaymentService may need more parameters than account number and amount to process the payment. Every time you need to add a new parameter, your PaymentService interface  method signature will change, and all other classes like OrderService that depends on PaymentService has to change as well to change its arguments to passed. But, if you create a value object like PaymentDetail, the method signature does not have to change. You add the new field to the PaymentDetail class.

Tip #3: Design patterns promote looser coupling

The PaymentService will not only be used by the OrdersServiceMain, but can be used by other classes like RequestServiceMain, CancelServiceMain, etc. So, if you want to change the actual implementation of PaymentService between PaymentServiceImpl and BasicPaymentServiceImpl without having to change OrdersServiceMain, RequestServiceMain, and CancelServiceMain, you can use the factory design pattern as shown by the PaymentFactory class. You only have to make a change to the PaymentFactory class to return the right PaymentService implementation.

package com.coupling;

import java.math.BigDecimal;

public class OrderServiceMain {
 
 public static void main(String[] args) {
  //loosely coupled as it knows only about the factory
  PaymentService payService  = PaymentFactory.getPaymentService();
  OrderService orderService = new OrderServiceImpl(payService);
  orderService.process(12345, BigDecimal.valueOf(250.00));
 }
}

package com.coupling;

public final class PaymentFactory {

 private static PaymentService instance = null;
 
 private PaymentFactory(){}
 
 public static PaymentService getPaymentService(){
  if(instance == null){
   instance = new PaymentServiceImpl();
  }
  
  return instance;
 }
 
}

Tip #4: Using Inversion of Control (IoC) Containers like Spring, Guice, etc. 

Dependency Injection (DI) is a pattern of injecting a class’s dependencies into it at run time. This is achieved by defining the dependencies as interfaces, and then injecting in a concrete class implementing that interface to the constructor. This allows you to swap in different implementations without having to modify the main class. The Dependency Injection pattern also promotes high cohesion by promoting the  Single Responsibility Principle (SRP), since your dependencies are individual objects which perform discrete specialized tasks like data access (via DAOs) and business services (via Service and Delegate classes).

The Inversion of Control  (IoC) container is a container that supports Dependency Injection. In this you use a central container like Spring framework, Guice, or HiveMind, which defines what concrete classes should be used for what dependencies throughout your application. This brings in an added flexibility through looser coupling, and it makes it much easier to change what dependencies are used on the fly. The basic concept of the Inversion of Control pattern is that you do not create your objects but describe how they should be created.

You don't directly connect your components and services together in code but describe which services are needed by which components in a configuration file. A container (in the case of the Spring framework, the IOC container) is then responsible for hooking it all up. Applying IoC, objects are given their dependencies at creation time by some external entity that coordinates each object in the system. That is, dependencies are injected into objects. So, IoC means an inversion of responsibility with regard to how an object obtains references to collaborating objects.

For example, in Spring you will wire up the dependencies via an XML file:

    <bean id="orderService" class="com.coupling.OrderServiceImpl">
  <constructor-arg ref="paymentService"/> 
    </bean>
 
    <bean id="paymentService" class="com.coupling.PaymentServiceImpl" />

You can also use annotations to inject dependencies. The @Resource annotation injects PaymentService.

package com.coupling;

import java.math.BigDecimal;
import javax.annotation.Resource;

public class OrderServiceImpl implements OrderService {

 @Resource
 PaymentService payService;

 public OrderServiceImpl(PaymentService payService) {
  this.payService = payService;
 }

 public void process(int accountNumber, BigDecimal amount) {
  // some logic
  payService.handlePay(new PaymentDetail(accountNumber, amount));
  // some logic
 }
}

Tip #5: High cohesion often correlates with loose coupling, and vice versa.

What is cohesion? Cohesion is the extent to which two or more parts of a system are related and how they work together to create something more valuable than the individual parts. You don't want a single class to perform all the functions (or concerns) like being a domain object, data access object, validator, and a service class with business logic. To create a more cohesive system from the higher and lower level perspectives, you need to break out the various needs into separate classes like PaymentDetail, PaymentService, PaymentDao, PaymentValidator, etc. Each class concentrates on one thing.

Coupling happens in between classes or modules, whereas cohesion happens within a class. So, think, tight encapsulation, loose (low) coupling, and high cohesion.


Tip #6: Favor composition over inheritance for code reuse

You will get a better abstraction with looser coupling with composition as composition is dynamic and takes place at run time compared to implementation inheritance, which is static, and happens at compile-time.  The guide is that inheritance should be only used when subclass ‘is a’ super class. Don’t use inheritance just to get code reuse. If there is no ‘is a’ relationship then use composition for code reuse.  Overuse of implementation inheritance (uses the “extends” key word) can break all the subclasses, if the super class is modified. Do not use inheritance just to get polymorphism. If there is no ‘is a’ relationship and all you want is polymorphism then use interface inheritance with composition, which gives you code reuse. More elaborate explanation on this -- Why favor composition over inheritance in Java OOP?


You may also like How to create well designed Java classes?

Which Jar has a particular class? or from which jar a class was loaded? solving jar hell issues

The following 3 questions are frequently asked by Java developers as an industrial strength Java project will have 100+ jar files. How often have you come across a Java application that requires different versions of the same library? How often do you see exceptions like NoSuchMethodError or IllegalArgumentException. Here are some tips to solve the JAR hell problem. These 6 tips will go a long way in resolving your jar problems.

Q1. Which Jar has a particular class?

Tip#1: go to findJAR.com and search for the class file. For example, I want to find the jar file that has org.apache.commons.io.FileUtils

You can drill through to find a Maven download link.

Tip#2: Unix command "find" and grep

find ./lib -name "*.jar" -exec sh -c 'jar -tf {}|grep -H --label {} 'org.apache.commons.io.FileUtils'' \;

Tip#3: In Windows or DOS

 
for %i in (*.jar) do @jar tvf %i | find "org/apache/commons/io/FileUtils.class"

Q2. from which jar a class was loaded?

Tip#4. To identify from which jar a particular class was loaded from, add the following snippet of code to  a location where it gets executed.

   Class klass = org.apache.commons.io.FileUtils.class;

   CodeSource codeSource = klass.getProtectionDomain().getCodeSource();

    if ( codeSource != null) {
        System.out.println(codeSource.getLocation());
    }

At run time it will print the jar file from which "FileUtils" was loaded.

package test;

import java.security.CodeSource;

public class WhichJarLoadedTest {

 public static void main(String[] args) {
  Class klass = org.apache.commons.io.FileUtils.class;

  CodeSource codeSource = klass.getProtectionDomain().getCodeSource();

  if (codeSource != null) {
   System.out.println(codeSource.getLocation());
  }
 }

}

Output:

 file:/C:/Users/akumaras/workspace/WtsPlayProject/lib/commons-io-2.4.jar
 

 Q3. Why commons-io version 2.1 was chosen over version 2.4?


Tip #5: In Maven, due to its transitive dependencies behavior, multiple versions of same jar could be pulled in. You need to determine which jar is bringing in the this duplicate or wrong version of the jar and exclude it in the pom.xml file. The verbose flag instructs the dependency tree to display conflicting dependencies that were omitted from the resolved dependency tree. For example, to see why commons-io 2.0 was chosen over commons-io 2.4

  mvn dependency:tree -Dverbose -Dincludes=commons-io
  

The 3 handy commands to solve jar hell issues in maven are mvn dependency:tree, mvn dependency:analyze, and mvn help:effective-pom. The IDEs like eclipse provide tools to analyze dependencies. In eclipse, double click on a pom.xml file, and then select the "Dependency Hierachy" tab to analyze why a particular jar was chosen.

<dependency>
 <groupId>org.jboss.resteasy</groupId>
 <artifactId>resteasy-jaxrs</artifactId>
 <version>${resteasy.version}</version>
</dependency>

commons-io-1.4 is a very old version. Now to to exclude older version and include commons-io-2.4, you need to do the following in the pom.xml file.

  
<dependency>
 <groupId>org.jboss.resteasy</groupId>
 <artifactId>resteasy-jaxrs</artifactId>
 <version>${resteasy.version}</version>
 <exclusions>
      <exclusion>
          <groupId>commons-io</groupId>
          <artifactId>commons-io</artifactId>
      </exclusion>
 </exclusions> 
</dependency>
<dependency>
        <groupId>commons-io</groupId>
     <artifactId>commons-io</artifactId>
     <version>2.4</version>
</dependency>

Tip #6:  uber-jar is an "over-jar", and uber is the German word for above or over. uber-jar is defined as one that contains both your package and all its dependencies in one single JAR file (Note: jars cannot have other jars). The advantage is that you can distribute your uber-jar and not care at all whether or not dependencies are installed at the destination, as your uber-jar actually has no dependencies. Maven has a plugin known as the "Apache Maven Shade Plugin".

 This plugin can also be used in scenarios where you are using two jars X and Y. Y is a library and X has classes that uses some old classes from library Y with same names causing a jar hell issue.  You can solve your problem by renaming the package of the class that you don't want.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <executions>
        <execution>
        <phase>package</phase>
        <goals>
            <goal>shade</goal>
            </goals>
        <configuration>
                <relocations>
                    <relocation>
                            <pattern>com.myapp.MyClass</pattern>
                               <shadedPattern>com.myapp.rename.MyClass</shadedPattern>
                    </relocation>
                </relocations>
                    <promoteTransitiveDependencies>true</promoteTransitiveDependencies>
        </configuration>
        </execution>
    </executions>
</plugin>