What are the different ways a Java thread gets blocked or suspended? How will you debug Java threading issues?

1. It has been put to sleep for a set amount of time

 
   public void run(){
       try{
           while(true){
               this.sleep(1000);
               System.out.println("looping while");
            }
        }catch(InterruptedException ie){
        ie.printStackTrace();
  }
   }

2. The thread is suspended by call to wait( ), and will become runnable on a notify or notifyAll message.

 
class ConsumerProducer {

   private int count;
  
   public synchronized  void consume() {
       while(count == 0) {
          try {
        wait();
    }
     catch(InterruptedException ie){}
       }
       count--;   //consumed     
   }   


   public synchronized void produce() {
         count++;
   notify(); //notify the waiting consumer that count is incremented 
   }
   
}

3. Using join( ) method, which means waiting for a thread to complete. In the example below, the main thread that spawned worker threads t1 and t2 will wait on the t1.join() line until t1 has finished its work, and then will do the same for t2.join( ).

 
    Thread t1 = new Thread(new EventThread("event-1"));
    t1.start();
    Thread t2 = new Thread(new EventThread("event-2"));
    t2.start();

    while (true) {
        try {
           t1.join();
           t2.join();    
        }
        catch (InterruptedException e) {
            e.printStackTrace();
        }
     }

4. Threads will be blocked on acquiring an intrinsic or explicit lock. Understanding Java locks and synchronized keyword.

5. Threads can get blocked on a long running I/O operations. For example, on a call  to long running database operation.

Threads block on I/O so that other threads may execute whilst the I/O operation is being performed.

class MyServer implements Runnable {
  public void run() {
    try {
       ServerSocket ss = new ServerSocket(PORT);
       while (!Thread.interrupted()){
           new Thread(new Handler(ss.accept())).start();
           // one thread per socket connection
           // every thread created this way will essentially block for I/O
   }
   } catch (IOException ex) { 
      ex.printStacktrace();
   }
 }
 
   //...
}

In Java 5 NIO (New I/O) was introduced to perform non-blocking I/O with selectors and channels.

Q. How will you go about debugging threads that are blocked?
A. Creating thread dumps.Thread dumps are very useful for diagnosing synchronization problems such as blocked threads causing deadlocks, thread starvation, etc.

• The trick is to take 5 or 6 sets of thread dumps at an interval of 5 seconds between each to have a dump file that has 25 to 30 seconds worth of run time action. 

• For thread dumps, use kill -3 in Unix and CTRL+BREAK in Windows. There are tools like Thread Dump Analyzer (TDA), Samurai, etc to derive useful information from the thread dumps to find where the problem is.

• For example, Samurai colors idle threads in grey, blocked threads in red, and running threads in green. You must pay more attention to those red ones, as to why they are blocked for 5+ seconds.

Manually reviewing the code for any obvious thread-safety issues. There are static analysis tools like Sonar, ThreadCheck, etc for catching concurrency bugs at compile-time by analyzing their byte code.

List all possible causes and add extensive log statements and write test cases to prove or disprove your theories.

There are tools like JDB (i.e. Java DeBugger) where a “watch” can be set up on the suspected variable. When ever the application is modifying that variable, a thread dump will be printed.

There are dynamic analysis tools like jstack and JConsole, which is a JMX compliant GUI tool to get a thread dump on the fly. The JConsole GUI tool does have handy features like “detect deadlock” button to perform deadlock detection operations and ability to inspect the threads and objects in error states. Similar tools are available for other languages as well.

What is a volatile variable in Java and when should you use it?

Java Multi-threading Interview Questions and Answers

Beginner Q&A	More beginner Q&A	Beginner locks	Beginner thread communication	Beginner thread sequencing	Intermediate Q&A	Volatile variable
Advanced Q&A	Printing odd and even numbers	Thread pools	ExecutorService	Atomic operations	CountDownLatch and CyclicBarrier	Semaphores and mutexes

Three of the Java keywords that are very popular in job interviews are 1) volatile 2) transient 3) const

1) volatile is popular because it can be a bit tricky to understand and know when to use it.
2) transient is popular as it is not widely used, but handy in object serialization.
3) const is popular because it is a reserved keyword, but is not currently used. The final keyword  on a reference variable just means that the reference cannot be changed to reference a different object. But, the object itself can be changed if their fields are not final. The const is reserved to make the whole object not changeable, but currently not used.

Let's focus now on the "volatile" keyword.

Q. What is a volatile key word in Java?
A. The volatile keyword is used with object and primitive variable references to indicate that a variable's value will be modified by different threads. This means

• The value of this variable will never be cached locally within the thread, and all the reads and writes must go to the main memory to be visible to the other threads. In other words the keyword volatile guarantees visibility.
• From JDK 5 onwards, writing to a volatile variable happens before reading from a volatile variable. In other words, the volatile keyword guarantees ordering, and prevents compiler or JVM from reordering of the code.
  

Q. How does  volatile keyword differ from the synchronized keyword?
A. 

1. The volatile keyword is applied to variables of both primitives and objects, whereas the synchronized keyword is applied to only objects.
2. The volatile keyword only guarantees visibility and ordering, but not atomicity, whereas the synchronized keyword can guarantee both visibility and atomicity if done properly. So, the volatile variable has a limited use, and cannot be used in compound operations like incrementing a variable, etc.

Wrong use of volatile in a compound operation

volatile int counter = 0;

public void increment(){
   counter++;
}

Right use of volatile. Example1:

volatile boolean status = false;

//...

public void process(){
   while(!status){
   //....
   }
}

Or in lazy singleton. Example2: Double checked locking

public final Class MySingleton {

     private static volatile MySingleton instance = null;

     private MySingleton( ){}

     public static MySingleton getInstance() {
            if(instance == null) {
                 synchronized (MySingleton.class) {
                         if(instance == null) {
                                 instance = new MySingleton();
                         }
                 }
            }

            return instance;
     }

}

Important: Synchronized keyword (i.e. locking) can guarantee both visibility and atomicity, whereas volatile variables can only guarantee visibility. A synchronized block can be used in place of volatile but the inverse is not true.

So, if you are not sure where to use, then favor the "synchronized" keyword.


Q. Why is locking of a method or block of code for thread safety is called "synchronized" and not "lock" or "locked"?
A. When a method or block of code is locked with the reserved "synchronized" key word in Java, the memory (i.e. heap) where the shared data is kept is synchronized. This means,

• When a synchronized block or method is entered after the lock has been acquired by a thread, it first reads any changes to the locked object from the main heap memory to ensure that the thread that has the lock has the current info before start executing.
• After the synchronized  block has completed and the thread is ready to relinquish the lock, all the changes that were made to the object that was locked is written or flushed back to the main heap memory so that the other threads that acquire the lock next has the current info.

This is why it is called "synchronized" and not "locked". This is also the reason why the immutable objects are inherently thread-safe and does not require any synchronization. Once created, the immutable objects cannot be modified.

How to capture user input in Java or interact with user via command line

#1: Using the System.console( ) . This will not work within eclipse IDE. You can try it on a DOS or Unix command-line.

package com.user.input;

import java.io.Console;

public class Example1 {
 
  public static void main(String[] args) {
   Console console = System.console();
   String input = console.readLine("What is your name:");
   System.out.println("User input = " + input);
 }
}

The output:

C:\temp>java -cp C:\Users\akumaras\workspace\test\bin com.user.input.Example1

What is your name:Arul
User input = Arul

C:\Users\akumaras\workspace\test\bin\com\user\input>

#2: The Scanner class.

package com.user.input;

import java.io.Console;
import java.util.Scanner;

public class Example2 {
 
  public static void main(String[] args) {
   Scanner reader = new Scanner(System.in);
   System.out.println("Enter first operand?");
   int operand1=reader.nextInt();
   
   System.out.println("Enter second operand?");
   int operand2=reader.nextInt();
   
   System.out.println("The sum is: " + (operand1 + operand2));
 }
}

The output:

Enter first operand?
5
Enter second operand?
6
The sum is: 11

#3: Decorating System.in with BufferedReader and SystemInputReader.

package com.user.input;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

public class Example3 {
 
  public static void main(String[] args) throws NumberFormatException, IOException {
   //System.in is decorated with InputStreamReader and BufferedReader
   //buffering is required for efficiency
   BufferedReader br = new BufferedReader( new InputStreamReader(System.in));
   
   System.out.println("Enter first operand?");
   int operand1 = Integer.parseInt(br.readLine());
  
     
   System.out.println("Enter second operand?");
   int operand2=Integer.parseInt(br.readLine());
   
   System.out.println("The sum is: " + (operand1 + operand2));
 }
}

The output:

package com.user.input;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

public class Example3 {
 
  public static void main(String[] args) throws NumberFormatException, IOException {
   //System.in is decorated with InputStreamReader and BufferedReader
   //buffering is required for efficiency
   BufferedReader br = new BufferedReader( new InputStreamReader(System.in));
   
   System.out.println("Enter first operand?");
   int operand1 = Integer.parseInt(br.readLine());
  
     
   System.out.println("Enter second operand?");
   int operand2=Integer.parseInt(br.readLine());
   
   System.out.println("The sum is: " + (operand1 + operand2));
 }
}

Note: You can also decorate it with DataInputStream. A good example of the decorator design pattern implemented in the Java I/O API.

Top 10 causes of performance issues in Java

Cause #1: The JVM spends more time performing garbage collection due to

• improper Garbage Collection (GC) configuration. E.g. Young generation being too small.
• Heap size is too small (use -Xmx). The application footprint is larger than the allocated heap size.
• Wrong use of libraries. For example, XML based report generation using DOM parser as opposed to StAX for large reports generated concurrently by multiple users. 
• Incorrectly creating and discarding objects without astutely reusing them with a flyweight design pattern or proper caching strategy.
• Other OS activities like swap space or networking activity during GC can make GC pauses last longer. 
• Any explicit System.gc( ) from your application or third party modules.

Run your JVM with GC options such as 

• -verbose:gc (print the GC logs) 
• -Xloggc: (comprehensive GC logging) 
• -XX:+PrintGCDetails (for more detailed output) 
• -XX:+PrintTenuringDistribution (tenuring thresholds)

to understand the GC patterns.

Cause #2: Bad use of application algorithms, strategies, and queries. For example

• SQL queries with Cartesian joins.
• SQL queries invoking materialized views
• Regular expressions with back tracking algorithms.
• Inefficient Java coding and algorithms in frequently executed methods leading to death by thousand cuts.
• Excessive data caching or inappropriate cache refresh strategy.
• Overuse of pessimistic locking as opposed to favoring optimistic locking.

Cause #3:  Memory leaks due to

• Long living objects having reference to short living objects, causing the memory to slowly grow. For example, singleton classes referring to short lived objects. This prevents short-lived objects being garbage collected.
• Improper use of thread-local variables. The thread-local variables will not be removed by the garbage collector as long as the thread itself is alive. So, when threads are pooled and kept alive forever, the object might never be removed by the garbage collector.
• Using mutable static fields to hold data caches, and not explicitly clearing them. The mutable static fields and collections need to be explicitly cleared.
• Objects with circular or bidirectional references.
• JNI memory leaks.

Cause #4: Poor integration with external systems without proper design & testing.

• Not properly deciding between synchronous vs asynchronous calls to internal and external systems. Long running tasks need to be performed asynchronously. 
• Not properly setting service timeouts and retries or setting service time out values to be too high.
• Not performing non happy path testing and not tuning external systems to perform efficiently.
• Unnecessarily making too many network round trips.

Cause #5: Improper use of Java frameworks and libraries.

• Using Hibernate without properly understanding lazy loading versus eager fetching and other tuning capabilities.
• Not inspecting the SQLs internally generated by your ORM tools like Hibernate.
• Using the deprecated libraries like Vector or Hashtable as opposed to the new concurrent libraries that allow concurrent reads.
• Using blocking I/O where the the Java NIO (New I/O) with non blocking capability is favored. 
• Database deadlocks due bad schema design or  application logic.
• Spinning out your own in efficient libraries as opposed to favoring proven frameworks.

Cause #6: Multi-threading issues due to to deadlocks, thread starvation, and thread contention. 

• Using coarse grained locks over fine grained locks. 
• Not favoring concurrent utility classes like  ConcurrentHashMap, CopyOnWriteArrayList, etc. 
• Not favoring lock free algorithms.

Cause #7: Not managing and recycling your non memory resources properly.

• Not pooling your valuable non memory resources like sockets, connections, file handles, threads, etc. 
• Not properly releasing your resources back to its pool after use can lead to resource leaks and performance issues. 
• Use of too many threads leading to more CPU time being used for context switching. 
• Hand rolling your own pooling without favoring proven libraries. 
• Using a third-party library with resource leaks.
• Load balancers leaking sockets.

Cause #8: Bad infrastructure designs and bugs.

• Databases tables not properly partitioned. 
• Not enough physical memory on the box.
• Not enough hard disk space.
• Bad network latency. 
• Too many nodes on the server.
• Load balancers  not working as intended and not performing outage testing.
• Not tuning application servers properly.
• Not performing proper capacity planning.
• router, switch, and DNS server failures. 

Cause #9: Excessive logging and not using proper logging libraries with capabilities to control log levels like debug, info, warning, etc. System.out.println(....) are NOT ALLOWED. Favor asynchronous logging in mission critical and high volume applications like trading systems. 

Cause #10: Not conducting performance tests, not monitoring the systems, and lack of documentation and performance focus from the start of the project.

• Not performing performance test plans with tools like JMeter with production like data prior to each deployment.
• Not monitoring the systems for CPU usage, memory usage, thread usage, garbage collection patterns, I/O, etc on an on going basis. 
• Not defining SLA's (Service Level Agreements) from the start in the non-functional specification.
• Not maintaining proper performance benchmarks to be compared against with the successive releases and performance tests.
• Not looking for performance issues in peer code reviews or not having code reviews at all.

Java and JSON tutorial converting JSON to object and vice versa with Jackson library

Q. What is JSON and when to favor it over XML?
A. JSON (JavaScript Object Notation) is a lightweight, text-based, language-netral like XML, but less verbose than XML data exchange format. JSOJ is used in Web services to exchange data between client and server. Client makes ajax requests to get snippets of data in JSON format from the server via RESTful web service calls. JSON is easy for humans and machines to read and write. JSON can be represented as objects and arrays.

XML

<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

JSON employee object:

{ 
  "name":"Peter", 
  "nameType":"first", 
  "age":"25"
}

JSON with Name as a separate object

{ 
  "name": {
     "name":"Peter",
  "type":"first"
  }
  "age":"25"
}

JSON Array:

{"employees":[
    {"name":"Peter", nameType="first", "age":"25"}, 
    {"name":"John", nameType="first", "age":"52"},
    {"name":"Simon", nameType="first", "age":"34"}
]}

Jackson library is one of the popular libraries for converting JSON to Java object and vice versa. Here is a simple step by step example.

Step 1: You need to have jackson-core-asl-xxx.jar and jackson-mapper-asl-xxx.jar files in the classpath.

 <dependencies>
 <dependency>
  <groupId>org.codehaus.jackson</groupId>
  <artifactId>jackson-mapper-asl</artifactId>
  <version>1.9.5</version>
 </dependency>
 <dependency>
  <groupId>org.codehaus.jackson</groupId>
  <artifactId>jackson-core-asl</artifactId>
  <version>1.9.5</version>
 </dependency>
  </dependencies>

Step 2: The Employee Java object.

package com.json;

public class Employee {

 private String name;
 private String type;
 private int age;

 public String getName() {
  return name;
 }

 public void setName(String name) {
  this.name = name;
 }

 public String getType() {
  return type;
 }

 public void setType(String type) {
  this.type = type;
 }

 public int getAge() {
  return age;
 }

 public void setAge(int age) {
  this.age = age;
 }

 @Override
 public String toString() {
  StringBuilder sb = new StringBuilder();
  sb.append("name=" + this.name);
  if (type != null) {
   sb.append("\ntype=" + type);
  }
  sb.append("\n");
  sb.append("age=" + age);
  return sb.toString();
 }
}

Step 3: Jackson in action.

package com.json;

import java.io.IOException;

import org.codehaus.jackson.JsonParseException;
import org.codehaus.jackson.map.JsonMappingException;
import org.codehaus.jackson.map.ObjectMapper;

public class JsonExample {
 
 public static void main(String[] args) {
  String json = "{ \"name\":\"Peter\", \"type\":\"first\", \"age\":\"25\"}";
  
  ObjectMapper mapper = new ObjectMapper();
  
  try {
   //convert JSON to Java object
   Employee readValue = mapper.readValue(json.getBytes(), Employee.class);
   System.out.println("object:" + readValue);
   
   //convert Java object to JSON
   mapper.writeValue(System.out, readValue);
   
  } catch (IOException  e) {
   e.printStackTrace();
  } 
 }
}

Output:

object:name=Peter
type=first
age=25
{"name":"Peter","type":"first","age":25}

Converting Java Map to JSON and vice versa using Jackson

package com.json;

import java.io.IOException;
import java.util.Map;

import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.type.TypeReference;

public class JsonExample {

 public static void main(String[] args) {
  String json = "{ \"name\":\"Peter\", \"type\":\"first\", \"age\":\"25\"}";

  ObjectMapper mapper = new ObjectMapper();

  try {
   // convert JSON to Map
   Map mapObject = mapper.readValue(json.getBytes(), 
     new TypeReference

Output:

object:{name=Peter, type=first, age=25}
{"name":"Peter","type":"first","age":"25"}

Java and XSLT tutorial to transform XML documents

Q. What is XSLT?
A. XSLT (Extensible Stylesheet Language Transformations) is a language for transforming XML documents into other XML, HTML, or plain text documents. XSLT is based on functional prgramming ideas, but it is not a full functional programming language as it lacks ability to treat functions as a first class data type.  XSLT supports XPath to query parts of XML. The major disadvantage of XSL is that it can be very verbose.

Step 1: The source XML

<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

Step 2: The style sheet (i.e XSL) that defines how to transform

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
 <xsl:template match="/Employee">
    <HTML>
      <HEAD>
        <TITLE></TITLE>
      </HEAD>
      <BODY>
        <H1>
    Hello  
          <xsl:value-of select="name"/>
        </H1>
      </BODY>
    </HTML>

 </xsl:template>
</xsl:stylesheet>

Step 3: The Java code to perform the transformation by applying the above XSL.

package com.xml;

import java.io.ByteArrayInputStream;

import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

public class XsltTransform {

 public static void main(String[] args) {

  String xml = "<Employee><name type=\"first\">Peter</name><age>25</age></Employee>";
  String xsl = "<xsl:stylesheet xmlns:xsl=\"http://www.w3.org/1999/XSL/Transform\" version=\"1.0\">\r\n" + 
    " <xsl:template match=\"/Employee\">\r\n" + 
    "    <HTML>\r\n" + 
    "      <HEAD>\r\n" + 
    "        <TITLE></TITLE>\r\n" + 
    "      </HEAD>\r\n" + 
    "      <BODY>\r\n" + 
    "        <H1>\r\n" + 
    "    Hello  \r\n" + 
    "          <xsl:value-of select=\"name\"/>\r\n" + 
    "        </H1>\r\n" + 
    "      </BODY>\r\n" + 
    "    </HTML>\r\n" + 
    "\r\n" + 
    " </xsl:template>\r\n" + 
    "</xsl:stylesheet>";

  try {
   Source xmlSource = new StreamSource(new ByteArrayInputStream(xml.getBytes()));
   Source xslSource = new StreamSource(new ByteArrayInputStream(xsl.getBytes()));
   Result result = new StreamResult(System.out);
   
   TransformerFactory factory = TransformerFactory.newInstance();
   Transformer transformer = factory.newTransformer(xslSource);
   transformer.transform(xmlSource, result);

  } catch (TransformerException  e) {                         
   e.printStackTrace();
  }

 }
}

Output:

<HTML>
<HEAD>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<TITLE></TITLE>
</HEAD>
<BODY>
<H1>
    Hello  
          Peter</H1>
</BODY>
</HTML>

Finding your way around the FIX specification

FIX and Java

FIX tutorial in Java with QuickFIX/j simple example

QuickFix Java example sending and receiving FIX messages

Finding your way around the FIX specification

If you are using FIX spec 4.4 then go to https://fixspec.com/FIX44. The tutorials with QuickFIX/J used NewSingleOrder as the request.

The message has the fields like Tag number.

FIX message for NewOrderSingle request

8=FIX.4.4 9=123 35=D 34=53 49=CLIENT1 52=20140731-01:59:47.671 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140731-01:59:47.669 10=238  

Understanding the message:

Tag:

  8  is StandardHeader "BeginString" denoting the FIX version 4.4.
  9  is BodyLength 123
 35 is MsgType, which is "D"
 34 is MsgSeqNum "53"
 49 is SenderCompID, which is "CLIENT1" from the sender.cfg configuration file
 52 is SendingTime
 56 is TargetCompID, which is "FIXServer" from the sender.cfg configuration file
 11 is ClOrdID, is a unique order id, which is "DLF"
 40 is OrdType, "2" means Limit order
 44 is Price, which is "25.4"
 54 is Side, 1 means Buy.
 55 is Symbol from the Instrument.
 60 is TransactionTime
 10 is CheckSum form the StandardTrailer.


There are different types of data and you can go through them at your leisure.

QuickFix Java example sending and recieving FIX messages

FIX and Java

FIX tutorial in Java with QuickFIX/j simple example

QuickFix Java example sending and receiving FIX messages

Finding your way around the FIX specification

If you like to work on trading applications, you need to know the  FIX protocol and a FIX
 engine like QuickFIX/J. 

This extends our simple QuickFIX Java example tutorial. This assumes that you understand the first QuickFIX Java tutorial.


Step 1: The sender or the initiator FIXSender.java has a System.out.println(..) in the fromApp(..,..) method to print the execution report returned by receiver or the acceptor FIXReceiver.java

package com.example;

import quickfix.Application;
import quickfix.DoNotSend;
import quickfix.FieldNotFound;
import quickfix.IncorrectDataFormat;
import quickfix.IncorrectTagValue;
import quickfix.Message;
import quickfix.RejectLogon;
import quickfix.SessionID;
import quickfix.UnsupportedMessageType;

public class FIXSender implements Application {

 @Override
 public void fromAdmin(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, 
 IncorrectTagValue, RejectLogon {
 }

 @Override
 public void fromApp(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, 
         IncorrectTagValue, UnsupportedMessageType { 
  
  System.out.println("Sender fromApp: " + arg0);
 }

 @Override
 public void onCreate(SessionID arg0) {}

 @Override
 public void onLogon(SessionID arg0) {}

 @Override
 public void onLogout(SessionID arg0) {}

 @Override
 public void toAdmin(Message arg0, SessionID arg1) {}

 @Override
 public void toApp(Message msg, SessionID sessionId) throws DoNotSend {
  System.out.println("Sender toApp: " + msg.toString());
 }

}

Step 2: The onMessage(..,..) method is extended to return an execution report in FIXReceiver.java

package com.example;

import quickfix.Application;
import quickfix.DoNotSend;
import quickfix.FieldNotFound;
import quickfix.IncorrectDataFormat;
import quickfix.IncorrectTagValue;
import quickfix.Message;
import quickfix.RejectLogon;
import quickfix.Session;
import quickfix.SessionID;
import quickfix.SessionNotFound;
import quickfix.UnsupportedMessageType;
import quickfix.field.AvgPx;
import quickfix.field.ClOrdID;
import quickfix.field.CumQty;
import quickfix.field.ExecID;
import quickfix.field.ExecType;
import quickfix.field.LastPx;
import quickfix.field.LeavesQty;
import quickfix.field.OrdStatus;
import quickfix.field.OrdType;
import quickfix.field.OrderID;
import quickfix.field.OrderQty;
import quickfix.field.Price;
import quickfix.field.Side;
import quickfix.field.Symbol;
import quickfix.fix44.ExecutionReport;
import quickfix.fix44.MessageCracker;
import quickfix.fix44.NewOrderSingle;

public class FIXReceiver extends MessageCracker implements Application {

 @Override
 public void onMessage(NewOrderSingle order, SessionID sessionID) throws FieldNotFound, IncorrectTagValue {
  System.out.println("Receiver onMessage..  " + order);
  Symbol symbol = new Symbol();
  Side side = new Side();
  OrdType ordType = new OrdType();
  OrderQty orderQty = new OrderQty();
  Price price = new Price();
  ClOrdID clOrdID = new ClOrdID();
  order.get(ordType);

  if (ordType.getValue() != OrdType.LIMIT) {
   throw new IncorrectTagValue(ordType.getField());
  }
  
  
  order.get(symbol); 
  order.get(side); 
  order.get(orderQty); 
  order.get(price); 
  order.get(clOrdID); 
  
  ExecutionReport executionReport = new ExecutionReport(
    new OrderID("1"), 
    new ExecID("1"), 
    new ExecType(ExecType.NEW), 
    new OrdStatus(OrdStatus.FILLED), 
    side, 
    new LeavesQty(0), 
    new CumQty(orderQty.getValue()), 
    new AvgPx(price.getValue())
  );
 
  executionReport.set(clOrdID); 
  executionReport.set(orderQty); 
  executionReport.set(side); 
  executionReport.set(new LastPx(price.getValue())); 
  
  if (order.isSetAccount()) {
   executionReport.set(order.getAccount()); 
  } 
  
  try {
   Session.sendToTarget(executionReport, sessionID);
  } catch (SessionNotFound e) {
   e.printStackTrace();
  } 
 
 }

 @Override
 public void fromAdmin(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, IncorrectTagValue,
   RejectLogon {
 }

 @Override
 public void fromApp(Message arg0, SessionID arg1) throws FieldNotFound, IncorrectDataFormat, IncorrectTagValue,
   UnsupportedMessageType {
  System.out.println("Receiver fromApp..  " + arg0);
  crack(arg0, arg1); // calls onMessage(..,..)
 }

 @Override
 public void onCreate(SessionID arg0) {
  System.out.println("Receiver onCreate.. " + arg0);
 }

 @Override
 public void onLogon(SessionID arg0) {
  System.out.println("Receiver onLogon.." + arg0);
 }

 @Override
 public void onLogout(SessionID arg0) {
 }

 @Override
 public void toAdmin(Message arg0, SessionID arg1) {
 }

 @Override
 public void toApp(Message arg0, SessionID arg1) throws DoNotSend {
 }

}

Output: Receiver

<20140801-06:43:48, FIX.4.4:FixServer->CLIENT1, event> (Session FIX.4.4:FixServer->CLIENT1 schedule is daily, 00:00:00-UTC - 00:00:00-UTC)
<20140801-06:43:48, FIX.4.4:FixServer->CLIENT1, event> (Created session: FIX.4.4:FixServer->CLIENT1)
Receiver onCreate.. FIX.4.4:FixServer->CLIENT1
press <enter> to quit
<20140801-06:43:54, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=71 35=A 34=7 49=CLIENT1 52=20140801-06:43:54.569 56=FixServer 98=0 108=30 10=156 )
<20140801-06:43:54, FIX.4.4:FixServer->CLIENT1, event> (Accepting session FIX.4.4:FixServer->CLIENT1 from /127.0.0.1:62832)
<20140801-06:43:54, FIX.4.4:FixServer->CLIENT1, event> (Acceptor heartbeat set to 30 seconds)
<20140801-06:43:54, FIX.4.4:FixServer->CLIENT1, event> (Received logon)
<20140801-06:43:54, FIX.4.4:FixServer->CLIENT1, event> (Responding to Logon request)
<20140801-06:43:54, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=71 35=A 34=7 49=FixServer 52=20140801-06:43:54.605 56=CLIENT1 98=0 108=30 10=147 )
Receiver onLogon..FIX.4.4:FixServer->CLIENT1
<20140801-06:43:56, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=122 35=D 34=8 49=CLIENT1 52=20140801-06:43:56.613 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140801-06:43:56.611 10=164 )
Receiver fromApp..  8=FIX.4.4 9=122 35=D 34=8 49=CLIENT1 52=20140801-06:43:56.613 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140801-06:43:56.611 10=164 
Receiver onMessage..  8=FIX.4.4 9=122 35=D 34=8 49=CLIENT1 52=20140801-06:43:56.613 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140801-06:43:56.611 10=164 
<20140801-06:43:56, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=125 35=8 34=8 49=FixServer 52=20140801-06:43:56.700 56=CLIENT1 6=25.4 11=DLF 14=45 17=1 31=25.4 37=1 38=45 39=2 54=1 150=0 151=0 10=073 )
<20140801-06:44:27, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=59 35=0 34=9 49=FixServer 52=20140801-06:44:27.436 56=CLIENT1 10=116 )
<20140801-06:44:27, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=59 35=0 34=9 49=CLIENT1 52=20140801-06:44:27.442 56=FixServer 10=113 )
<20140801-06:44:57, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=60 35=0 34=10 49=FixServer 52=20140801-06:44:57.436 56=CLIENT1 10=151 )
<20140801-06:44:57, FIX.4.4:FixServer->CLIENT1, incoming> (8=FIX.4.4 9=60 35=5 34=10 49=CLIENT1 52=20140801-06:44:57.440 56=FixServer 10=151 )
<20140801-06:44:57, FIX.4.4:FixServer->CLIENT1, event> (Received logout request)
<20140801-06:44:57, FIX.4.4:FixServer->CLIENT1, outgoing> (8=FIX.4.4 9=60 35=5 34=11 49=FixServer 52=20140801-06:44:57.443 56=CLIENT1 10=155 )
<20140801-06:44:57, FIX.4.4:FixServer->CLIENT1, event> (Sent logout response)

Output: Sender

<20140801-06:43:53, FIX.4.4:CLIENT1->FixServer, event> (Session FIX.4.4:CLIENT1->FixServer schedule is daily, 00:00:00-UTC - 00:00:00-UTC)
<20140801-06:43:53, FIX.4.4:CLIENT1->FixServer, event> (Created session: FIX.4.4:CLIENT1->FixServer)
<20140801-06:43:54, FIX.4.4:CLIENT1->FixServer, outgoing> (8=FIX.4.4 9=71 35=A 34=7 49=CLIENT1 52=20140801-06:43:54.569 56=FixServer 98=0 108=30 10=156 )
<20140801-06:43:54, FIX.4.4:CLIENT1->FixServer, event> (Initiated logon request)
<20140801-06:43:54, FIX.4.4:CLIENT1->FixServer, incoming> (8=FIX.4.4 9=71 35=A 34=7 49=FixServer 52=20140801-06:43:54.605 56=CLIENT1 98=0 108=30 10=147 )
<20140801-06:43:54, FIX.4.4:CLIENT1->FixServer, event> (Received logon)
Sender toApp: 8=FIX.4.4 9=122 35=D 34=8 49=CLIENT1 52=20140801-06:43:56.613 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140801-06:43:56.611 10=164 
<20140801-06:43:56, FIX.4.4:CLIENT1->FixServer, outgoing> (8=FIX.4.4 9=122 35=D 34=8 49=CLIENT1 52=20140801-06:43:56.613 56=FixServer 11=DLF 38=45 40=2 44=25.4 54=1 55=BHP 60=20140801-06:43:56.611 10=164 )
<20140801-06:43:56, FIX.4.4:CLIENT1->FixServer, incoming> (8=FIX.4.4 9=125 35=8 34=8 49=FixServer 52=20140801-06:43:56.700 56=CLIENT1 6=25.4 11=DLF 14=45 17=1 31=25.4 37=1 38=45 39=2 54=1 150=0 151=0 10=073 )
Sender fromApp: 8=FIX.4.4 9=125 35=8 34=8 49=FixServer 52=20140801-06:43:56.700 56=CLIENT1 6=25.4 11=DLF 14=45 17=1 31=25.4 37=1 38=45 39=2 54=1 150=0 151=0 10=073 

Top 8 things you must know about Java 8 to start coding on the job and talking at the job interviews

Are you ready to get cracking with Java 8 new features? Here are a comprehensive Java 8 features, tutorials, and examples. Drill down into each.

#1: Transforming your thinking from OOP to FOP: One needs to get used to the transformation from imperative programming to functional  programming. You like it or not, you will be using functional programming in Java 8, and interviewers are going to quiz you on functional programming. Fortunately, Java is not a fully functional programming language, and hence one does not require the full leap to functional programming. Java 8 supports both imperative and functional programming approaches.


#2: Here are top 6 Java 8 features you can start using now by downloading Java 8. 

1. Interface can have static and default methods.
2. Lambda expression for supporting closures.
3. Java 8 adopts Joda. 
4. Java 8 corrects its lack of good support for JavaScript with the engine Nashorn. 
5. Parallel processing piggy backing the Java 7 Fork/Join.
6. The java.util.concurrent.atomic package has been expanded to include four new classes -- LongAccumulator, LongAdder, DoubleAccumulator and DoubleAdder

#3: Writing functional programs with Java 8 with simple examples. 

The Operate interface with the annotation @FunctionalInterface. This annotation ensures that you can only have a single abstract method (aka SAM). You can have additional default and static method implementations. The default methods provide default behaviors, and the closures are used to have  special behaviors. The static methods are used as helper methods. 

So, no longer you need to have the pre Java 8 patterns like

• java.utilCollection is an interface and java.util.Collections is utility class with helper methods
• java.nio.file.Path is an interface  and java.nio.file.Paths is a helper class.

Thanks to Java 8 as you can have static helper methods. 

#4: 7 useful Java 8 miscellaneous additions worth knowing and talking about in the job interviews if asked. 

1. String.join( ) method, which is an opposite of String.split(...). 
2. Comparator interfaces have a number of useful methods to sort objects with options to nest multiple fields with thenComparing(...), and other handy methods like nullsFirst( ), nullsLast( ), naturalOrder( ), reverseOrder( ), etc.
3. In Java 8, java.util.Optional class has been added to deal with optional object references.
4. In cases where you want to detect result overflow errors in int and long, the methods addExact, subtractExact, multiplyExact, and toIntExact in Java 8, the java.lang.Math class throws an ArithmeticException when the results overflow. 
5. The ability to lazily read lines from a file with Files.lines(...).
6. In Java 8, you can repeat the same annotation with the @Repeatable annotation. 
7. Java 8 introduces the "Type Annotations", which are annotations that can be placed anywhere you use a type.

#5: Different ways to sort a collection of objects in pre and post Java 8. 

1. Java library Comparator pre Java 8
2. Apache commons library BeanComparator pre Java 8
3. Google Gauva library functional programming style Pre Java 8
4. Java 8 functional programming approach. Concise and powerful.

#6: Java 8 functional programming is not a silver bullet. Where does it really shine? When will you use it?

1. Where the functional code has increased readability and maintainability
2. Where performance improvement is possible with Fork/Join parallel processing.
3. Where code becomes easier to refactor in the future
4. Where the code is easier to test and debug.

So, OO programming and functional programming can co-exist.

#7: Understanding Java 8 Streams. A stream is an infinite sequence of consumable elements (i.e a data structure) for the consumption of an operation or iteration. Any Collection can be exposed as a stream. The operations you perform on a stream can either be intermediate (map, filter, sorted, limit, skip,concat, substream, distinct, etc) producing another stream or terminal (forEach, reduce, collect, sum, max, count, matchAny, findFirst, findAny, etc) producing an object that is not a stream. Basically, you are building a pipeline as in Unix.

ls -l | grep "Dec" | Sort +4n | more

#8: Using the Predicate functional interface. The java.util.function package in Java 8 has a number of functional interfaces, and Predicate interface  is used for filtering objects from a collection.

Java 8 using the Predicate functional interface

The java.util.function package in Java 8 has a number of functional interfaces. Let's look at the Predicate<T> interface that is used for filtering objects from a collection. If you are using a Java version pre Java 8 refer Creating sub lists from a list using predicates to filter.

Here is the Java 8 example:

Step 1: Create a Driver domain class using the builder design pattern. Refer How to create elegant immutable objects in Java for the Driver code.

public class Driver {
 enum SEX {
  MALE, FEMALE
 }

 private String name;
 private int age;
 private SEX sex;

    //builder, and getters,
 
}

If not sure how to apply the builder design pattern to the Driver class.

Step 2: Define a static method to have methods that return Predicates via static methods.

package com.java8.examples;

import java.util.function.Predicate;

public class DriverPredicate {

 public static Predicate<Driver> isEligibleForDriving() {
  return p -> p.getAge() > 18;
 }

 public static Predicate<Driver> isEligibleMaleDriver() {
  return p -> p.getAge() > 18 && p.getSex() == Driver.SEX.MALE;
 }
 
}

Step 3: Create sub lists using the predicates. It is much easier to create sub lists in Java 8.

package com.java8.examples;

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

public class DriverTest {

 public static void main(String[] args) {
  List<Driver> drivers = Arrays.asList(
    new Driver.Builder().name("John").age(34).sex(Driver.SEX.MALE).build(),
    new Driver.Builder().name("James").age(24).sex(Driver.SEX.MALE).build(),
    new Driver.Builder().name("Samantha").age(16).sex(Driver.SEX.FEMALE).build()
  );

  
  //using the predicates
  List<Driver> eligibleDriverList = drivers.stream().filter(DriverPredicate.isEligibleForDriving())
    .collect(Collectors.toList());

  List<Driver> eligibleMaleDriverList = drivers.stream().filter(DriverPredicate.isEligibleMaleDriver())
    .collect(Collectors.toList());
  
  System.out.println("eligibleDriverList-->" + eligibleDriverList);
  System.out.println("eligibleMaleDriverList-->" + eligibleMaleDriverList);
  
 }

}

Output:

eligibleDriverList-->[Driver [name=John, age=34, sex=MALE], Driver [name=James, age=24, sex=MALE]]
eligibleMaleDriverList-->[Driver [name=John, age=34, sex=MALE], Driver [name=James, age=24, sex=MALE]]


Optional: You could have written the above trivial code without the DriverPredicate class as shown below. But, in industrial strength applications, you will end up violation the DRY (Don't Repeat Yourself) principle by repeating the filter logic in lambda expressions in multiple places. If the rule changes, you need to modify it in multiple locations.

package com.java8.examples;

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

public class DriverTest {

 public static void main(String[] args) {
  List<Driver> drivers = Arrays.asList(
    new Driver.Builder().name("John").age(34).sex(Driver.SEX.MALE).build(),
    new Driver.Builder().name("James").age(24).sex(Driver.SEX.MALE).build(),
    new Driver.Builder().name("Samantha").age(16).sex(Driver.SEX.FEMALE).build()
  );

  
  //using the predicates
  List<Driver> eligibleDriverList = drivers.stream().filter(p -> p.getAge() > 18)
    .collect(Collectors.toList());

  List<Driver> eligibleMaleDriverList = drivers.stream().filter(p -> p.getAge() > 18 && p.getSex() == Driver.SEX.MALE)
    .collect(Collectors.toList());
  
  System.out.println("eligibleDriverList-->" + eligibleDriverList);
  System.out.println("eligibleMaleDriverList-->" + eligibleMaleDriverList);
  
 }

}

Java and XPath tutorial to extract a subset of an XML

Q. What is XPath?
A. XPath is a query language to extract a part of XML document as an SQL is used to extract a part of a database data or a REGEX (i.e. regular expression) is used to extract a part of text. The XPath expressions can return,

XPathConstants.STRING
XPathConstants.NUMBER
XPathConstants.BOOLEAN
XPathConstants.NODE
XPathConstants.NODESET 

Here is a very basic example of XPath in Java to process a very basic XML

<Employee>
   <name type="first">Peter</name>
   <age>25</age>
</Employee>

package com.xml;

import java.io.ByteArrayInputStream;
import java.io.IOException;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;

import org.w3c.dom.Document;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class XpathQuery {

 public static void main(String[] args) {

  String xml = "<Employee><name type=\"first\">Peter</name><age>25</age></Employee>";
  DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
  factory.setNamespaceAware(true);
  DocumentBuilder builder;
  Document document = null;

  try {
   builder = factory.newDocumentBuilder();
   document = builder.parse(new ByteArrayInputStream(xml.getBytes()));

   XPathFactory xpathFactory = XPathFactory.newInstance();
   XPath xpath = xpathFactory.newXPath();

   // get employee by name with XPath expression
   // get name of the employee with age > 18
   XPathExpression expr = xpath.compile("/Employee[age>18]/name/text()");
   NodeList nodes = (NodeList) expr.evaluate(document, XPathConstants.NODESET);
   for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println("age > 15 : " + nodes.item(i).getNodeValue());
   }

   // get the age of Peter
   expr = xpath.compile("/Employee[name='Peter']/age/text()");
   nodes = (NodeList) expr.evaluate(document, XPathConstants.NODESET);
   for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println("age of peter : " + nodes.item(i).getNodeValue());
   }

   // get first name where type=first
   expr = xpath.compile("/Employee/name[@type='first']/text()");
   nodes = (NodeList) expr.evaluate(document, XPathConstants.NODESET);
   for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println("attr type='first': " + nodes.item(i).getNodeValue());
   }

  } catch (ParserConfigurationException | IOException | SAXException
    | XPathExpressionException e) { // Java 6
                           
   e.printStackTrace();
  }

 }
}

Output:

age > 15 : Peter
age of peter : 25
attribute type='first': Peter

Note: Like SQL, you need to learn the XPath query language or syntax. For example @Type means attribute "type", and "/" means root node, etc. Google for XPath syntax to learn more.