Scalable Straight Through Processing System (OLTP) vs OLAP in Java

Large mission critical applications use Straight Through Processing, and these systems need to be highly scalable. So, when you apply for these high paying jobs, it really pays to prepare for your job interviews with the following questions and answers.

Q. What is Straight Through Processing (STP)?
A. This is the definition from INVESTOPEDIA.

"An initiative used by companies in the financial world to optimize the speed at which transactions are processed. This is performed by allowing information that has been electronically entered to be transferred from one party to another in the settlement process without manually re-entering the same pieces of information repeatedly over the entire sequence of events."

Q. How will you go about designing a STP system?
A.  Most conceptual architectures use a hybrid approach using a combination of different architectures based on the benefits of each approach and its pertinence to your situation. Here is a sample hybrid approach depicting an online trading system, which is a STP system.

The above system is designed for:

• Placing Buy/Sell online trades real time. The trades are validated first and then sent all the way to Stock Exchange system using the FIX (Financial Information eXchange) protocol. Here some usefil links on FIX.

Are you going for programming jobs at investment banks or financial institutions? Knowing some investment and trading terms will be useful?

• Once the trade is matched, the contract notes are asynchrnously issued via the SETTLEMENT queue, and processed by an ESB (Enterprise Service Bus) system like web Methods, Tibco, or Websophere MQ. Here are some useful links on asynchronous processing.

1. Java interview questions and answers on asynchronous processing
2. Asynchronous processiong with Spring tutorial
3. Asynchronous processing with Apache Camel (ESB)

The above system is an operational OLTP (i.e. On-Line Transaction Processing) system. These systems are also known as STP (i.e. Straight Through Processing) system. This leads to another question.

Q. What is the difference between OLTP and OLAP?

A. OLTP stands for On-Line Transaction Processing and OLAP stands for On-Line Analytical Processing. OLAP contains a multidimensional or relational data store designed to provide quick access to pre-summarized data & multidimensional analysis. 

 

MOLAP: Multidimensional OLAP – enabling OLAP by providing cubes.
ROLAP: Relational OLAP – enabling OLAP using a relational database management system

OLTP	OLAP
Creates operational source data from transactional systems as shown in the above diagram. This data is the source of truth for many other systems.	Data comes from various OLTP data sources as shown in the above  diagram.
Transactional and normalized data is used for daily operational business activities.	Historical, de-normalized and aggregated multidimensional data is used for analysis and decision making. This is also known as for BI (i.e. Business Intelligence).
Data is inserted via short inserts and updates. The data is normally captured via user actions.	Periodic (i.e. scheduled) and long running (i.e. during off-peak) batch jobs refresh the data. Also, known as ETL process as shown in the below diagram.
The database design involves highly normalized tables.	The database design involves de-normalized tables for speed. Also, requires more indexes for the aggregated data.
Regular backup of data is required to prevent any loss of data, monetary loss, and legal liability.	Data can be reloaded from the OLTP systems if required. Hence, stringent backup is not required.
Transactional data older than certain period can be archived and purged based on the compliance requirements.	Contains historical data. The volume of this data will be higher as well due to its requirement to maintain historical data.
The typical users are operational staff.	The typical users are management and executives to make business decisions.
The space requirement is relatively small if the historical data is regularly archived.	The space requirement is larger due to the existence of aggregation structures and historical data. Also requires more spaces for the indexes.