Scalable Straight Through Processing System (OLTP) vs OLAP in Java
Large mission critical applications use Straight Through Processing, and these systems need to be highly scalable. So, when you apply for these high paying jobs, it really pays to prepare for your job interviews with the following questions and answers.
Q. What is Straight Through Processing (STP)?
A. This is the definition from INVESTOPEDIA.
"An initiative used by companies in the financial world to optimize the speed at which transactions are processed. This is performed by allowing information that has been electronically entered to be transferred from one party to another in the settlement process without manually re-entering the same pieces of information repeatedly over the entire sequence of events."
Q. How will you go about designing a STP system?
A. Most conceptual architectures use a hybrid approach using a combination of different architectures based on the benefits of each approach and its pertinence to your situation. Here is a sample hybrid approach depicting an online trading system, which is a STP system.
The above system is designed for:
- Placing Buy/Sell online trades real time. The trades are validated first and then sent all the way to Stock Exchange system using the FIX (Financial Information eXchange) protocol. Here some usefil links on FIX.
- Once the trade is matched, the contract notes are asynchrnously issued via the SETTLEMENT queue, and processed by an ESB (Enterprise Service Bus) system like web Methods, Tibco, or Websophere MQ. Here are some useful links on asynchronous processing.
The above system is an
operational OLTP (i.e. On-Line Transaction Processing) system.
These systems are also known as STP (i.e. Straight Through
Processing) system. This leads to another question.
Q.
What is the difference between OLTP
and
OLAP?
A.
OLTP stands for On-Line Transaction Processing and OLAP stands for
On-Line Analytical Processing. OLAP contains a multidimensional or
relational data store designed to provide quick access to
pre-summarized
data & multidimensional
analysis.
MOLAP: Multidimensional OLAP – enabling OLAP by providing cubes.ROLAP: Relational OLAP – enabling OLAP using a relational database management system
OLTP
|
OLAP
|
Creates operational
source data from transactional systems as shown in the above
diagram. This data is the source of truth for many other systems.
|
Data comes from various
OLTP data sources as shown in the above diagram.
|
Transactional and
normalized data is used for daily operational business activities.
|
Historical,
de-normalized and aggregated multidimensional data is used for
analysis and decision making. This is also known as for BI (i.e.
Business Intelligence).
|
Data is inserted via
short inserts and updates. The data is normally captured via user
actions.
|
Periodic (i.e.
scheduled) and long running (i.e. during off-peak) batch jobs
refresh the data. Also, known as ETL process as shown in the below
diagram.
|
The database design
involves highly normalized tables.
|
The database design
involves de-normalized tables for speed. Also, requires more
indexes for the aggregated data.
|
|
Data can be reloaded
from the OLTP systems if required. Hence, stringent backup is not
required.
|
Transactional data older
than certain period can be archived and purged based on the
compliance requirements.
|
Contains
historical data. The volume of this data will be
higher as well due to its requirement to maintain historical data.
|
The typical users are
operational staff.
|
The typical users are
management and executives to make business decisions.
|
The space requirement is
relatively small if the historical data is regularly archived.
|
The space requirement is
larger due to the existence of aggregation structures and
historical data. Also requires more spaces for the indexes.
|
Labels: Architecture, Scalability
2 Comments:
Dear Sir,
I've learned so much from your books and blog. Hope to be able to meet you and work with you sometime
Thanks for your compliments, and glad that my resources helped you.
Post a Comment
Subscribe to Post Comments [Atom]
<< Home