Scaling your application -- Vertical Vs Horizontal scaling

Many organization face scalability and performance issues. So, it is really worth knowing your way around these topics.

Q. What is the difference between performance and scalability?
A. The performance and scalability are two different things.

For example, if you are in the business of transporting people in a horse carriage, the performance is all about utilizing more powerful horses to transporting your people quicker to their destination. Scalability is all about catering for increase in demand for such transportation as your business grows by either increasing the capacity of individual actors (e.g. carriage capacity) or adding more actors (e.g. horses and carriages).


Q. What are the different types of scalability?
A. Vertical and Horizontal scaling.

Vertical Scaling: You can increase the capacity of a horse carriage or use more powerful horses to reduce the time it takes to reach the destination. In a computer term, increase CPU, memory, etc to increase the capacity or tune the code/ database to reduce the time it takes to process. This means we have just increased the capacity of each actor -- horse and/or carriage. You can also vertically scale an application via multi-threading or using non-blocking I/O.

Horizontal Scaling: In a horizontal scaling model, instead of increasing the capacity of each individual actor in the system, we simply add more actors to the system. This means more horses and carriages. In terms of the computers, adding more nodes and servers.


Q. How will you scale your data store?
A. The scalability of database is critical because data is often a shared resource, and it becomes the main contact point for nearly every web request. The most important question you have to ask when considering the scalability of your database is, “What kind of system am I working with?” Are you working with a read-heavy or a write-heavy system?

Scaling Reads: If your website is primarily a read-centric system, vertically scale your data store with a caching strategy that uses memory cache (e.g. ehcache) or a CDN (Content Delivery Newtork). You can also add more CPU/RAM/Disk to scale vertically.

Scaling Writes: If your website is primarily a write-heavy system, you want to think about using a horizontally scalable datastore such as MongoDB (NoSQL database), Riak, Cassandra or HBase. MongoDB is a NoSQL database with great features like replication and sharding built in. This allows you to scale your database to as many servers as you would like by distributing content among them. A database shard ("sharding") is the phrase used to describe a horizontal partition in a database or search engine. The idea behind sharding is to split data among multiple machines while ensuring that the data is always accessed from the correct place. Since sharding spreads the database across multiple machines, the database programmer specifies explicit sharding rules to determine which machines any piece of data will be stored on. Sharding may also be referred to as horizontal scaling or horizontal partitioning. Oracle uses (RAC - Real Application Cluster) where small server blades are genned-in to an Oracle RAC cluster over a high-speed interconnect.

Q. What is BigData?
A. Big data is the term for a collection of data sets so large and complex that becomes very difficult to work with using most relational database management systems and desktop statistics and visualization packages, requiring instead "massively parallel software running on tens, hundreds, or even thousands of servers". Apache™ Hadoop® is an open source software project that enables the distributed processing of large data sets across clusters of commodity servers.  It is designed to scale up from a single server to thousands of machines, with a very high degree of fault tolerance.

Hadoop uses MapReduce to understand and assign work to nodes in the cluster and HDFS(Hadoop Distributed File System), which is file system that spans all the nodes in a Hadoop cluster for data storage.


Q. What are the general scaling practices for a medium size system in Java?
A.

•  Using non-blocking IO and favoring multi-threading.
•  Vertical scaling -- more CPU, RAM, etc.
•  Caching data.
•  Favor stateless idempotent methods.
•  Using big JVM heaps
•  Using JMS -- publish/subscribe model
• Using resource pooling - e.g. database connection pooling, JMS connection factory pooling, thread pooling, etc.

Q. What are the general scaling practices for a large size system in Java?
A.

Use RTSJ (Real Time Specification for Java):  Java has the following  real time difficulties:

• During garbage collection all threads are blocked and the garbage collection time can expand to minutes. These huge latencies effectively limit memory which limits scalability.
• Increased garbage collection latencies make Java less useful for application that use heart beats, make real-time trades, etc.
• Java supports a strict priority based threading model.

To overcome this,  the Java Community introduced a specification for real-time Java, JSR001 (RTSJ -- Real Time Specification for Java)

RTSJ addressed these critical issues by mandating a minimum specification for the threading model (and allowing other models to be plugged into the VM) and by providing for areas of memory that are not subject to garbage collection, along with threads that are not preemptable by the garbage collector. These areas are instead managed using region-based memory management.


Use Big Data like MongDB.
Use distributed cache.
Use Server clusters/JVM clustering (e.g. terracotta).
SEDA based architecture.
Cloud computing.