Java 5 Executor Framework - why use thread pools?

Q. What is a thread pool, and how will you create them in Java? Why do you need an Executor framework?
A. A thread pool consists of a collection of worker threads. A work queue is optional though most of the advanced implementations have a configurable work queue. The threads in the pool constantly run and  check the work queue for new work. If there is new work to be done they execute this Runnable.

In Java 5, Executor framework was introduced with the java.util.concurrent.Executor interface. This was introduced to fix some of the shortcomings discussed below.

1. The Executor framework is a framework for standardizing invocation, scheduling, execution, and control of asynchronous tasks according to a set of execution policies.


2. Even though the threads are light-weighted than creating a process, creating them utilizes a lot of resources. Also, creating a new thread for each task will consume more stack memory as each thread will have its own stack and also the CPU will spend more time in context switching. Creating a lot many threads with no bounds to the maximum threshold can cause application to run out of heap memory. So, creating a ThreadPool is a better solution as a finite number of threads can be pooled and reused. The runnable or callable tasks will be placed in a queue, and the finite number of threads in the pool will take turns to process the tasks in the queue.


Here is the sample code:



import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Sum  implements Runnable {
 
    private static final int NO_OF_THREADS= 3;
 
    int maxNumber;
 
    public Sum(int maxNumber) {
       this.maxNumber = maxNumber;
    }
 
    /** method where the thread execution will start **/
    public void run(){
        int sum = 0;
        for (int i = 0; i = maxNumber; i++) {
           sum += maxNumber;
        } 
        
        System.out.println("Thread " + Thread.currentThread().getName() + " count is " + sum);
    }
    
    
    /** main thread. Always there by default. **/
    public static void main(String[] args) {
       ExecutorService executor = Executors.newFixedThreadPool(NO_OF_THREADS);   // create a pool of 3 threads
       for (int i = 10000; i < 10100; i++) {
          Runnable worker = new Sum(i);               // create worker threads
          executor.execute(worker);                   // add runnables to the work queue 
       }
  
       // This will make the executor accept no new threads
       // and finish all existing threads in the queue
       executor.shutdown();
  
       // Wait until all threads have completed
       while (!executor.isTerminated()) {

       }
  
       System.out.println("Finished all threads");
    }

}

3. The Runnable interface's void run( ) method has no way of returning any result back to the main thread. The executor framework introduced the Callable interface that returns a value from its call( ) method. This means the asynchronous task will be able to return a value once it is done executing.

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;


public class Sum  implements Callable<String> {
 
 private static final int NO_OF_THREADS = 3;
 
 int maxNumber;
 
 public Sum(int maxNumber) {
    this.maxNumber = maxNumber;
 }
 
  /** method where the thread execution will start
    *  this can return a value
    */
    public String call(){
        int sum = 0;
        for (int i = 0; i <= maxNumber; i++) {
            sum += maxNumber;
        } 
        
        return Thread.currentThread().getName() + " count is " + sum;
    }
    
    
    /** main thread. Alwyas there by default. **/
    public static void main(String[] args) {
      ExecutorService executor = Executors.newFixedThreadPool(NO_OF_THREADS);                       // create a pool of 3 threads
      List<Future<String>> list = new ArrayList<Future<String>>(10);  // provides facility to return results asynchronously
     
      for (int i = 10000; i < 10100; i++) {
        Callable<String> worker = new Sum(i);                 // create worker threads 
        Future<String> submit = executor.submit(worker);      // add callables to the work queue
        list.add(submit);                                            // provides facility to return results asynchronously
      }
  
      //process the results asynchronously when each thread completes its task
      for (Future<String> future : list) {
        try {
            System.out.println("Thread " + future.get());
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (ExecutionException e) {
           e.printStackTrace();
        }
      }
  
  
      executor.shutdown();
  
      System.out.println("Finished all threads");
   }

}

The output is something like

Thread pool-1-thread-1 count is 100010000
Thread pool-1-thread-2 count is 100030002
Thread pool-1-thread-3 count is 100050006
Thread pool-1-thread-1 count is 100070012
Thread pool-1-thread-1 count is 100090020
...

4. The various Executor implementations provide different execution policies to be set while executing the tasks. For example, the ThreadPool supports the following policies:

  • newFixedThreadPool: Creates threads as tasks are submitted, up to the maximum pool size, and then attempts to keep the pool size constant.
  • newCachedThreadPool: Can add new threads when demand increases, no bounds on the size of the pool.
  • newSingleThreadExecutor: Single worker thread to process tasks, Guarantees order of execution based on the queue policy (FIFO, LIFO, priority order).
  • newScheduledThreadPool: Fixed-size, supports delayed and periodic task execution.
5. The ExecutorService provides facilities to shut down an application gracefully, abruptly, or somewhere in-between.

Q. What design pattern does the executor framework use?
A. The Executor is based on the producer-consumer design pattern, where threads that submit tasks are producers and the threads that execute tasks are consumers. In the above examples, the main thread is the producer as it loops through and submits tasks to the worker threads. The "Sum" (i.e. a worker thread) is the consumer that executes the tasks submitted by the main (i.e. consumer) thread. Check this blog to learn  more detailed explanation on the producer-consumer design pattern.


39 comments:

  1. Amazing article. I desire you up-to-date your blog significantly more often, I just cannot seem to be to acquire adequate of your blog. I preserved your blog in my bookmarks. Would it be feasible to do a guest post sometime?…

    ReplyDelete
  2. Good work.. keep it up

    ReplyDelete
  3. thanks for this blogger..
    I want to add one more question. Like what design pattern used in executor frame work?
    Ans: Produces Consumer design pattern

    Very nice effort by you. Please keep it up ..

    ReplyDelete
  4. Good point. I have updated it. Thanks for the feedback.

    ReplyDelete
  5. I did not understand most of the stuff :(

    ReplyDelete
  6. Do the rest of the multi-threading before you go to thread pools with the executor framework.

    Alternatively, if you post any specific doubts, one of the followers/readers or I can try and clarify it for you. This will also help me improve the post.

    ReplyDelete
  7. Thanks for your wonderful information which helped us to join java online training

    ReplyDelete
  8. Thanks for good posting on Thread pool

    ReplyDelete
  9. Hi,

    Is it safe to send JMS messages using the results of the thread completion.

    //process the results asynchronously when each thread completes its task from

    ReplyDelete
  10. Not sure what you mean. Can you please clarify it a bit more?

    ReplyDelete
  11. Hi ,
    Is it safe to use Executor framework within J2EE container?

    Thanks
    Ram

    ReplyDelete
  12. Spawning your own thread within a JEE container is discouraged because all resources within the environment are meant to be managed, and potentially monitored, by the server. There are ways to do this "correctly", but it is dependent on the platform being used. For example, The commonj WorkManager can be used for WebSphere and WebLogic JEE container as well as others

    ReplyDelete
  13. Hi Blogger,I think this is good online tutorial for learning totally java related technologies for freshers as well as experience candidates...U doing good job.One small doubt regarding web services...what is the use of web services in real time projects can you post one good example.

    Thanks

    Chandu

    ReplyDelete
  14. Good article. gave a good high level understanding of the executor framework. Can you please let me know what does spring use? I believe it uses the executor framework. Does the Executor framework used only for execution part? as in is it only a meeting point for a supplier and consumer OR does it handle the intricacies of thread handling?

    ReplyDelete
  15. Good Article ...

    ReplyDelete
  16. Very Nice work...really appriciated...

    ReplyDelete
  17. Very good work. I need to send about 100,000 emails (no spam) html with attachment. You can do it with thread pool? Also need to know their status, is this possible?

    Thanks in advance

    ReplyDelete
  18. very nice article. good understanding on thread pools using concurrency package.

    ReplyDelete
  19. Fantastic article... I am a newbie to multithreading / executors and was wondering what they were... This article is very good... thanks...

    ReplyDelete
  20. good article .. thank you.

    ReplyDelete
  21. Hello sir..very nice blog i have ever found.

    ReplyDelete
  22. Nice Article Appreciate your efforts !!!!

    ReplyDelete
  23. very informative articles.... thanks.

    ReplyDelete
  24. I stumbled on your blogs by chance and can't speak enough about how good , explanatory they are. You have done an amazing job in putting complex stuff in such a simple way. Thanks a lot and keep up the good work !

    ReplyDelete
  25. Hi Arulkumaran,
    Overall your explanation is good. But few places I see there are some fundamental mistake. If you please correct those places that it will be good for beginners.
    The first line of this page says,"A thread pool is a collection of runnables with a work queue." I think that is not correct. Essentially a ThreadPool consists of a collection of Worker Thread. The work queue is optional. though most of the advanced implementations have a configurable work queue. But the point is a thread pool is definitely not "a collection of runnables".

    ReplyDelete
  26. Thanks Santhanu for your valuable feedback, and it has been fixed.

    ReplyDelete
  27. Hi,
    In sample code the thread pool size is fixed no i.e 3. I want to know is there any memory issues if i use thread pool size as 100. How can i decide thread pool size for avoiding memory issues.

    Thanks,

    Amar Goud

    ReplyDelete
  28. It depends on how may CPU cores your machine has. It is not recommended to have too many threads. There is cachedThreadPool as well that you can use instead of the fixed thread pool. You need to make this value configurable in a properties file, and also need to conduct performance testing to monitor CPU and memory usage.

    ReplyDelete
  29. Hi Arulkumaran,
    Is there any calculation to decide no of threads feasible for one CPU Core.

    Thanks,
    Amar Goud

    ReplyDelete
  30. No. That is why performance testing is important. You can get some guidance to start with using Amdahl's law.

    ReplyDelete
  31. It is recommended to implement number of threads=number of processors+1

    Ex:ExecutorService executorService=Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors()+1);

    ReplyDelete
  32. Simply Brilliant.

    ReplyDelete
  33. Please correct the for loop in Runnable. Make it as below
    for(int i =0; i <= maxNumber; i++)

    instead of for (int i = 0; i = maxNumber; i++)

    ReplyDelete