Tuesday, October 3, 2017

Threads in Managed Environments. Work Managers

We pay for modern application servers since they provide a managed environment for our applications. An application server implements some APIs, for example Java EE 7 or Java EE 8, as well as provides some capabilities such as application life-cycle management, transaction management, resource access and thread management.


Thread pool


An application server uses a thread pool to provide the thread management capability. While an application deployed on the server works, an application thread isn't created when a new request is accepted but taken from the pool. This approach protects the server from creating a lot of threads and overwhelming the operating system by the duty to process too many threads. The goal has been pursued through blocking accepted requests if there are no threads in the pool.

The IT team can specify the following parameters of the thread pool:

  • thread priority - ranges threads created by a number of pools by priority. A user request to a business critical application hangs other threads in the system.

  • number of threads - limits the number of concurrent threads executing requests. Modern application servers, for example Oracle WebLogic, let us set up the limit not only as a constant value but also as a reference to a data source so the maximum number of thread would be equal to the capacity of the connection pool related to the data source. On thread gets a connection to the database.

The application server takes into account the above parameters in cooperation with some inner optimizations by analyzing the current workload, the number of available processors and the amount of free memory.


Configuring the WebLogic thread pool


Oracle WebLogic Server implements a concept referenced as 'Work manager'. Work manager is a mechanism that allows you to manage and utilize threads and create rules/guidelines to follow when assigning requests to threads. Work manager may be configured at the domain level, application level, and module level for a single application, servlet, or EJB-component.

A global Work manager can be created using the administrator console or WLST. An administrator can log in to the Weblogic console, select Environment -> Work Managers from the domain structure tree.


Clicking on the link opens the Summary of Work Managers page on which a button for creating the following configuration elements is present:
  • Work Manager. The below objects are used as parameters for the appropriate properties of a Work manager.

  • Response Time Request Class specifies a response time goal in milliseconds. Response time goals are not applied to individual requests. Instead, WebLogic Server computes a tolerable waiting time for requests with that class by subtracting the observed average thread use time from the response time goal, and schedules schedule requests so that the average wait for requests with the class is proportional to its tolerable waiting time (Here and bellow, the quotes are from the official documentation).

  • Fair Share Request Class specifies the average thread-use time required to process requests. The default is 50. Let's imagine, there are two modules: A and B, also two fair share request classes are configured, the first one has a priority equals to 80 while another one has a priority equals to 20. WebLogic Server ensures 80% of overall execution time for module A and 20% to module B. The value of a fair share request class is specified as a relative value, not a percentage. Therefore if two request classes were defined as 400 and 100, they would have the same relative values as 80 and 20 or 4 and 1, respectively.

  • Context Request Class specifies a class for requests based on context information, such as the current user or the current user’s group.

  • Maximum Threads Constraint limits the number of concurrent threads executing requests from the constrained work set. The default is unlimited. A max-threads-constraint can be defined in terms of the availability of a resource that requests depend upon, such as a connection pool.

  • Minimum Threads Constraint guarantees a number of threads the server will allocate to affected requests to avoid deadlocks. The default is zero.

  • Capacity Constraint causes the server to reject requests only when it has reached its capacity. The default is -1. Note that the capacity includes all requests, queued or executing, from the constrained work set.


Constraints take precedence over request classes.


An application uses a global-defined Work manager as a template. Each application creates its own instance of the Work manager that executes requests for this application and isolates these works from others. So, each application can be stoped without any negative impact on another one. Although each application uses its own instance of the Work manager, constraints and request classes are shared among them.

An application-level or module-level Work manager should be configured in one of the following weblogic-...xml file:
  • weblogic-application.xml - for an application
  • weblogic-ejb-jar.xml - for an EJB-component
  • weblogic.xml - for a servlet

For example, a weblogic.xml which contains a Work manager as well as Maximum and Minimum threads constraints is shown bellow:



How to use a Work manager in application code


WebLogic Server implements the JSR-237 Timer & WorkManager API as the CommonJ library. Let's explain the commonj.work Java package.

The Work Manager API provides the following interfaces:

  • WorkManager contains a number of methods to submit a work for execution. Each instance of WorkManager returns a WorkItem.

  • WorkItem is used to get a status of a submitted to a WorkManager instance of Work.

  • Work allows to asynchronous execute a code. The code should be put into the method run() of the interface. In other words, this is a task executed by the WorkManager API.

  • WorkListener is a callback interface that provides communication between a WorkManager and a submitted task specified as an instance of Work. The WorkListener interface can be used for getting the status of a Work. Note: WorkListener is always executed within the same JVM the original thread, which started the Work, runs in.

  • WorkEvent sends to a WorkListener as an image of a Work executed by a WorkManager.

  • RemoteWorkItem is an extension of the WorkItem interface and allows to works be executed remotelly. Serialized works can be executed on any member of a WebLogic cluster.

An example of leveraging the API in a code:


To inject a WorkManager into the JNDI-tree and make it available from the servlet, the WorkManager must be specified as a resource-reference in the web.xml deployment descriptor:


A WorkManager can be not only lookuped from the JNDI-tree but also injected into the servlet as a resource:


The output of the servlet is present below. Both tasks specified in the code is executed by the wm/somews WorkManager in one thread since the MinThread and MaxThread constraints for the Work manager is set up to one. The servlet itself works by the default Work manager.


Note: Work Manager API doesn't support neither failover for being executed works nor persistent mechanisms for them. If a WebLogic Server instance should fail or are administratively shut down, any work instance is being executed on the server will be lost.

How to set up a Work manager for a Servlet


A server administrator can specify a custom Work manager for a deployed component so the component will use this Work manager for dispatching incoming requests. Components in this context include Servlets, EJBs, OSB services, etc.

In order to specify a Work manager for a Servlet, the JNDI-name of the Work manager has to be assigned to the wl-dispatch-policy Servlet parameter:


Well, the Servlet uses exactly one thread to handle all incoming requests:


How to set up a Work manager for a Session EJB


In order to assign a Work manager to a session bean, the JNDI-name of the Work manager must be specified in the wls:dispatch-policy subelement of the wls:weblogic-enterprise-bean element, which describes the EJB, in a weblogic-ejb-jar.xml file.


The Work manager also can be specified in the file.

The code of the session bean is the following:


EJB prints the Work manager's name:


How to set up a Work manager for a Message-Driven Bean (MDB)


An MDB listerns for messages from a JMS- or JCA- data sources. If no Work manager is assigned for an MDB, the default one will be in use and 16 threads will be performing message consumption. Sometimes this number of threads might be considered as excessive or insufficient.

A Work manager is assigned to an MDB in the same way as for a session bean: the JNDI-name of the Work manager must be specified in the wls:dispatch-policy subelement of the wls:weblogic-enterprise-bean element, which describes the MDB, in a weblogic-ejb-jar.xml file.


The code of an MDB is the following:


If there is a Max thread constraint for the Work manager wm/DemoWorkManager is equal to 2, the following messages will be displayed on the console:


Well, as it can be observed, the messages are printed in pairs.

The number of setup subscribers is displayed on the WebLogic console. For the above example, the number equals to the constraint is assigned to the Work manager - 2:


Note: The following WebLogic resources also must be specified in order to performing message consumption concurrently in two threads:
  • A connection factory to be used by the MDB:


  • Set the value of the Maximum Messages per Session parameter to one:


How to set up a Work manager for a Message-Driven POJO


A Message-Driven POJO is a plain old Java class which implements the javax.jms.MessageListener interface but is no an EJB. The Spring Framework can be leveraged to make a connection between a Message-Driven POJO and a message queue.

The Spring Framework provides a support for the CommonJ API such as the org.springframework.scheduling.commonj.WorkManagerTaskExecutor class. An instance of the class could be injected into other components provided by the Spring Framework which contain the appropriate properties; for example, into instances of the org.springframework.jms.listener.SimpleMessageListenerContainer class.


The code of a Message-Driven POJO may look like the following:


A reference to the Work manager must be specified in the web.xml file otherwise the Work manager would be unavailable for the Spring Framework.


The Message-Driven POJO starts listening to the queue after application deployment. In doing so, regardless of the actual Work manager configuration, WebLogic console always will display 16 subscribers on the queue - the default value.


If the max thread constraint is set up to one, messages from the queue will be consumed strictly in one thread:


If the max thread constraint is increased to three, messages from the queue will be consumed concurrently by three ones every time:


No further configuration for the message queue for a Message-Driven POJO is needed. An example of Message-Driven POJO can be found on GitHub.

UPD 03.10.2017

Why our Work Managers may need some tuning


First of all, I need to say that the standard ('default') Work Manager is entirely permissible: a separate Work Manager with the default configuration will be created during server starting for every deployed application. An additional Work Manager should be defined only in the following cases:

  • By default, all threads have the same priority; if this behaviour isn't suitable, the Fair Share parameter must be set.

  • There is a response time goal assigned to the server; the Response Time parameter must be set.

  • A deadlock (e.g., during server-to-server communication) might be met, a Minimum Thread Constraint should be created and assigned to the Work Manager.

  • Applications use a common JDBC connection pool, a maximum number of available threads (Maximum Threads Constraint) for the applications must be limited by the pool capacity.

Separately, if Oracle Service Bus is deployed to Oracle WebLogic and the Service Callout action is used, each Proxy- and Business-service invoked using a Service Callout should have its own Work Manager. More information can be found in the Following the Thread in OSB article by Antony Reynolds.

Work Manager configuration rules

There are a number of best practices to configure Work Managers:

Work Managers can be defined on the application level in a weblogic-application.xml file. Using local-defined Work Managers simplifies application portability between development, testing and production environments.

If some configuration parameters such as Request Class or Threads Constraint are shared among a number of Work Managers, the parameters have to be specified outside of the Work Manager definition. Also please remember the following rule: if a configuration component, for example Maximum Thread Constraint, is used by several Work Managers, the defined thread count is shared among that Work Managers as well. Let's have a look at the example: Work Managers A, B and C use the same Maximum Thread Constraint D and the constant has 4 assigned as the thread count, Work Manager A has got 3 threads from the pool while Work Manager B has got 1 thread, so there is no thread for Work Manager C.

Some applications, for example Oracle Service Bus, don't allow to define Work Managers using application deployment descriptors. Work Managers have to be specified on the global level using the WebLogic Administration Console or WebLogic Scripting Tools. These Work Managers can be used by any application deployed on the application server.

How an application is looking for the Work Manager

If numerous Work Managers are created, deployed applications may not work as expected. Every WebLogic administrator must understand the algorithm of the Work Manager lookup process. The algorithm is very simple:

  1. The application is looking for the Work Manager defined in the deployment descriptor.

  2. If the local-defined Work Manager isn't found, the application will look for the Work Manager defined on the server level.

  3. If the server-defined Work Manager isn't found, the following rules will be applied: the Fair Share value will be set up to the defaults, constraints and capacities will be set up to null, so these parameters won't matter.

Conclusions


The article considers one of the most important components of a modern application server, Work manager. The component is used for application server performance tuning according to the hardware capacity and the priority of executed tasks.

Oracle WebLogic Server leverages all capabilities provided by the concept of Work manager for concurrent thread management. Almost every Java EE component (a Servlet, Session- and Message-Driven Bean) and even POJOs as well as Oracle Service Bus services can be configured to manage used threads by the specified Work manager. A number of request classes and constraints may be taken into account to tune the behaviour of that Work Manager.

No one shall use the Thread class in an enterprise application to start new threads. These threads won't be managed by the application server and may be a root cause of a significant performance degradation. If a task has to be executed concurrently, the CommonJ API or Concurrency Utilities from Java EE 7 are the best solutions.

References



Would you like to give a 'Like'? Please follow me on Twitter!