Users familiar with business process flow and terms of BPMN could relate what performer are meant for in a process. It can be users, queues, group, system(external system), adapters. Adapters are one of the most widely used performer types in real time flows because of its flexibility to integrate with external systems and adapt the task required at that point of the business flow.
In layman terms, a Bizlogic adapters allows applications to invoke java classes residing in context of Bizlogic server which could be running on remote or local host, to perform a designated task. Adapter framework internally use reflection mechanism to inject the class and its performing method while that adapter workstep is being executed. This definition of adapter workstep is done during design time on BPM Studio(shown below):
Class name and method name are important regardless of developer availing generate code operation or not. Generate code option would generate a skeleton class with class name and performing method available, along with getters and setters for each dataslots that adapter workstep carry. A developer can merely not choose to check that option and write a class of his own with name and performing method same as specified in the dialog above and makes sure to deploy to correct location, so framework can has the accessibility on run time.
Types of Bizlogic Adapters:
There are two types of adapters basically, synchronous and asynchronous. Synchronous adapters are supposed to be executed synchronous to the flow, in other words, regardless of how much time it takes for the adapter class to complete execution Bizlogic or Bizsolo engine waits for the execution to finish.
While on the other hand, asynchronous adapters are just the opposite and it means that when Bizlogic calls this adapter it does not wait for its class' execution to finish as against former kind and proceeds further with application flow. While adapter can finish with its own pace. This kind of adapter is recommended for long running tasks if any we need to perform. Asynchronous adapters can further be categorized into two sub categories, one without output dataslots and one with output dataslots. One without output dataslot is usual asynchronous type and do not need any additional care to construct(the adapter is completed and the application continues to the next workstep). While on the other hand one with output dataslots do behave more like synchronous adapter type because, to complete such adapter it has to call the function completeCallerWS(). The parameters are BizLogic session, process template name, workitem name, and a hashtable that contains dataslot values for you to update before completing the adapter.
The adapter implementation typically uses Java Messaging Service (JMS) APIs and infrastructure. When an adapter is invoked, a JMS messsage gets sent to a JMS Queue. A Message Driven Bean (MDB) listening to this JMS Queue picks up the message, decodes it and executes the adapter
Framework categorizes adapters internally based upon what a developer chooses at design time:
for example, when a user selects an adapter to be "long running" it is internally categorized as enterprise adapter and destination where the messages for these adapters will be sent will be BLEnterpriseAdapterQueue. While on the other hand if user choose it to be not long running, it is categorized as process adapter and messages for these adapter are sent to BLAdapterQueue. There are dedicated MDBs listening to respective queues(mentioned above) to process messages coming in.
Process adapter type can be further categorized to inline adapters. Inline adapters were designed to overcome the overall JMS infra usage of adapter framework. When an engine detects an adapter with "execute in the same thread" flag checked, it directly executes the adapter in the same thread instead of sending a JMS message. Thus, the adapter is executed using the same thread that completed the previous workstep, it is more efficient than usual message sending and consuming way of adapter framework.
Adapter Class Loading:
Business manager application can reload latest version of adapter class dynamically from Bizlogic server without restarting the server. The Bizlogic server is configured to dynamically reload adapter classes from the SBM_HOME\ebmsapps directory defined in sbm.conf file as parameter sbm.application.home. So, when the resources specific to adapters are deployed it goes to sub directories corresponding to SBM_HOME\ebmsapps.
BizLogic allows loading class files from specific locations as well as from JAR files.
• Locations common to all applications
• Locations specific to an application
• SBM_HOME\ebmsapps\<application name>\lib
Basic essence of dynamic class loading(DCL) is, if there is a class that is already loaded, is modified, or is moved to a new location, then the class needs to be loaded again. The very check "if class is modified" is based on the last modified time of the class file. If the class is in a JAR file, then the check is based on the last modified time of the JAR file as well as the last modified time of the class file(We will talk about DCL in details on different post).
Troubleshooting adapter flow in Bizlogic engine:
In Savvion installation under conf directory the file bizlogic.conf has various debug keys that when enabled logs detailed information on operation type engine is currently executing. Key specific to external performer(adapters) is bizlogic.debug.ep, set this to true to get detailed info on adapter execution.
All supported application server has their own way of dealing with JMS related task, but we will take examples of most widely used application server among the prospects i.e IBM WebSphere Application Server(WAS).
WAS handles all JMS related task with its reserved thread pool named SIBJMSRAThreadPool. An example log message from Bizlogic engine appears as below:
>> BizLogic | DEBUG | ejbServer | BizLogic | EPM.readDataFromMsg(): | SIBJMSRAThreadPool : 6 |#]
So, one should look into such threads specifically for details on adapters execution. But, again there are many other task within Savvion that are handled by JMS infra so this thread pool might have threads performing other operations too(for example tasks from service MDB daemon). Threads that involve com.savvion.sbm.bizlogic.server.EPManager operations are one of your interests if you are troubleshooting adapter threads in JVM.
Frequently encountered issues with adapters on production:
BLAdapter thread count reaching zero!:
This problem usually shows up in application server like Weblogic. Even though this count means that configured executable thread count for this pool has reached its limit, internally, Weblogic pushes all StandBy thread count as well to serve the request coming in. But, if an environment is frequently hitting this situation, it is always recommended to tune this pool and increase the thread count(for Weblogic Server).
Remember, increasing thread count is easy but it very tightly depends on system resources, for example CPU capacity, memory availability and if the adapters frequently does DB operations then involved datasource's pool size as well. For WAS this here is the default setting and how to increase it to custom level.
Adapter stuck in activated state!:
Most frequent and troubling problem in production scenario. Usually when a user complains that adapter is stuck in activated state, it means the state of adapter is active and would not transition to the next state(i.e completed) on its own. There are many possibilities resulting in this state, for example an adapter performing particular operation when activated and took more than the JTA timeout. This is not a normal behavior because and adapter execution is supposed to be as quick as execution of independent java client, hence it requires manual intervention to push flow forward.
This problem is too dependent to environments and also kind of operation that adapter is supposed to perform. It is hard to replicate this behavior in house until such production load is generated and other factors that catalyzes are present. First approach that a level 1 support monitoring/maintaining application takes in such cases is, restart Bizlogic server and it works. If the requests are too much piled up some of these gets cleared up and then rest stagnates again at same state resulting in team taking multiple restart of Bizlogic server to clear entire request(waiting on this state)
As you already know adapter uses JMS infra to complete its execution(sending>Receiving>Processing of messages). If an adapter is stuck in activated state that message sent on behalf of this adapter is gone and is either not processed or lost and this is why it stays in the same state forever until Bizlogic server is restarted. What happens behind the scene is when the Bizlogic server is restarted, it actually re-sends all messages for adapter that were in activated/suspended state while server was being stopped. This is called restarting external services internally. This pushes the activated adapter message once again to the queue and makes it available for MDBs to process it. It is tiring process and most of all not acceptable on most production environment(although this is only first aid that could be done until now for this problem).
We could extract the essence of what happens during Bizlogic server restart and use it through a Bizlogic application to simulate restart. In other words, it is possible to extract the operation specific to external service that happens during restart and stuff it within Bizlogic adapter and run it as application within same environment. To go into further details, behind the scene Bizlogic engine calls >> BLControl.doRestartExternalServices(); to resend messages for already activate and suspended adapter. Now this call is specific to clustered environment and engines makes sure it checks for it( also controlled by parameter exposed in configuration file). It is very rare that this situation would show up on standalone environment, but if it does we might have to handle setting the param on the fly like >> BLUtil.self().setClustering(true);; to finally call doRestartExternalServices(). We will talk about that later if needed, but meanwhile after all we discussed lets compile it in short snippet and this is all we need to put in a performing method of an adapter of an Bizlogic application that can have just this adapter as a workstep:
System.out.println("Resuming external services");
System.out.println("Done Resuming external services");
If this application is deployed to the environment that run into mentioned problem, we can create instances of it to signal engine to restart external services the way it happens when Bizlogic server is restarted. It will avoid manual bouncing of servers to clear out the stuck adapter stack(refer).
Note: User can detect all adapter worksteps in activated status by executing the following query:
select process_instance_id, workstep_name from bizlogic_workstepinstance where type=107 and status =18;
Adapter suspended but the resume does not work!:
When the adapter execution fails and throws an exception to the caller, the BizLogic server captures that remote exception, logs the message and stack trace, suspends the Adapter workstep. BizLogic tries to re-invoke the adapter. After specified re-tries(configurable), BizLogic generates two events, EP_AFTERBREAK and W_SUSPENDED.
Then, according to the requirement, applications can use any of the events to take further action. EP_AFTERBREAK is fired to register that retry for executing adapter was done and it could be fixed within span retries.
Bizlogic.log and EJBServer.log (SystemOut.log in WAS) is the best place to look for the reason of failure in such cases. If the logs are rolling over with high rate we can look up the event for the PI that has adapter failing and filter those events mentioned above on this adapter workstep. These events carry stack(reason) for failure. Any admin user having access to BPM Portal admin tab can browse to Audit event section and can filter with event, PT, PI etc.