Debatching in SOA suite 11g

Debatching is a feature where in you break a large input data in to a number of small inputs,this is generally done in order to reduce load on the server.

This will be very clear with the following example

Let suppose i have an input file with following data in a text file

called input.txt

ram,123,ibm
Shyam,234,cisco
Pasha,345,CTS
Kallol,456,TCS

As you can see it is a comma separated value.

We have a file adapter which will be polling for the record.
But assume that due to perfromance issue we want that the record
should be processed two at a time that is i want that first only

ram,123,ibm
Shyam,234,cisco

should be processed and then

Pasha,345,CTS
Kallol,456,TCS

should be processed in second stretch,however i dont want to intiate the process again.In this scenario i will use the debatcing feature of SOA Suite 11g.

This will process a single record twice using two records at a time thus will create two instance for the record.

We will see how we can achieve this target in SOA Suite.




My process will look something like this,

where in i am polling the same record which i have taken as example here

then finally in am just writing this to another file location.So its just a simple file inobund outbound scenario.

Now the only setting that you need to understand in debatching is in file filtering page as shown below




AS you can see we have selected the option

Files contain Multiple Messages PublishMessages in Batches of

and then i have chosen value as "2"

which means i want my data to be processed in a batch of "2"

Now if you will go to the code level .jca file for this

you will find that following property has been added for this

property name="PublishSize" value="2"

So now our requirement is fulfilled.

Now you can download my project ,change the location of input and output as per your server location and check the process with the same input as i have metioned here.

Two instance will be created and two output files will be generated for it.

failed to load resource: xmlGetAuditTrail.jsp?referenceId=bpel%3A//localhost/<<domain-name>>/<<process-name>>%7E1.0/5121210

you have recently upgraded to SOA Suite 10.1.3.5

and after that when you are trying to log in to BPEL console

you are getting some erros

It is something like this

failed to load resource: xmlGetAuditTrail.jsp?referenceId=bpel%3A//localhost/<<domain-name>>/<<process-name>>%7E1.0/5121210

this is beacuse of a jsp time out and can be resolved by increasing the
timeout values

go to the following location

$SOA_HOME\j2ee\oc4j_soa\applications\orabpel\console\ui\util2\1001.js

There search for rInfo.timeout

Increase this timeout from 20 to some higher value and try to open the console again ,it should work this time

SOAScheduler in SOA Suite 11g

Got an excellent white paper on how to create a scheduler for SOA Suite 11g.

I have not implemented it by my self but the document seems preety clear with the implementation.Please try to follow the same and let me know if you face any issues.

User is not authorized to login to Middleware Administration Server farm. User should be part of one or more Administrative roles to be able to login.

This issue mainly occurs when you have configured your SOA Suite with external ldap and you are tying to log in to the console with the users in your ldap.

There are lot of issues related to this.I will try to cover all the issue and try to explain this issue in detail.


I am covering this for SOA Suite 11g

When ever you create an authentication provider there are few things which you have to keep in mind.

1>The LDAP authentication provider should be first in the list of providers.
2>The control flag for LDAP and default authentication provider should be set to sufficient.

Once you do all these changes you should restart your server.

After restart you should be able to see all the users and groups in the weblogic console provided you have given the correct details for user and groups in configuration.


Now lets consider few error scenario and try to understand this.

Before that we will try to understand two terms

1>Authentication.
2>Autharization.

Authentication-Authentication is any process by which a system verifies the identity of a User who wishes to access it.

Now let suppose you will try to log in to the console with some unknown user and password so the system will not be able to authenticate it because it doesn't have any information on this user ,In that case you might receive an error like

Invalid User Name and/or Password.

This same error will occur even if you will try to use the LDAP user to login and use some different password as the user and password details provided in the ldap will not match the one provided by you that is authentication error.


Now next condition is that you will use the LDAP user which is shown up in weblogic console and you are also providing correct password but you are getting some different error

User is not authorized to login to Middleware Administration Server farm. User should be part of one or more Administrative roles to be able to login.

This is authorization error.

Authorization is the process of giving someone permission to do or have something.

In this case the user is authenticated but the user does not have the privilege to log in to the console this can be resolved by giving the user the roles to access the page.


Further there can be issue like

"@ User "weblogic" is not found in configuration "jazn.com" Check if the user
exists in the repository specified by the configurations. Check the error stack
and fix the cause of the error. Contact oracle support if error is not fixable."


this is a very known issue as been documented in following


It clearly says that if you have provider in this order

1>LDAP
2>Default

you will not be able to log in to the console using weblogic as provider.

These are the main issue which are being faced when ever we integrate our SOA with external ldap.

Getting Exception ORABPEL-11624 DBActivationSpec Polling Exception and BPEL instance are not created

You are on SOA Suite 10g and your BPEL process contains a database adapter.

Database adapter is polling data from a table and triggering bpel instances.

Adapter is polling the data but the bpel instance is not getting created.

checked the log files and got following error

ORABPEL-11624
DBActivationSpec Polling Exception.
Query name: [pollData], Descriptor name: [pollData.getData]. Polling the database for events failed on this iteration.
If the cause is something like a database being down successful polling will resume once conditions change. Caused by ORABPEL-00000
JTS transaction was marked rollback only.


Checked the thread dump and there found that most of the threads are waiting.

Don't know the exact reason of the issue.

But i made the following changes


as per the document

11.1 MaxTransactionSize Should be Limited for Oracle Database Adapter

In Database adapter, set a limited value for MaxTransactionSize. For example, set

MaxTransactionSize=10

Setting higher values to MaxTransactionSize will likely time out.


In my case it was set to unlimited.

Changed this to 100

Further i was sending data continuously send the data in a batch size of 200 and it started working

Request Entity Too Large While deploying a Process from BPEL Console

While trying to deploy a BPEL process from BPEL Console you are getting Following error message

Request Entity Too Large
The requested resource /BPELConsole/SupplyOrder/OrderToDisplay.jsp
does not allow request data with POST requests, or the amount of data provided in the request exceeds the capacity limit.

The reason is that the request entity size is much higher than the limit set in LImitRequestBody in apache server.

As per Apache document


LimitRequestBody directive

This directive specifies the number of bytes from 0 (meaning unlimited) to 2147483647 (2GB) that are allowed in a request body.

The LimitRequestBody directive allows the user to set a limit on the allowed size of an HTTP request message body within the context in which the directive is given (server, per-directory, per-file or per-location). If the client request exceeds that limit, the server will return an error response instead of servicing the request. The size of a normal request message body will vary greatly depending on the nature of the resource and the methods allowed on that resource. CGI scripts typically use the message body for passing form information to the server. Implementations of the PUT method will require a value at least as large as any representation that the server wishes to accept for that resource.

This directive gives the server administrator greater control over abnormal client request behavior, which may be useful for avoiding some forms of denial-of-service attacks.

If, for example, you are permitting file upload to a particular location, and wich to limit the size of the uploaded file to 100K, you might use the following directive:

LimitRequestBody 102400

The Oracle application server internally use Apache server.

We need to configure this to a higher value than the requested entity.

It can be set in following file

SOA_HOME\APache\APache\conf\httpd.conf

There you can set the limit request body to a value as per your requirement.

One thing you have to keep in mind is the value mentioned over there is in byte

We have

1 MB = 1024 KB

1 KB = 1024 Bytes

Therefor 1 MB = 1048576 Bytes.

So if you have a maximum of 1 MB of entity you need to set it to 1048576

SERVICEUNAVAILABLEEXCEPTION_STARTPAGE_STARTUP_INITIALIZE

You have installed SOA Suite 11.1.1.3 and created a domain for SOA and BAM.

Everything is up and running but while trying to log in to the BAM console you are getting following exception

SERVICEUNAVAILABLEEXCEPTION_STARTPAGE_STARTUP_INITIALIZE

BAM-0023: Service oracle.bam.adc.common.core.IDataStore is unavailable

EJB Exception occurred during invocation from home or business: weblogic.ejb.container.internal.StatelessEJBHomeImpl@b0e6d6e threw exception: oracle.bam.common.remoting.ServiceUnavailableException: BAM-00223: Service oracle.bam.adc.common.core.IDataStore is unavailable.>







Don't know the exact reason for this issue but the installation was done something like this by selecting following components



As you can see some extra components have been selected.

My requirement is only soa,bam and bpm server to be installed in the same domain

I recreated the domain this time by selecting first

Oracle SOA Suite(this automatically selected the related schema)
Oracle Enterprise Manager
Oracle Business activity Monitoring

Then created the domain and this time i was able to log in to the bam without any issues.

I further extended my domain again to contain BPM also.

Performance issue in SOA Suite 11g (100% cpu)

In soa suite 11.1.1.3 ,i have deployed many processes

The soa-server hangs at 100% and both EM and Weblogic-Console do not respond.

The cpu shows 100% and everytime i have to restart the server to make it working ,it works for sometimes but again after some time it hangs saying 100% cpu

I got following exceptions

oracle.xml.xpath.XPathException: value does not match required type

Caused By: java.sql.SQLException: Connection has been administratively disabled. Try later.

Exception [TOPLINK-4002] (Oracle TopLink - 11g Release 1 (11.1.1.3.0) (Build 100323)): oracle.toplink.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connection has been administratively disabled. Try later.
Error Code: 0
[TopLink Info]: 2009.11.07 10:59:45.125--ClientSession(89899333)--Communication failure detected when attempting to create transaction on database. Attempting to retry begin transaction.
Use JRockit instead of Sun's JDK

2>Review the performance tuning roadmap from oracle and find tune your jvm parameters.

3>There are lot of bugs associated with cpu utilization issue.
need to apply patches for those bugs,you can check with Oracle support on the patch fixes for the bugs.

4>Take heap dump and anaylse it using IBM memory analyzer tool.

5>If you are going for production it is suggested not to use embedded ldap server for storing the user and groups instead you should go for external ldap.

BPEL 10g purge Script

Bpel 10g purge script has been documented very well in the Oracle white paper

and this is the main concept in bpel 10g purge script

The purge only remove the rows from the extents and but does not coalesce the space.
To shrink the space:
1. Enable row movement for the table.
SQL> ALTER TABLE xml_document ENABLE ROW MOVEMENT;
Options:
2.1 Shrink table but don't want to shrink HWM (High Water Mark).
SQL> ALTER TABLE xml_document SHRINK SPACE COMPACT;
2.2 Shrink table and HWM too.
SQL> ALTER TABLE xml_document SHRINK SPACE;
2.3. Shrink table and all dependent index too.
SQL> ALTER TABLE xml_document SHRINK SPACE CASCADE;
3. Disable row movement for the table.
ALTER TABLE xml_document DISABLE ROW MOVEMENT;


Upgrade to the latest version to get the new purge script


If you will have a look in the purge script of ESB you will find that there are no script to delete the data from ESB_MONITOR topic and other topics this is because there are internal triggers implemented in code level which deletes the data from these tables also.