Around 4 years back, We had a implementation where we pull the data from the remote database and dump it into remedy. For this we follow the procedure where we inovke the shell script and using JDBC and Java API's we inovke the procedure and dump in remedy. We pull around 12 k Records daily (Both Update and Insert). It is taking around 4-6 hours to run the load.

I checked the views where it get the data from DB they are pretty fast, came to know that its not problem with remote DB.
Mostly its towards the insertions taking long time.

I dont know Java API or how to debug/Optimize tihs performance.

Please help in this regards.

Thanks,
Mani

A bit late perhaps....

There are several places the time can be spent. And it could be other sources than just blaming Java.

1) Extracting data from source.

Are you using thousands of "API-calls" to pull the data from the remote database?
Can it be done by one "API-call" that selects what you want in one go?
An "API-call" will have an overhead and delay in itself in addition to the usefull stuff it is going to do. So having as few as possible is in most cases the best.

It could be easy to do a meassurement by just pulling the data from the external system (no work on data and no integration with ARS).



2) Handling the data in your glue-code.

In the glue-code you can in most cases optimize a bit, but most of the time is spent using the API-calls integrating the other systems.
However one can do stupid things that lead to wasting time in the glue code. There is no upper limit of this :-)

Again it can be meassured by starting a timer for whatever processing has to be done by the glue-code.

3) For the import updating and inserting entries there shouldn't be to many pits to fall into.
For each insert there should be maximum:
* 1 getlistentry
* 1 createentry

For each update:
* 1 getlistentry
* 1 getentry
* 1 setentry

If you need more than this, then perhaps there is something that belong to the processing stage. And if information is needed to be pulled from ARS, then it should be done efficient for all entries instead of on-deman for each and every entry to be inserted-updated.

Meassure this by doing a clean import (insert/update) of your records. For this to be of any value, you have to disable all the workflow that triggers on the import. You want to keep the workflow out of the picture.

4) Finally the workflow that gets triggered by the insert/update. 12000 records in 6 hours gives 1.8 seconds per record.

For this part one can manually or via the glue code import a single entry and having the filters enabled on the server.

Then by examining the filter-log, there is a resolution of subseconds on the various actions/filters running. It should be possible to spot where the time is spent.

If you get rid of some filter-operation that could for 0.3 seconds for each entry, then you have saved an hour of your runtime.



So to summarize:
* dont assume that Java is the one to blame.
* divide the flow into separate stages and meassure each of them.
* look into the stage that use the most time.

,