The memory used by the arserver.exe process is of concern. Once arserver.exe
starts to reach 900KB things start to go wrong. Using AR Server 5.0.1 I have
seen the problem with the error described in your first post. The first time
the error occurred all escalation processes shutdown when memory utilization
exceeded 1GB. The second time the error occurred escalations continued to
process, but I was unable to apply any changes through the Admin Tool when
memory utilization again exceeded 1GB.
2 changes were made in conjunction. 1 was applying the latest patch for
5.0.1. 2 was reducing the number of list and fast threads available to the
server. Since you are already on the latest patch, the 2nd option may be
your best bet. On your next restart enable thread logging to determine how
threads are being used, then adjust your minimum and maximum accordingly. I
would suggest reading the start of the log to determine if there is a large
number of threads opened on start (or within 15 minutes of start). The
minimum threads on startup should be derived from this number to lower the
additional overhead of opening new threads.
Making the 2 changes reduced the amount of memory used by AR System to 200MB
on average. This number will probably increase with the upgrade to 6.3 (just
because 6.3 uses more memory), but should not increase too much as more
memory issues are being resolved with a code cleanup.
Since you are connecting to a remote Oracle database, I would suggest
checking for patches on the Oracle client. Especially since 9i seems to have
so many memory leak issues.
Are your View Forms pointing to Views created from a Linked Server? If so,
you might want to consider having a DTS package move the data from the
Linked Server to a local table. When Remedy queries a View created from a
Linked Server, Remedy reads a View of the View which returns all records (at
least with a Linked Server that's not SQL Server). For every query, Remedy
will read every record in the table. For escalations this means a Remote
View with 2000 records will read all 2000 records, then read the details of
the 1st record, read all 2000 records, then read the details of the 2nd
record, and so on. By the time its all done Remedy has read 4,002,000
records.
Finally I would suggest monitoring Virtual Page Size. A lot of times when
looking at Memory the Page Size is ignored and only Physical Memory is
viewed. Most applications will reserve more memory in Virtual than in
Physical, even with Remedy. Often the Virtual Page Size used by an
application is 110% of the Physical. So if you're running 1GB Physical, then
1.1GB Virtual, a Page File with 3GB available will be quickly dissolved by
the remaining applications.
These are just starting points for investigating the memory problems. Any
positive changes to the system should be noted, as well as negative changes,
to determine the next course of action. I would suggest, and it is only a
suggestion, that once a method has been discovered for reducing memory
utilization that works for you, do not stop at just bringing down the memory
utilzation to a level where the errors won't occur (like around 800K), but
continue to find ways to bring memory utilization down even further, until
you reach a comfortable level.
- Daniel
-----Original Message-----
From: Ballantyne, Dougal [OS-IE] [mailto:DOUGAL.BALLANTYNE@SAIC.COM]
Sent: Friday, July 08, 2005 10:14 AM
To: arslist@ARSLIST.ORG
Subject: Re: Arserver.exe crashing
I have no escalation errors or escalations against view forms. Also I have
checked the arerror.log and it has happen a couple of times before. Only
this morning it happened 4 times. Also the memory usage of arserver.exe
reached over 1GB.
I have since restarted the services and it has dropped back to 40MB. Also
got a script watching the mem usage.
Logged a ticket with Remedy but they wanted API,SQL,Filter,Escalation
logging combined and Thread logging enabled. Managed to write over 50MB in
2mins, so can't really leave it running...
-Dougal
-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:arslist@ARSLIST.ORG] On Behalf Of Axton
Sent: 08 July 2005 13:59
To: arslist@ARSLIST.ORG
Subject: Re: Arserver.exe crashing
Does this coincide with an escalation running on your server that generates
a large number of errors? Also, do you have any escalations running against
view forms?
Axton
UNSUBSCRIBE or access ARSlist Archives at http://www.ARSLIST.org
(Support: mailto:support@arslist.org)
UNSUBSCRIBE or access ARSlist Archives at http://www.ARSLIST.org
(Support: mailto:support@arslist.org)
===================================================================
NOTICE - CONFIDENTIAL AND PRIVILEGED - This e-mail may contain
privileged and confidential information and is intended only for the
addressee named above. If you received this message in error,
please immediately notify the sender by return e-mail and delete the
original message; any distribution, copying or use of this e-mail
by you is strictly prohibited and may be unlawful.
UNSUBSCRIBE or access ARSlist Archives at http://www.ARSLIST.org
(Support: mailto:support@arslist.org)