Running SAP systems on a Windows operating system provided me with some difficulties when it came to understanding memory usage. I was used to Unix based systems and using terms like “swap file”, “virtual memory” etc.
Hopefully this article will help you diagnose SAP Java system memory issues on Windows, especially “Out Of Memory” (OOM) error situations in the SAP Java stack.
It has been written based on Windows Server 2003 64bit with SAP Netweaver 7.0 (2004s) and Oracle 10g (10.2.x).
REFERENCES:
Understanding memory usage in Windows 2000.
SAP Note 723909 v33 – Java VM Settings For J2EE 6.40/7.0
SAP Note 879377 – Adjusting SDM Java Memory Usage
SAP Note 716604 v56 – Sun J2SE Recommended Options
SAP Note 313347 – Windows Editions Memory Support
Netweaver 7.0 SR2 Java Windows Oracle Installation Guide.
WINDOWS MEMORY TERMINOLOGY
Before we go into the memory details of an SAP system on Windows, it is useful to understand the memory terminology of the Windows operating system.
This is useful if you come from a UNIX background.
RAM – Random Access Memory, physical installed memory in the server.
Page File – An area of hard disk (or other storage medium) used for temporary storage of memory blocks. Generally this should be manually set according to the installation guide for the SAP system.
Virtual Memory – The logical memory accessed by user space program (calculator or any other app) this is roughly the summary of physical RAM plus page file size.
Kernel Memory – This is the area of virtual memory and RAM used by the operating system for device driver memory (non-user space programs).
Page Faults – When a process tries to access a portion of virtual memory that has been swapped to disk, a page fault occurs and the swapped portion is read back into RAM from disk potentially displacing an existing memory page).
Page – A block of memory usually around 4K in size.
DETERMINE OS MEMORY
We need to first determine how much memory is actively being used at the operating system level and how much memory is available to us.
On the Windows server, open Windows Task Manager and click on the “Performance” tab:
This will show the current memory usage of the entire server.
Ignore the line graphs and histograms as the bottom section of the pane is more important.
The following sections are described:
Physical Memory – Total Total amount of RAM installed in server.
Physical Memory – Available RAM available for CPU processes.
Physical Memory – System Cache RAM being used by the file cache.
Commit Charge – Total Size of virtual memory in use.
Commit Charge – Limit Max amount of virtual memory.
Commit Charge – Peak Virtual memory used high water mark.
Kernel Memory – Total Paged and non-paged mem used by the operating system’s kernel e.g. device drivers etc.
Kernel Memory – Paged Virtual memory set aside for the kernel.
Kernel Memory – Nonpaged Physical RAM set aside for the kernel.
The most important to watch when measuring memory usage of an SAP system are the following:
Commit Charge – Peak: This is the high water mark for memory usage on the server. If this is close to the maximum available virtual memory (Commit Charge – Total) then you could be suffering performance issues caused by excessive paging to/from hard disk.
Physical Memory – System Cache: This is the file system cache that stores data accessed from the hard disk in RAM for more efficient data retrieval.
This value is maintained by the operating system but can be influenced by setting the performance options in “My Computer -> Properties -> Advanced -> Settings (Performance) -> Advanced” shown below:
Which is also related to (or rather overridden by) the setting of “Maximize data throughput for network applications” in the “File and Printer Sharing for Microsoft Networks” of the local network connections in control panel:
MANAGE OS MEMORY
Before you try and tune the memory in the SAP system, you should first explore all avenues of optimising the amount of RAM available in the operating system.
We can see that there are three factors that affect the amount of RAM available for use with programs:
1, Other programs running.
2, The file system cache.
3, Device drivers loaded in the system (kernel).
Reducing the number of running programs and services is a good way of freeing up memory.
The amount of memory used by a specific program or service can be seen in Task Manager on the “Processes” tab:
Only the needed applications for the application and its management should be running.
The svhost.exe binary signifies services that are visible in the services applet in the Administrative Tools applet in Windows Control Panel.
Disabling redundant services will improve available RAM.
You should seek to reduce the file system cache as much as possible.
Since the SAP system is not a file server, any file system level buffering of data is only wasting resources. This is especially true for the Oracle database which buffers read data in memory anyway, which is then potentially buffered again in SAP.
You can reduce the system cache by setting the “Maximize throughput for network applications” as discussed in the previous section. This should be done on all servers within the SAP landscape (apart from specific cache servers, subject to SAP recommendations).
This is generally recommended by SAP in the installation guide for the Netweaver system.
Lastly, disable any redundant devices from the device manager.
These devices require kernel memory, some of which is taken directly from RAM.
You can see the memory ranges allocated to devices in the device manager accessed from the “Computer Management -> Device Manager” option from the Administrative Tools applet in Windows Control Panel.
Simply select “View -> Resources by Connection” in the Computer Management panel:
Expand the “Memory” item from the tree on the right and the devices will be listed with their memory address ranges:
The address ranges are in hexadecimal notation and in the form [RegionLetter][HEX Values] e.g. A0000000 – AFFFFFFF. The value in hex is bytes and is calculated as follows: FFFFFFF = 4294967295 bytes in decimal. Divide the decimal by 1024 accordingly to get a value in Kilobytes, Megabytes or Gigabytes.
Devices which are not required can be disabled (right click and choose “Disable”).
DETERMINE SAP JAVA SYSTEM MEMORY
The memory allocated to an SAP system comprises of three parts:
1, The Database instance memory.
If the database is on the same server as the SAP system, then some of the operating system memory will be taken up by the database itself.
2, The ABAP system memory.
If the SAP system is a dual stack system with an ABAP instance also on the same server then some memory will also be taken up by the ABAP instance.
3, The Java stack memory.
The Java stack comprises of mainly three (sometimes four) running Java Runtime Environments (Server, Dispatcher & SDM).
The main memory of the server must be divvied up between these 3 main components of the SAP system.
Since we are concentrating on the Java stack only, we will assume that SAP recommendations were followed correctly for the setup of the database and ABAP components.
Using Windows Task Manager it is possible to determine the amount of memory in use by the SAP Java system.
The jlaunch.exe processes represent the Java Runtime Environments of the Java stack and therefore the memory usage for each process equates to the current virtual memory usage for the SAP processes:
In the Windows Task Manager you must change the view to be able to see the process ID of a process:
Ensure that you select the following columns:
PID,
Memory Usage,
Peak Memory Usage,
Page Faults
Virtual Memory Size,
Paged Pool,
Non-paged Pool
With these columns selected, we can get a far better overview of the memory usage for the SAP components:
You can now see that the actual virtual memory usage (VM Size) of the PID 5144 (what we think is the SAP Java server process) is ~1.7GB in size (1,678,188 K).
The summary of the VM Size column should roughly add up to the size of the “Commit Charge – Total” value displayed on the “Performance Tab”.
Generally you can guess that the jlaunch processes with the highest memory usage will be the SAP Java server processes (since they have the largest memory settings out of the box), followed by the dispatcher and finally the SDM (with the smallest default settings).
For confirmation, it is possible to find the exact process id using the jcmon.exe program.
Use option 30 (Shared Memory Menu), then select option 2 (Display Process Data):
The running components will be displayed and we can confirm that PID 5144 is in fact the server0 process:
DETERMINE SET MEMORY VALUES
We have determined what virtual memory the SAP Java processes are actually using but we need to see the configuration settings of the Java processes so we know what to expect at startup and maximum use.
There are a number of different areas of memory for a Java Runtime Environment.
The following is a very brief overview:
Heap Size The total amount of memory stack available to a Java Runtime Environment process (excludes the resident memory for the binary). The minimum heap memory is set using parameter “–Xms”.
New Size Also known as the “young generation”, is the amount of heap memory allocated by default for new java classes created within the JRE. The new size is set using parameter “–NewSize”.
Perm Size Also know as the “old generation” or “tenured generation” is the amount of heap memory allocated by default for java classes that have survived a number of garbage collections and are deemed “permanent” in the JRE. The perm size is set using parameter “–PermSize”.
Any java classes that are no longer referenced are cleared from memory during the garbage collection cycles which are either time-triggered or based on a percentage of free heap size.
The sections of heap memory and the heap memory total size itself can be set within bounds using the “-MaxXXXXSize” arguments or “-Xmx” (for the heap itself) to the JRE.
The JRE will not break these bounds.
If the amount of Heap allocated is not enough for the running JRE it will constantly try to garbage collect more aggressively. This will be seen in the developer trace files e.g. dev_server0, dev_dispatcher and dev_sdm.
If the JRE is still unable to reference enough memory, then a Java.Lang.OutOfMemoryError will be thrown. If this is visible in the dev_server0 trace file, then this is the JRE that threw the error and was suffering the problem (more than likely).
If no maximum heap size is set using the –Xmx parameter, then the JRE will attempt to expand the heap dynamically when required (it may not shrink back).
If the Windows virtual memory is exhausted, then the JRE will eventually throw a Java.Lang.OutOfMemoryError and Windows may throw its own error.
Now we understand roughly how the JRE memory is utilised, we can see the settings for the respective SAP JREs for the server, dispatcher and SDM.
The JRE server0 memory allocation can be seen either through ConfigTool by selecting “Cluster-Data -> Instance ID XXXXXX” -> Server” and checking the parameters on the right hand side:
Or by looking in the jlaunch process trace file:
This concludes the analysis of memory usage for the Java stack.
Using this information you should now be able to determine if this meets the recommendations by SAP.
SAP JAVA STACK MEMORY RECOMMENDATIONS
During the creation of this article a number of SAP notes became of significance.
Generally SAP try to recommend certain settings during installation of the SAP software, and then at a later date, they will release a note that details changes to the installation settings.
The base install of SAP Netweaver 7.0 Java (SR3) seems to install with the following settings:
Server:
-Xms 1024m
-NewSize 171m
-MaxNewSize 171m
-PermSize 256m
-MaxPermSize 256m
Dispatcher:
-Xms 170m
-NewSize 57m
-MaxNewSize 57m
SDM:
-Xms 256m
The installation documentation specifically states that the sizing of the Windows page file is essential for correct operation of the SAP system.
The document states in the section “3.1.2 Requirements Checklist for a Central System”:
“Minimum RAM:
1.5 GB
To check RAM, in the Windows Explorer choose Help About Windows.
Paging File Size: 1 times RAM plus 8 GB”
This would infer that on a system with 10GB of RAM, the page file should be 18GB in size. This would give a total virtual memory size of 28GB.
According to note 723909, SAP’s latest recommendations for sizing the JRE memory is as follows:
“For 32 bit platforms we recommend to start the server nodes with 1GB
whereby the initial and the maximal heap size should be equal: -Xmx1024m
-Xms1024m
Higher values may cause troubles, for example see notes 736462 and 664607
about Windows DLLs preventing big java heap.
We recommend using additional server nodes instead of huge heaps.
For 64 bit platforms (that means not only your OS but also your JDK
supports 64 bit) we recommend using 2GB: -Xmx2048m -Xms2048m
Take into account while planning your productive landscape: for NW 7.0
(04s) there is a general recommendation to use 64bit systems, see note
996600.
2.2 For dispatcher nodes it’s sufficient to set the heap size to 171m on 32
bit and to 256m on 64 bit platforms. There is no need to apply the
parameters mentioned under 4-9 below.
Default size of the young generation (one third of the heap) is ok for
dispatchers.
If the instance contains more than three server nodes we recommend to add
50MB of dispatcher heap for each additional server node.
2.3 The max heap size (plus max perm size) of all Java server nodes
must fit completely into the physical memory in order to avoid heavy
paging on OS level. You must also consider the space needed for OS
and other processes.
For each JavaVM on the server, all Java memory must fit completely
into physical memory (no paging)!
Therefore the rule of thumb for number of server nodes for SUN and HP
JavaVM with 1 GB Heap each will be
#ServerNodes = (AvailableMemory / 1.5 GB)
and factor 2.5 should be used instead of 1.5 for 64 bit with 2GB heap size.”
The additional note 716604 recommends some specific settings for the SUN JVM:
“For 6.40/7.00:
For the heapsize and the size of the new generation please take care of the
recommendations of note 723909.
For Windows (ia64 and x86 (64Bit)) and Linux/ia64 (Itanium) platforms set
the stack size to at least 2MB for each serverID, the dispatcher and the
Java based tools:
-Xss2m
Additionally, increase the stack size of the JIT compiler threads at least
on the server nodes of the J2EE Engine to 4MB on these platforms with
following parameter (effective on Windows starting with 1.4.2_17 b12 resp.
1.4.2_18):
-XX:CompilerThreadStackSize=4096”
Any modifications to the Java stack should be proven in the DEV and QA environments before application to production.
