This blog contains experience gained over the years of implementing (and de-implementing) large scale IT applications/software.

Hardening SAP Hostagent SSL Connections

You may have recently had a penetration test and in the report you find that the SSL port for the SAP Hosagent (saphostexec) are listed as allowing weak encryption cipher strength and older SSL protocols.
You want to know how you can remedy this.

In this post I will show how we can appease the Cyber Security penetration testing team, by hardening the SSL ciphers and protocols used for connections to the Hostagent.

What Are Weak Ciphers?

Ciphers, like Triple-DES and Blowfish use 64-bit block sizes (the cipher text is split up into blocks of 64-bit in length) which makes a block cipher more vulnerable to compromise, compared to a cipher that uses a larger 128-bit block size.

The cipher is agreed upon during the setup of the SSL connection between the client and the server (SAP Hostagent in our scenario).
If a server advertises that it supports weaker ciphers, and a client elected to use one of the supported weaker ciphers during the SSL connection negotiation, then the connection could be vulnerable to decryption.

What Are Older SSL Protocols?

Through time the SSL protocol has been improved and strengthened.
The SSL protocol versions go from SSL v1.0 to SSL v3.0, then re-named to TLS and the versions again incremented from TLS 1.0, TLS 1.1, TLS 1.2 and the most recent TLS 1.3 (in 2018).

The old SSL versions of the protocol are deprecated and should not be used. The slightly newer TLS versions 1.0 and 1.1 are also now widely deprecated (do not confuse “deprecated” with “unused”).

It is therefore recommended, generally, to use TLS 1.2 and above.

Why Harden the Hostagent SSL Service?

Now we have an appreciation of our older ciphers and protocols, let’s look at the Hostagent.
Usually the PEN test report will highlight the SSL TCP port 1129, and the report will state two things:

  • The SSL ciphers accepted by the Hostagent include weaker ciphers (such as RC4).
  • The SSL protocols accepted by the Hostagent include TLS 1.0 and 1.1.

The above issues present opportunities for hackers that may allow them to more easily compromise a SAP Hostagent on a SAP server.
Whilst this may not sound too bad, it is just the Hostagent, when we realise that the Hostagent runs as the Linux root (or Windows SYSTEM user) and there are known vulnerabilities that allow remote exploitation, we can see that the Hostagent could be a window into the SAP system as the highest privileged user on the server!
It is therefore quite important to try and protect the Hostagent as much as possible.

How Can We Harden the Hostagent SSL Service?

To ensure that weak ciphers are not used, the server needs to be configured to not use them. In the context of SAP Hostagents, they are the SSL servers and they need to be configured to only use stronger ciphers.

The SAP Hostagent is really the same as the SAP Instance Agent in disguise.
Because of this, it is possible to find documented parameters that allow us to harden the SSL service of the Hostagent in the same way.

By following SAP note 510007, we can see two SAP recommended parameters and settings that can be used to harden the SSL ciphers used:

  • ssl/ciphersuites = 135:PFS:HIGH::EC_P256:EC_HIGH
  • ssl/client_ciphersuites = 150:PFS:HIGH::EC_P256:EC_HIGH

The SAP note 510007 includes an extremely good description of the SAP cryptographic library’s capabilities, the role of SSL and even some commentary on the probability of an older protocol being abused.
I feel that the note has been written by someone with a lot of experience.

The above two parameters apply a numeric calculation that selects an appropriate strength of cryptographic ciphers to be used for server and client connectivity.
With the Hostagent, we are more concerned with the server side, but the Hostagent can also do client calls, so we apply both parameters in unison.

The values assigned to the two parameters are described by the SAP note as being good, but also allow flexibility for backwards compatibility with the older SAP and non-SAP software. Again the SAP note stresses the importance of compatibility (and having stuff continue to work) versus security.

What is the Impact of the Parameters?

To be able to see the impact to the Hostagent, we first need to see what the Hostagent supports out-of-the-box.

Thanks to a great post here:
we can use a super simple shell script (on Unix/Linux) to call the OpenSSL executable, make a connection to the target server (the Hostagent) and check the list of ciphers and protocols that are advertised.
The code from the above site is here:

for v in ssl2 ssl3 tls1 tls1_1 tls1_2; do 
   for c in $(openssl ciphers 'ALL:eNULL' | tr ':' ' '); do 
      openssl s_client -connect localhost:1129 -cipher $c -$v < /dev/null > /dev/null 2>&1 && echo -e "$v:\t$c" 

You can see that I have placed “localhost” and “1129” in the code.
This is because I am running the script on a Linux host with a SAP Hostagent installed, and the SSL port is 1129 (default).

The output is something like this (depending on your version of the Hostagent):

tls1: AES256-SHA 
tls1: AES128-SHA 
tls1: RC4-SHA 
tls1: RC4-MD5 
tls1: DES-CBC3-SHA 
tls1_1: ECDHE-RSA-AES256-SHA 
tls1_1: AES256-SHA 
tls1_1: ECDHE-RSA-AES128-SHA 
tls1_1: AES128-SHA 
tls1_1: RC4-SHA 
tls1_1: RC4-MD5 
tls1_1: DES-CBC3-SHA 
tls1_2: ECDHE-RSA-AES256-GCM-SHA384 
tls1_2: ECDHE-RSA-AES256-SHA384 
tls1_2: ECDHE-RSA-AES256-SHA 
tls1_2: AES256-GCM-SHA384 
tls1_2: AES256-SHA 
tls1_2: ECDHE-RSA-AES128-GCM-SHA256 
tls1_2: ECDHE-RSA-AES128-SHA 
tls1_2: AES128-GCM-SHA256 
tls1_2: AES128-SHA 
tls1_2: RC4-SHA 
tls1_2: RC4-MD5 
tls1_2: DES-CBC3-SHA

You can see that we have some RC4 and some DES ciphers listed in the TLS 1.0, TLS 1.1 and TLS 1.2 sections.
We now use SAP note 510007 to decide that we want to use the more secure settings that remove these weaker ciphers.

In the case of SAP Host Agents, we adjust the profile file /usr/sap/hostctrl/exe/host_profile (as root), and add our two SAP recommended parameters (mentioned previously):
ssl/ciphersuites = 135:PFS:HIGH::EC_P256:EC_HIGH
ssl/client_ciphersuites = 150:PFS:HIGH::EC_P256:EC_HIGH

NOTE: You should be running the latest SAP Hostagent, this is very important for security of your system. There are known vulnerabilities in older versions that allow remote compromise.

Once set, we need to restart the agent:

/usr/sap/hostctrl/exe/saphostexec -restart

We can re-execute our check script to see that we have a more secure configuration:

tls1: AES256-SHA 
tls1: AES128-SHA 
tls1_1: ECDHE-RSA-AES256-SHA 
tls1_1: AES256-SHA 
tls1_1: ECDHE-RSA-AES128-SHA 
tls1_1: AES128-SHA 
tls1_2: ECDHE-RSA-AES256-GCM-SHA384 
tls1_2: ECDHE-RSA-AES256-SHA384 
tls1_2: ECDHE-RSA-AES256-SHA 
tls1_2: AES256-GCM-SHA384 
tls1_2: AES256-SHA 
tls1_2: ECDHE-RSA-AES128-GCM-SHA256 
tls1_2: ECDHE-RSA-AES128-SHA 
tls1_2: AES128-GCM-SHA256 
tls1_2: AES128-SHA

The more insecure ciphers are removed, but we still see those older protocols (TLS 1.0 and TLS 1.1) in the list.
We decide that we would like to further harden the setup by removing those protocols.

If we look at SAP note 2384290, we can see that an alternate set of parameter values are provided:

  • ssl/ciphersuites = 545:PFS:HIGH::EC_P256:EC_HIGH
  • ssl/client_ciphersuites = 560:PFS:HIGH::EC_P256:EC_HIGH

Let’s apply these and re-run the test for a final time.
We can see that we get a super refined list of protocols and ciphers:

tls1_2: ECDHE-RSA-AES256-GCM-SHA384 
tls1_2: ECDHE-RSA-AES256-SHA384 
tls1_2: ECDHE-RSA-AES256-SHA 
tls1_2: AES256-GCM-SHA384 
tls1_2: AES256-SHA 
tls1_2: ECDHE-RSA-AES128-GCM-SHA256 
tls1_2: ECDHE-RSA-AES128-SHA 
tls1_2: AES128-GCM-SHA256 
tls1_2: AES128-SHA

Our Hostagent SSL service is now as secure as it can be at this point in time, within reason. If we try and adjust the ciphers any further, we may end up breaking compatibility with other SAP systems in your landscape.


We’ve seen how applying two SAP standard parameters to the SAP Hostagent and restarting it, can significantly strengthen the posture of the Hostagent’s SSL service.

However, we need to be cautious of compatibility with other SAP and non-SAP software in the landscape, which may talk to the Hostagent only with older protocols.

As a final note, you may be wondering if we can remove the HTTP service from the Hostagent? At this point in time I have not found a SAP note that would indicate this is possible or recommended. However, since the HTTP protocol is known to be insecure, just don’t use it. This is in comparison with SSL which should be secure, but might not be as secure as it could be.

Critical SAP Host Agent Security Changes in PL47 – PermissionPolicy

The SAP Host Agent is a critical part of the SAP landscape infrastructure, used to control and, importantly, help automate some aspects of SAP systems.
Generally, writing custom scripts for the Host Agent has been easy.
With experience, it’s easy to see how the Host Agent could be easily abused in such a way that could allow highly privileged access to the server host, without certain security considerations being implemented.

As of the SAP Host Agent 7.21 PL47, the security of the SAP Host Agent and the way that it executes custom scripts is changing.
In this post I will describe how this could break a few things.

What Can The Host Agent Be Used For?

In my experience I have used the Host Agent for the following:

  • Detecting SAP instances on a server host.
  • Patching SAP instances on a server host.
  • Starting/Stopping SAP instances on a server host.
  • Executing scripts on a server host.

Some of the above actions have been performed direct on the server, from SAP BPA (Business Process Automation), from scripts or from tools like Postman, and a lot of the time from SAP LaMa (Landscape Management).

See a previous post for a more detailed example: How an Azure hosted SAP LaMa Controlled SAP System Starts Up

In the majority of cases I have been calling custom scripts, written to perform specific tasks on the target server host.
The scripts are generally hosted in a central location, accessible from all server hosts. This makes it simple to call whichever script.

To be able to execute a custom script, a Host Agent operation descriptor file is required to be deployed into the operations.d directory of the Host Agent home executable directory (usually /usr/sap/hostctrl/exe or C:\Program Files\SAP\hostctrl\exe).
The descriptor allows the Host Agent to understand how to execute the custom script. It contains, for example, the target platform (Windows\Linux), the name and path for the target script, which operating system user is needed to execute the script and any parameters.

On Linux, the descriptor can be specified to execute the target script as any operating system user on Linux, including the root user.
For this reason, the Host Agent and it’s installation directory location are owned by the root user. All files are only modifiable by the root user.

On Windows it is more secure by default.
The Windows security mechanisms prevent the Host Agent from executing any script as any user other than the Computer SYSTEM user (this is the user that the Host Agent executes as). NOTE: I have a workaround for this which I have developed.

Even though the Host Agent installation location and descriptor location and files are not necessarily easily modified, the weakest link in the security chain is the target script/executable and the location of the target script/executable.

What is Changing With Patch Level 47?

From June 2020, with the introduction of Host Agent 7.21 PL47, a new set of security requirements (PermissionPolicy) are introduced, which make the Host Agent more secure when executing custom scripts.

In fact, the changes were introduced before PL47, probably in PL44 or 45, as I remember seeing the PermissionPolicy check output in a previous trace file. It was obviously disabled by default in those prior patch levels.

The main changes introduced by the new PermissionPolicy are:

  • The target script and its directory must be owned by the same user as is specified in the descriptor file for the execution of the script, or it should be executable by the root user (Linux).
  • The script’s source directory must be writeable by this same user or root (Linux), or be writeable by the primary group of the user.
  • If the script is located on an NFS share, “root squash” must be disabled.

What Is Impacted By the New PermissionPolicy Change?

  • Any descriptor in the Host Agent operations.d directory, will be impacted.
  • Any target script will be checked by the new Host Agent security policy.
  • Only Linux/Unix servers will be affected due to the way that Windows security works (as mentioned before).

Because the new security policy affects Linux and affects any descriptor, this will also have a direct impact on some SAP HANA HSR operations performed from SAP LaMa, plus impact any custom operations that you have created.

By default the new security policy is enabled in the Host Agent as soon as you apply patch level 47.

How to Minimise Disruption?

A lot of customer implement the Host Agent auto-update feature, which saves significant effort when applying the frequent SAP Host Agent patches to the entire landscape.

The auto-update feature has one downside; it’s too easy to apply a patch to the whole landscape without reading the SAP notes to discover the contents of the patch or any changes in the patch. Make sure you always read the notes and make sure your auto-update architecture is designed to allow selective roll-out of the Host Agent patches to a portion of your landscape at a time (not the whole landscape in one go).

See here for a brief overview of SAP Host Agent auto-update.

The SAP note 2932953 mentions a method of adjusting the descriptor file to disable the new PermissionPolicy setting completely.
However, this needs pro-active adjustment, since some of the operations affected may only be used in a HANA HSR failover scenario (you will not know it doesn’t work until you need to use it).

Disabling the new security policy is obviously not a long term solution, since it could be enforced in the future.

Remember: Make your desired PermissionPolicy changes to your descriptor files before you apply the Host Agent patch.

SAP Instance Agent as Mini-Web Server

In this post, I will show you how a little known feature of the SAP Instance Agent (available on every “modern” SAP system) can be used to serve files via HTTP and HTTPS.

You may be thinking, “This is basic stuff, I know this already“, in which case, you may only be interested in the very last paragraph of this post 😉

Why Not Just Use a Real Web Server?

During SAP projects there always comes a point where software needs to be distributed throughout the landscape, where end-users need access to a predefined set of software, or where scripts need to be centralised and downloadable onto multiple target servers.
Essentially, you need a common file distribution point.

Some projects have large budgets and some have small budgets. This post is for those with small budgets. Those projects where using everything twice, if possible, becomes an artform.

Under What Circumstances Would I Possibly Want to Use Such a Method?

Let’s imagine a scenario where you have a set of Korn shell scripts that you would like centralised across the SAP landscape.
There is/was no budget for a common fileshare and there is no budget to have someone setting one up.
Instead you develop an offline deployment approach, where the scripts are pulled down from a central repository on a schedule.
You decide that the central repository needs to be a web server and the scripts will be downloaded by HTTP.

What Is the SAP Instance Agent?

The SAP Instance Agent is the agent that does the work, when you run sapcontrol functions.
It is a small set of executables that come as part of the SAP Kernel and generally you get one Instance Agent installation per SAP instance.

For example, if you have an ASCS instance, there will be an instance agent installed under /usr/sap/<SID>/ASCS<##>/.

You can see the Instance Agent running by querying the list of running processes with ps:

>ps-ef | grep sapstartsrv

as1adm     1969      1  0 06:31 ?        00:00:01 /usr/sap/AS1/ASCS00/exe/sapstartsrv pf=/usr/sap/AS1/SYS/profile/AS1_ASCS00_sapas1ase1 -D -u as1adm

You can see that the binary executable responsible is “sapstartsrv”.

You will also notice that the SAP Host Agent also has a “sapstartsrv”. This is because the Host Agent and the Instance Agent are siblings, sharing a similar code-set, just with different functions.

How Do You Access the SAP Instance Agent?

In the history of the SAP product range, SAP created a Java based GUI tool called SAP MMC. The SAP MMC can be used to administer SAP instances on the local server via the Instance Agent. To be able to start the SAP MMC, it was distributed from the Instance Agent by HTTP. I’m not going to go into the SAP MMC because it will be going away completely eventually.

Generally, whenever you run sapcontrol and call a function, you are accessing the SAP Instance Agent:

sapcontrol -nr 00 -function GetSystemInstanceList

07.05.2020 07:01:32
hostname, instanceNr, httpPort, httpsPort, startPriority, features, dispstatus
sapas1ase1, 0, 50013, 50014, 1, MESSAGESERVER|ENQUE, GREEN

What Is the Document Root For the Instance Agent?

In Web Server lingo, the “document root” is the highest level directory that the web server can serve files from.

For the Instance Agent, this is /usr/sap/<SID>/<INST>/exe/servicehttp.
Example: /usr/sap/AS1/ASCS00/exe/servicehttp.

If we create a simple text file in the document root, we can access it via HTTP:

echo "Hello Darryl" > /usr/sap/AS1/ASCS00/exe/servicehttp/index.html

We can use wget to access the file like so:

wget -q -O -

Hello Darryl

Can We Use HTTPS?

You can use HTTPS, the TCP port is the secure port 5##14 (50014 in our example).
Because the certificate used by the SAP Instance Agent is self-signed, you will need to trust the certificate directly.

We can use wget again as follows, but with the secure port and telling wget to not check the SSL certificate:

wget --no-check-certificate -q -O -

Hello Darryl

Are the Files Persisted Forever?

The files that you create under the document root of the SAP Instance Agent, are not persisted indefinately.
They get removed when the SAP instance is started.
Do not confuse this with when the Instance Agent is started.

An example, the Instance Agent is started up automatically by sapinit on server boot. Only when the SAP instance that is served by the Instance Agent, is started (i.e. sapcontrol -nr 00 -function Start), do the files in the document root location get cleansed, just before the instance is started.

I have previously put together a nice diagram about the interactions of various components during SAP instance startup. See the post here:

How an Azure hosted SAP LaMa Controlled SAP System Starts Up

How Can We Make Our Files Persist?

You can make your files persist, by creating a filesystem soft link from the document root and re-creating this link as part of the SAP instance start process:

mkdir /home/as1adm/docroot
echo "Hello Darryl" > /home/as1adm/docroot/index.html
ln -s /home/as1adm/docroot /usr/sap/AS1/ASCS00/exe/servicehttp/

Then you need to add an entry to the SAP instance profile to re-create the link on startup.
FIrst we need to establish the current number of “Execute” tasks in the profile:

grep Execute_ *

AS1_ASCS00_sapas1ase1:Execute_00 = immediate $(DIR_CT_RUN)/sapcpe$(FT_EXE) pf=$(_PF) $(_CPARG0)
AS1_ASCS00_sapas1ase1:Execute_01 = immediate $(DIR_CT_RUN)/sapcpe$(FT_EXE) pf=$(_PF) $(_CPARG1)
AS1_ASCS00_sapas1ase1:Execute_02 = local rm -f $(_MS)
AS1_ASCS00_sapas1ase1:Execute_03 = local ln -s -f $(DIR_EXECUTABLE)/msg_server$(FT_EXE) $(_MS)
AS1_ASCS00_sapas1ase1:Execute_04 = local rm -f $(_EN)
AS1_ASCS00_sapas1ase1:Execute_05 = local ln -s -f $(DIR_EXECUTABLE)/enserver$(FT_EXE) $(_EN)

We add a new entry as follows:

echo "Execute_05 = local ln -s /home/as1adm/docroot $(DIR_EXECUTABLE)/servicehttp/" >> AS1_ASCS00_sapas1ase1

Upon starting the ASCS instance, the profile is read and the link created.

You would then access the index.html as follows:

wget --no-check-certificate -q -O -


Even if you don’t use the above method for serving basic file content over HTTP, there is another use if you are running your SAP system in Azure.
We can use the HTTP capability of the SAP Instance Agent as a method to dynamically control which back-end VMs are accessible through an Azure ILB. This is an interesting concept, especially when introduced with a couple of other posts I have written over the past year.
I will elaborate further on this over the next few months.

Staying on-top of SAP Kernel Patches

Staying on top of patching & maintenance in a large SAP landscape can be a daunting and exhausting task.
The BASIS team need to understand every nook & cranny of software installed in the landscape, provide a strategy for patching it, and do the patching with minimal effort and business outage.
It’s harder than you can imagine!

There are numerous tools available to help automate a lot of the process; however, there’s one aspect that still needs some grey matter applying!

Analysing the SAP security notes, reading the latest Kernel distribution notes and staying on top of software component developments is, something that is not very easily automated.
As an example:  How do you determine the criticality of a bug in the Kernel in a feature that you don’t currently use, but plan to use?  Or how do you decide which future version of a Kernel is most stable for you to move to?

The BASIS team need to review the information, classify it and apply the appropriate actions.

For this reason, one of my daily reading habits, is checking the “SAP Kernel: Important News” wiki page.
Endless amounts of useful information covering the whole Kernel spectrum from 721 to Kernels not even in general use yet.
You will gain useful information from understanding what Kernels are available, what doesn’t work properly and the future direction of the Kernels in current distribution.

It’s easily read on the commute into the office.


If you haven’t already seen it, quite some time ago I wrote a brief blog post on the Flight Recorder for the NW AS Java stack.
Many years later, I’ve still very rarely seen a company use the flight recorder information.
Snapshots (also known as SAP Kernel Snapshot) in the NW AS ABAP stack are much the same thing.

When a serious condition occurs within the ABAP stack, a system message is registered in the system log (SM21) and a snapshot of the current system status is generated.
An example condition could be that a work process has died without warning, or that there was a lack of resources (background or dialog), hard shutdown of the SAP system/server or that the dispatcher queue was full.

Each of these scenarios is logged under system log message code “Q41”, in category “DP” and “Process No” “000”.
The only difference between the failures is the text following “reason: “ which is passed in by the calling Kernel function.

Q41 in system log

According to the SAP notes, the initial feature was provided in 2013 as per SAP note: 1786182 “CreateSnapshot: Collecting developer traces using sapcontrol”.
It was provided as an additional set of web service functions on the SAP instance agent (sapstartsrv) and is therefore accessible from outside of the SAP system if need be.

Originally, it appears to have been designed to be operated independently outside the SAP system, however SAP note 2640476 – “How to analyze Server Snapshot with kernel snapshot analyser” from 2019, indicates that it was integrated into the SAP Kernel in 7.40 (i.e. that the Kernel itself can instigate the snapshot).

How can we use SNAPSHOTS?

You can either access the snapshot zip files directly on the O/S level using O/S commands to extract and inspect the files, or you can use the ABAP transaction code SNAPSHOTS to see an ALV list of snapshot files.

In the O/S the files are stored in /sapmnt/<SID>/global/sapcontrol/snapshots.

Usually, the sequence of access is dictated by an extraordinary event within the ABAP stack, then you may see the System log entry which will inform you of the existence of a snapshot, but you may choose to regularly check in SNAPSHOTS anyway as part of your daily checks.


As you can see the ABAP transaction incorporates the reason for the snapshot, whereas the O/S file listing is not so easy to identify.
If you want to use the O/S level, then unzipping the file will reveal a file called “description.txt” which states the reason for the snapshot:

From the SNAPSHOTS transaction (program RS_DOWNLOAD_SNAPSHOTS) you have the option to download the snapshot file to your front-end.
Here you can unzip the file and expose the contents.

Once you have extracted the snapshot zip file, you will see a tree structure under which will sit a number of XML files:

The names of the XML files are fairly self explanatory.
ABAPGetWPTable for example, is the name of the sapcontrol web service function that is used to get the ABAP Work Process Table (same as transaction SM50).

Opening any of the XML files is going to be a lot easier with Microsoft Excel.
Except the XML is not suitable for Excel without a little bit of manipulation (this is a real pain, but once in Excel you will love it).

Edit the XML file in a text editor and delete the header lines that are the result of the web service function call, leaving just the raw XML:

Save the file and then it will happily open in Excel!

As mentioned, this is a snapshot of the work process table at the point when an issue occurred.
Very useful indeed.
You have lots of other XML files to examine.

Plus, as an added bonus, further down the directory structure of the snapshot zip file, is a complete XML snapshot of all the developer trace files for this app server:

How can we manually create SNAPSHOTS?

You can manually create and administer the snapshots (they will need clearing down) using the SAP instance agent (sapcontrol) web service commands as follows:

sapcontrol -nr <##> -function [snapshot_function]
  CreateSnapshot [<description> [<datcol_param> [<analyse_severity -1..2>
  [<analyse_maxentries> [<analyse_starttime YYYY MM DD HH:MM:SS>
  <analyse_endtime YYYY MM DD HH:MM:SS> [<maxentries>
  [<filename1> … <filenameN>]]]]]]]
  ReadSnapshot <filename> [<local filename>]
  DeleteSnapshots <filename1> [<filename2>… <filenameN>]

The ABAP transaction SNAPSHOTS only allows you to view/download and delete the snapshots. You can not trigger them using any standard transaction that I can find.