This blog contains experience gained over the years of implementing (and de-implementing) large scale IT applications/software.

Tag: Web Dispatcher

SAP ICM/Web Dispatcher CLI Param Change Error -1

This specific error/situation has zero results in a google search, so it’s worth documenting.
You’re welcome!

Scenario:

You’re trying to use the icmon or wdispmon to modify a profile parameter value, but during the process you receive an error for the specific parameter.
Example: 

Reading value for parameter icm/HTTP/redirect failed (-1)

The error is reported and the menu just loops to the same menu you were on before.
Inside the trace file (dev_webdisp in our example), we see:

*** ERROR => IcmHandleMonAdmMsg: unknown protocol/service: HTTP/, opcode ICM_COM_OP_GET_SUBHDL_PARM. [icxxmsg.c    3440]

Notice that the “protocol” above is “HTTP”. It is trying to use HTTP to talk to the webdisp/icm.
If no HTTP port is present then it fails.

Solution:

Add a HTTP port using the webgui.
In my solution, adding port 80## to the instance, then I was able to manually make the change using wdispmon. No restart of the web disp was needed for either of those things. 🙂

You may also be interested in:

Azure Front Door in a SAP Context

In April 2019, Microsoft announced the general availability of the Azure Front Door service.
The highlight of this service is layer 7 (HTTP/S) load balancing.
In this post I want to briefly explore how Azure Front Door could sit in an example SAP landscape.

But We Have Azure Application Gateway…

Yes, while the Azure Front Door service does provide similar capabilities with regards to load balancing an HTTP/s based back-end service, the similarities end when we start to consider multi-regional distribution of services. That is, multiple Azure regions actively servicing global clients.

Azure Application Gateway

The Azure Application Gateway service is the go-to service for HTTP/S load balancing for your Azure hosted HTTP/S IaaS or Container based services that are contained within a region.

Event for some, limited, SAP uses, the Azure Application Gateway may be sufficient, but you really need an experienced SAP Solution Architect to help you plan your SAP landscape architecture at this point. The consequences of doing it wrong, could cause you to completely re-implement a new architecture pattern in your landscape and, of course, additional cost.

… and SAP Web Dispatcher

I have discussed the features of the SAP Web Dispatcher before.
The need for a SAP Web Dispatcher in a SAP landscape is clear and even more appropriate in a cloud deployment of SAP.
Just like Azure Application Gateway, the SAP Web Dispatcher’s context should be limited to a single region. This is especially true because it is IaaS, which means the VMs on which the Web Dispatcher is deployed, are themselves bound to a specific region.

However, what is not clear is how disparate Web Dispatcher systems (i.e. different SAPGLOBALHOST values) can be used in different regions to correctly load balance. This is not the same as a single system with different instances in different regions!

How It All Hangs Together

If we go back to the purpose of this post, I wanted to show how Azure Front Door could be used within the context of a SAP system deployment in Azure.

To help convey the idea, I’ve put together a simple diagram:

In the above diagram, you can see that the Azure Front Door service is used to balance inbound requests from a customer booking system, across multiple Azure regions, directly from the internet. This means that Azure Front Door is most definitely suited as a global customer facing load balancer.
An example scenario is a 2 (or more) region architecture with primary region and disaster recovery region. If the primary region for our customer booking system is unavailable, a DR could be invoked and customers could be routed to the DR region, allowing customer bookings to be taken.

In the diagram, traffic routed from Azure Front Door, is then (for the sake of example) routed through Azure Application Gateway. This is just for example, but in reality it’s not really needed. It could be that you have a real mixture of SAP and non-SAP in some converged sub-domain, and it may be easier to load balance this mix of URLs at this level.
The main point at this point is, you are committed to returning data from a single region.

In our example diagram, the Azure Application Gateway then routes traffic to the SAP Web Dispatcher, which load balances the traffic over the back-end SAP ECC system available application servers using the ABAP stack message server (a feature that is not easily replicated in any other load balancer).

Where Does Azure Traffic Manager Sit?

The Azure Traffic Manager service is a DNS based routing and distribution service. If your company is a multinational conglomerate with a latency sensitive web based customer service, then the Azure Traffic Manager can be used to route customers to their nearest region, where you have your web service hosted and where they can potentially get the speedist and most appropriate content.
If you have only 2 or 3 regions, do not have latency issues and have no need to provide region specific content, then Azure Front Door is probably what you need.

Summary:

I’ve tried to show how the Azure Front Door service can provide your internet sourced, customer entry point into your multi-region web service.
The diagram I’ve provided hopefully shows how Azure Front Door can be distinguished from other similar technologies in a SAP landscape including how Azure Application Gateway could also be in the mix (although rare).
Finally I discuss how Azure Traffic Manager may not always be appropriate for load distribution.

Useful Links

You may also be interested in:

SAP Web Dispatcher Reverse Proxy Features

If you read through any SAP documentation, you may be forgiven for thinking that the SAP Web Dispatcher is just a reverse HTTP proxy.
It can be located in front of a SAP WebAS and balance the load.
Therefore, it is a simple reverse proxy, right?

In this post, I am going to highlight some of the core features of the SAP Web Dispatcher, so that you may understand its strengths in comparison with other solutions such as Azure Application Gateway and even Azure Front-Door.

Heavily Engineered

There’s a common misconception that SAP is just another piece of software using an array of different components lumped together with some bits of Open Source. In some small cases this may be true of acquired software.
However, the core SAP software offerings are actually far more coherent and intricately linked than you may first imagine.

Ask any Oracle EBS administrator about their software stack and you will be impressed at how well the SAP software stack has been engineered.
This is especially true for the lower SAP Kernel level software components. The older parts of the software stack, are reused so often because of their robustness.

3 Routing Principals

The main thing to remember is that the SAP Web Dispatcher can route requests according to 3 main principles:

  1. Capability
    Is the desired target URL path served by the configured target back-end system(s).

  2. Availability
    Is the desired target URL path served by a configured back-end system that is available (i.e. not in maintenance mode).

  3. Capacity
    Are there more than one target back-end servers capable of handling the request and which one has more capacity.

Load Balancing Act

The SAP Web Dispatcher takes the HTTP/S request from the end-user and as part of the routing determination it analyses the target back-end system load.
It’s actually continually aware of the back-end systems.

There’s a great picture here, which highlights the load balancing methods used for the different types of SAP back-end: https://help.sap.com/viewer/683d6a1797a34730a6e005d1e8de6f22/7.40.18/en-US/4899c3d999273987e10000000a421937.html

What is not mentioned on the help.sap.com page linked above, is target back-end systems configured as “EXTSRV” (non-SAP routing) and also the “flat-file” routing method (very rarely used – at least, I’ve not used it).

The “EXTSRV” back-end systems will use basic round-robin to distribute the request between a comma separated list of target servers. Sticky-ness is achieved through HTTP headers, allowing the Web Dispatcher to determine which back-end system it routed your previous request to.

Even though “EXTSRV” is really designed for non-SAP back-ends, I have used “EXTSRV” for SAP systems, especially when using the SAP Web Dispatcher to avoid issues for system-to-system communications and wanting to avoid CORS issues (see CORS in a SAP Netweaver Landscape).

The “flat-file” method simply uses a static text file as a kind of false load response from a Message Server. The flat-file can be generated by anything and the Web Dispatcher configuration is then defined to route to whatever is in the flat-file.

Back-End

Apart from “EXTSRV” and “flat-file”, all other routing mechanisms use SAP proprietary methods to determine the back-end system load.
As you can see in the SAP Help page link referenced above, the SAP Web Dispatcher knows about the back-end because in the SAP Web Dispatcher configuration, we tell it what it is going to be routing to.

As an example, ABAP back-end systems are added to the Web Dispatcher profile file with the ABAP Message Server described in the configuration.
The Web Dispatcher connects to the target system’s Message Server and says “hello”.
Once connected, the SAP Web Dispatcher retrieves the list of URLs that are provided by the ABAP back-end system, the servers that are served by the Message Server and the relative load of those servers.
All of this information is used during the routing determination.

Protocols

The Web Dispatcher can handle HTTP 1.0, 1.1 and 2.0 (HTTP/2) protocols delivered over TLS (SSL).

Since Kernel 7.49, HTTP/2 has been supported in the Web Dispatcher and also in the ABAP Netweaver stack. This is significant for the latest HTTP based SAP UX known as SAP Fiori. The use of HTTP/2 allows request multiplexing over a single continuous TCP connection, reducing latency and increasing throughput.

NOTE: There are some great SAP blogs out on there on how and why to enable HTTP/2 for Fiori!

For many years now, the SAP Web Dispatcher has supported the Web Socket protocol.
The Web Socket protocol allows developers to utilise push-notifications and provide a more real-time interactive experience for HTML 5.0 content.
Bringing a closer level of integration with the consuming Web Browser.

Security

Some of the more complex uses of the SAP Web Dispatcher involve specific security scenarios.

One such scenario that comes to mind, is Principal Propagation, which can use the Web Dispatcher to front a set of common back-end systems.
The whole premise of Principal Propagation, is that the iDP (identity provider) is “impersonating” the authenticated user, by issuing a generated certificate of authenticity to the target system, on behalf of the user.
With a reverse proxy between the Web Browser and the target HTTP service, things can become complex because the generated X.509 client certificate can become consumed by the proxy server, instead of being forwarded to the target HTTP server.
To prevent the certificate from being interpreted in the wrong way, the SAP Web Dispatcher can be configured to shift the client certificate out to a predefined HTTP header., allowing a kind of X.509 client certificate forwarding.
(More information can be found here: Principal Propagation with SAP Cloud Platform).

Update Aug-2020: As pointed out by a reader, the SAP Web Dispatcher is also capable of reverse invocation. This is an added security feature which allows the target SAP system to open the connection to the SAP Web Dispatcher (instead of the other way around). The SAP Web Dispatcher then uses this open connection channel to send load balanced requests back to the target SAP system. The Reverse Invoke feature is obviously meant for scenarios where the Web Dispatcher exists in a separate network segment (DMZ) to the target SAP system, meaning you only need to open the firewall in the outbound (from the target SAP system) direction.
(Details here: https://help.sap.com/doc/7b196aab728810148a4b1a83b0e91070/1511%20000/en-US/frameset.htm)

Manageability

There’s nothing I like about trying to trace a HTTP call through a proxy server.
The SAP Web Dispatcher comes with it’s own secure administration page from where an administrator can enable advanced tracing capabilities.

The SAP Web Dispatcher makes it much easier to trace requests and responses, with the ability to show the complete unencrypted trace of SSL encrypted sessions (not using pass-through encryption).

The trace is able to show the exact ABAP work process number that processed the request in the target back-end system.

An administrator is able to move individual back-end systems into “maintenance mode” and provide a custom HTTP 503 (service unavailable) message, without affecting the other back-end systems serviced by the same Web Dispatcher.

The SAP Web Dispatcher comes with a vast array of configuration parameters to hone the characteristics of the service you are trying to deliver.
As an example, parameter “wdisp/handle_webdisp_ap_header” can be set to allow the Web Dispatcher to add additional HTTP headers to the request, thereby informing the target back-end system of the Web Dispatcher forward-facing TCP ports. This feature allows the target back-end systems to correctly rewrite HTML links and referral URLs, with the ports on which the SAP Web Dispatcher is listening for requests.
This is just one example of where the back-end SAP system is actually aware that it is being called via a SAP Web Dispatcher.

The Future

With the seemingly constant evolution of cloud based services, what do I imagine the future is for the SAP Web Dispatcher?
In my opinion it is here for another few years yet. The feature list is too specific to SAP landscapes for any real profit to be made by a competitive product.
However, what we may see in this hyper-competitive race for cloud adoption, is the use of a SaaS based version of SAP Web Dispatcher, provided for by the major cloud providers.
Right now, a SAP Web Dispatcher consumes far too much cost/resources/effort than it needs to. Therefore, a simple button click and subsequent configuration in something like the Azure Portal, would be a great saving and more importantly, a great incentive to potential cloud customers with SAP landscapes.

Summary

In this short article, we have discussed how the robust engineering of the SAP Web Dispatcher makes it the ideal front-end reverse proxy for the back-end systems of a SAP landscape.

In fact, in some situations it is the only possibility due to the way the Web Dispatcher is acutely SAP back-end aware, with many features built for native SAP compatibility.

Conversely we’ve seen how, in some situations, the back-end system is actually aware of the presence of the SAP Web Dispatcher and can rewrite HTML links and referral URLs accordingly.

We know the latest HTTP/2 protocol is supported and that this is in line with SAP’s goal of having Fiori as the future SAP presentation layer.

We discussed the extensive tracing capabilities, helping SAP administrators to diagnose complex HTTP connectivity, and authentication issues.

We can conclude that, SAP Web Dispatcher is not just a simple reverse proxy and its use within your SAP landscape is more than likely going to be beneficial in some way or another.
The SAP Web Dispatcher will be with us for a while longer.

References:

Ultimate Active-Active SAP Web Dispatcher Architecture in Azure?

I have never been fully satisfied with the reference architecture on the Microsoft site for running active-active SAP Web Dispatchers in an Azure IaaS platform.

Don’t get me wrong, from a high-level Azure perspective they are representative of what you will be desiring. However, they just lack enough detail to make you really think about the solution more than I feel you should need to.

To re-cap, in an Active-Active SAP Web Dispatcher in Azure, you rely on the inherent capabilities of the Azure Internal Load Balancer (ILB) to provide availability routing to the active-working VMs that are running the Web Dispatcher instances.

To help you (and me) understand what needs to be configured, I’ve put together what I feel is a pretty good low-level architecture diagram.

It’s almost a version two to SAP on Azure Web Dispatcher High Availability.

Show Us the Picture or It Never Happened!

Below is the diagram that I have created.
There is quite a lot of detail in it, and also quite a lot of detail that is not included (required O/S params, instance params and config for the network layer etc). It is really not as simple as you first imagine to include the required detail in one picture.

It Happened, Now Explain Please

If we look at the diagram above, we can easily see that WD1 is the SAP System name, with 2 instances of WD1, both with an instance number of 98 but installed against 2 virtual hostnames of sapwd1 and sapwd2.

Could we have installed on the server hostname directly? Yes, we could have. But that is not inline with a SAP Adaptive Computing Design principal, which is separation of the SAP instance from the host.

Notice that we have a Highly Available NFS share that hosts our SAP system instance profile files and a single shared PSE (SAPSSLS.pse).
We don’t show that this could be from a HA fileshare or NetApp or some other technology, but please use your imagination here. For production the Azure Files service is not currently supported.

Our ILB is configured to accept HTTP and HTTPS connections on their native ports (80 and 443) and it routes these through to the 8098 and 44398 ports that the Web Dispatchers are configured to listen on. You can configure whatever ports you want, but ultimately, having separately addressable back-end ports allows you to re-use the SSL port for Web Dispatcher administration and tie-down the access to a DMZ hosted Jump Box (definitely not on the diagram).

The ILB is probing both back-end VM members on tcp/8098 to see if the Web Dispatcher is responding. It’s a basic TCP SYN check (can I open a connection – yes, OK). For a better check, you can use a HTTP health probe on tcp/8098, which would allow you to set the Web Dispatcher to “maintenance” mode, causing a HTTP “service unavailable” response to be returned to the ILB probe, which would remove that particular Web Dispatcher from the ILB routing. If you also followed the other suggestion of accessing the admin page from the 44398 port via the virtual hostname, then you will see that an administrator would still have admin control for maintenance purposes. Nice.

We have a SAN enabled SSL certificate inside our shared PSE, with 3 Common Names associated with that certificate, one for the ILB “host” name (sapwd), and 1 for each of the virtual hostnames against which we have installed the Web Dispatcher instances (sapwd1 and sapwd2).

Our “icm/host_name_full” parameter sets both Web Dispatchers to think that they are accessed through sapwd.corp.net. However, we have to be careful that we do not use EXTBIND in this particular case, because we do not have the IP address of the ILB bound onto the servers (although if you read my post on how to add a secondary IP address on the Loopback device I can show you how it’s possible to do this and why you may want to).

How Do We Cater for DR?

Because we do not have a high disk I/O throughput on a Web Dispatcher VM, it is perfect to be protected by Azure Site Recovery (ASR).

This means the VM is replicated across to the Azure DR region (the region of your choice).

Like this:

But wait, we’re only replicating 1 VM! Yes, we don’t need to pay for both, since a cost-optimised approach would be to just re-install the second Web Dispatcher after a DR failover.

We have a dependency on some sort of NFS share replication to exist in the DR region, but it doesn’t necessarily need to be fancy in the case of the SAP Web Dispatcher, because very little will be changing on the /sapmnt/<SID> area.

NOTE: The replicated VM is not accessible until a failover is instigated.

What Happens In a Failover to DR

In a DR scenario, the decision to failover to the DR region is manual.
The decision maker can choose to failover as soon as the primary region is unavailable, or they can choose to wait to see if the primary region is quickly recovered by Microsoft.

I’ve put together a diagram of how this could affect our simple HA Web Dispatcher setup:

The decision to failover should not be taken lightly, because it will take a lot of effort to failback (for databases).

Generally the recommendation from Microsoft is to use an Azure Automation Runbook to execute a pre-defined script of tasks.

In our case, the runbook will create the ILB above the Web Dispatcher VM and add the replicated VM to the ILB.
Our runbook will also then add secondary IP addresses to the VM and finally adjust DNS for all our hostnames and virtual host names, assigning the new IP addresses to the DNS records.

Once our Web Dispatcher is online and working, we could choose to then build out a further VM and add it into the ILB back-end pool, depending on how long we think we will be living in the DR region.

Summary

Did we successfully include more detail in the architecture diagram? Yes we sure did!
Was it all the detail? No. There’s a lot that I have not included still.
Will I be enhancing this diagram? Probably; I hate leaving holes.

I’ve shown above how an active-active SAP Web Dispatcher architecture can work in Azure and how that could be setup for a DR protection.

We also briefly touched on some good points about separation of administration traffic, using a HTTP health probe for an ILB aware Web Dispatcher maintenance capability, and how the SSL setup uses a SAN certificate.

Would this diagram be more complicated by having an active-active HA Web Dispatcher alongside an ASCS or SCS? Yes, it gets more complicated, but there are some great features in the ILB that allow simplification of the rules which allow you to use the ILB for more than one purpose, saving cost.

Update Jun-2020: This duplicate Web Dispatcher architecture is known in SAP as “Parallel Web Dispatcher” and a basic description is visible here: https://help.sap.com/viewer/683d6a1797a34730a6e005d1e8de6f22/1709%20002/en-US/489a9a6b48c673e8e10000000a42189b.html

Update Mar-2021: Some of you have asked about how the “Maintenance Mode” activation works with the ILB. This is siply that the WDisp returns a HTTP 503 when Wdisp maintenance mode is enabled.
By default the ILB health probe will be “http://<the-vm>:<your-port>/”, but if you don’t have a back-end service allocated to “/” then you will get a HTTP 404 constantly. You need to adjust the URL to an actual working URL location based on the config of your back-end systems.
If you don’t want the health probe to make a call to an actual back-end system (during the health probe ping) then use parameter “icm/HTTP/file_access_<xx>” to define a custom local directory and place a blank file called “health.htm”. Then just adjust the health problem URL with the path to the “health.htm” and the health probe pings will never call a back-end system URL. It also means that you can touch or remove the health.htm to permit the ILB to use or not use that specific WDisp.

HowTo: Disable HANA Web Dispatcher

Scenario: The SAP HANA Web Dispatcher seems to be automatically running in HANA 1.0 SPS70.  I am supposing that this is mainly for the XS-Engine.

If you have already removed the XS-Engine (see my post here), then you can also disable the Web Dispatcher as follows (this will save around 300MB of memory).

From HANA Studio, change the daemon.ini configuration parameter “sapwebdisp -> instances” to “0” for your host(s):

HANA sapwebdisp instances

Restart the HANA instance.
The Web Dispatcher process will be no longer present:

HANA processes no sapwebdisp

No more Web Dispatcher.