This blog contains experience gained over the years of implementing (and de-implementing) large scale IT applications/software.

SAP HDBNSUTIL Failed

During the SAP HANA install, you get an error in the setup.glf log file for hdbnsutil failing with return code 6 (signal 6).

The error could be caused by the call of “hdbnsutil -initTopology” which creates the system and the initial license.   An out of memory error could occur if this is a VM with less than ~5GB of RAM allocated (hdbnsutil tries to allocate 4GB of memory).

You can check if this is the cause of the error, as hdbnsutil will produce a trace dump in the directory “/usr/sap/<SID>/HDB<##>/<host>/trace/”.
Check the trace files “nameserver_<host>.*.*.trc” for the error “OUT OF MEMORY”.
If this is the case, increase the memory to the VM and restart.

SAP – Oracle on VMware

Whilst researching the whole Oracle on VMware thing recently, I found a great number of articles on the subject.  Here’s a collection of the links I found most helpful whilst getting to know the principles of virtualisation all the way through to publicly available information on the performance of Oracle on the VMware vSphere platform.

Hardware Abstraction Layer definition (https://en.wikipedia.org/wiki/Hardware_abstraction_layer).

Hypervisor:
Wikipedia definition (https://en.wikipedia.org/wiki/Hypervisor),
VMware’s definition (https://blogs.vmware.com/vmtn/2007/03/hypervisor_that.html),
VMware’s definition (https://www.vmware.com/technical-resources/advantages/robust-foundation.html).

Intel VT-x Instruction Sets (https://www.intel.com/technology/itj/2006/v10i3/1-hardware/5-architecture.htm).

VMware ESX and ESXi:
What does ESX stand for (https://vmfaq.com/entry/32/),
Comparing ESX with ESXi (https://www.vmware.com/products/vsphere/esxi-and-esx/compare.html),
ESXi v5.0 released (https://www.vmware.com/company/news/releases/vmw-vsphere-ga-082511.html),
ESXi architecture white paper (https://www.vmware.com/files/pdf/ESXi_architecture.pdf).

VMware vSphere:
Product offerings (https://www.vmware.com/products/vsphere/mid-size-and-enterprise-business/buy.html),
Support (https://www.vmware.com/support/services/options.html),
Kits (https://www.vmware.com/products/datacenter-virtualization/vsphere/small-business/essentials-kits.html),
What’s new in v5 (https://www.vmware.com/files/pdf/techpaper/Whats-New-VMware-vCenter-Server-50-Technical-Whitepaper.pdf),
Supported guest OS (https://www.vmware.com/files/pdf/GuestOS_guide.pdf),
VMware Oracle support policy (https://www.vmware.com/support/policies/oracle-support.html),
VMware HCL (https://www.vmware.com/resources/compatibility),
vSphere Licensing and cost (https://www.vmware.com/files/pdf/vsphere_pricing.pdf),
Forrester Report – Total Economic Impact Of VMware vSphere Virtualizing Mission-Critical Oracle Databases (https://www.vmware.com/files/pdf/solutions/total-economic-impact-of-vmware-vsphere-oracle-database.pdf),
Oracle on VMware vSpehere 4 Essential Deployment Tips (https://www.vmware.com/files/pdf/Oracle_Databases_on_vSphere_Deployment_Tips.pdf),

RedHat Certifies Linux on VMware (https://hardware.redhat.com/show.cgi?id=674998).

Microsoft will support Windows on VMware (https://support.microsoft.com/?kbid=897615), (https://support.microsoft.com/kb/944987).

Oracle:
Support Oracle RDBMS, Linux on VMware (https://blogs.oracle.com/UPGRADE/entry/is_oracle_certified_to_run_on),
Oracle Linux same as RHEL (https://www.oracle.com/us/technologies/linux/025987.htm),
Oracle licensing (https://www.orafaq.com/wiki/Oracle_licensing), (https://www.oracle.com/us/corporate/pricing/specialty-topics/index.html),
Oracle License & Service Agreement (https://www.oracle.com/us/corporate/contracts/license-service-agreement/index.html),
Oracle Processor Core Factor Table (https://www.oracle.com/us/corporate/contracts/processor-core-factor-table-070634.pdf),
Oracle VM Tolly Group report (https://www.oracle.com/us/026997.pdf).
Running Oracle Production Databases on VMware (https://www.it-implementor.co.uk/2012/02/running-oracle-production-database-on.html).

Tolly Group (https://www.tolly.com).

SAP:
Running SAP on VMWare (https://www.vmware.com/solutions/partners/alliances/sap.html),
TCO and ROI of running SAP on VMWare (https://www.vmware.com/files/pdf/partners/sap/SAP_TCOROI_Customers_Final.pdf),
SAP Note: 1122387 – Linux: SAP Support in virtualized environments (https://service.sap.com/sap/support/notes/1122387),
SAP Note: 1122388 – Linux: VMware vSphere Configuration guidelines (https://service.sap.com/sap/support/notes/1122388),
SAP Note: 1492000 – General Support Statement for Virtual Environments (https://service.sap.com/sap/support/notes/1492000),
SAP Insider – 3 common misconceptions (https://www.vmware.com/files/pdf/partners/sap/sap-insider-virtualization-cloud.pdf)

Running Oracle Production Database on VMware

NOTE: For research links, see my later post here.

Are you considering running Oracle production databases on VMware?
Obviously you’ve considered the Oracle support policy on this.
How likely is it that you will get asked to “make it physical”?  or How will Oracle support deal with your call if you tell them you’re on VMware?

Well, there are some harsh responses from Oracle towards VMware customers if you have access to Metalink (sorry, My Oracle Support).
First you should read this doc: 1071005.1 (HOTSPOT ERROR DURING 32-BIT 11GR2 CLIENT INSTALL ON 64-BIT (X86_64) SUSE (VMWARE)).
Then read this document: 1075717.1 (Installing 32-bit RDBMS Client software on x86_64 Linux.)
I don’t think the first problem’s resolution is justified.  Do you?
I have found Oracle notes that state the following:

(1) Make sure you are logged into the Server Console directly, and that you are NOT trying to install the patch over a remote connection (such as Terminal Services, Remote Desktop, Timbuk2, PCAnywhere, VNC, VMWare, etc.) “

As for installing over a remote connection, we do not support this, because Oracle cannot control the way the permissions are setup, over the remote connection.

So VMware console is a remote connection…

Then there is this document:
https://www.oracle.com/us/solutions/sap/wp-o10g-rac-config-win-303805.pdf

This doc is an Oracle whitepaper which details SAP Netweaver / Oracle Databse 10gR2 RAC on Windows 2003.
In the doc it shows an example hosts file which is clearly from a VMware hosted server.  Double standards I feel!

This is just not a nice place to be if you’re trying to convince a company that VMware is a solid, supported tool to run production databases, and that Oracle even support RAC on it now.  Maybe some of the notes are old, before the acceptance by Oracle that VMware is becoming big in many companies.  Or maybe Oracle’s RAC support is an illusion of good will, whilst they quietly (or not so) improve the Oracle VM product.
The best way to tell, would be the acceptance of Oracle that VMware’s vCPU allocation is acceptable as a form of hard partitioning, so that you can bring the Oracle DB license cost down by runnining on VMware, in the same way you can on Oracle VM.

Virtualisation Conundrum

Whilst researching ideas for using my VMware ESXi test rig, I came across this blog site: https://weinshenker.net/blog/2011/07/26/oracle-redhat-vmware/
I’ve added it to my Fav’s as a *must read* once in a while.
It’s packed with some very interesting articles concerning Linux, Virtualisation and more importantly, Oracle.

It may have changed my mind about using Oracle Enterprise Linux.  I’m considering shifting up the Linux tree to RedHat, or experimenting with Fedora.

See the Wikipedia tree diagram https://upload.wikimedia.org/wikipedia/commons/8/8c/Gldt.svg on the right of the Wikipedia Linux Distro page.

Clone a VMware ESXi Virtual Machine from the Command Line

My version of VMware vSphere is obviously not the full “knock your socks off” version and prevents me from right clicking and cloning an existing VM.
Not to be downhearted, there is another way…

Pre-requisites:
– VMware vSpehere client.
– Access to the VMware ESXi server console & keyboard or an SSH client (like PUTTY for Windows).
– The VMware ESXi root user’s password.

Process:
– Create the new VM using vSphere client.
– Give it a very small hard disk.
– Remove the hard disk from the VM once created.

– Enable local tech support mode from VMware *console* screen, press ALT+F1 and log in as “unsupported” with the root password.
or
– Enable remote tech support mode from VMware *console* screen, then use an SSH client to log into the server as root.

– Once you have a command line, locate the source virtual machine’s datastore directory under /vmfs/volumes/<datastorename>:
# cd /vmfs/volumes/ds1/source_vm

– Change to the new target VM datastore directory:
# cd /vmfs/volumes/ds1/target_vm

– Now use vmkfstools to clone the disk (note the use of thin-provisioning, you don’t have to):
# vmkfstools -i "/vmfs/volumes/ds1/source_vm/source_file.vmdk" "/vmfs/volumes/ds1/target_vm/target_file.vmdk" -d thin

– Back in the vSphere client, add the new disk to the target virtual machine and boot it.

– Bob is your uncle.