This blog contains experience gained over the years of implementing (and de-implementing) large scale IT applications/software.

PowerShell Encrypt / Decrypt OpenSSL AES256 CBC

A few months back I had a Korn shell script which used OpenSSL to encrypt some text using AES 256 CBC.
I managed, through the power of stackoverflow.com and various other blogs, to write a Java routine to perform the exact same encrypt/decrypt.
This allowed me to encrypt in Korn on Linux and decrypt in Java which was running inside a SAP Netweaver application server, or the other way around (encrypt in Java and decrypt in Korn using OpenSSL).

About 2 months after that, I needed the same set of routines to be written in PowerShell, allowing the same encrypted text to be encrypted on Linux with OpenSSL and decrypted on Windows in PowerShell (no need for OpenSSL).

I forked the PowerShell code which did the initial encryption and wrote the decryption routine which I’ve published as a Github gist here:

https://gist.github.com/Darryl-G/d1039c2407262cb6d735c3e7a730ee86

Encrypting HANA Data Volumes

Out of the box, the HANA database is not encrypted.
For those businesses that demand encryption, from within HANA Studio you can activate the encryption of the HANA data volumes.
The HANA documentation supplied with SPS7, suggests that the recommended approach is to encrypt the data volumes as part of the HANA installation process.  This is due to the “Copy-on-write” method used to persist data on disk.  An already used database may still have unencrypted data pages in it, even after encryption is enabled.

You should note that the words “data volumes” means the location of the database files for the indexserver and the statistics server, which is usually something like “/hana/data/<SID>/mnt0000<n>/hdb0000<n>/*.dat”.

Tip: You can check which values for “<n>” in the above, will be used, by checking the “Landscape Overview” tab and the “Volumes” tab within that.

Prerequisites:
– SAP recommend changing the SSFS encryption key after installation of the HANA system.  This key is used to store the data volume encryption key and others.
– Disk space of the data volume areas (we discuss this below anyway).
– Take a cold backup of the HANA DB area and config files.
– Access to the HANA Studio (you can do this with SQL access and hdbsql, but I don’t show this).

Let’s begin:

From within HANA Studio, open the “Security” tab and navigate to the “Data Volume Encryption” tab.
You will notice that encryption is not already enabled.
Tick the “Activate encryption of data volumes” tick box, then click the “Execute” button:

HANA volume encryption activation

HANA data volume encryption execute

The encryption process will start immediately (“Encryption Pending” then “Encryption Running”):

HANA volume encryption pending

The status is updated when each server process finishes:

HANA volume encryption running

Once the encryption process is completed successfully, the tick box is enabled again for you to be able to de-encrypt the volumes should you wish to reverse the process:

HANA volume encryption encrypted

My small (empty) HANA DB @ sps7, took approximately 10 minutes to encrypt.
The data volume sizes increased as part of the encryption process.
The indexserver data volume went from 324MB used to 341MB used:

HANA volume encryption indexserver size

HANA volume encryption indexserver size

Of more importance is the dramatic increase in the allocated (Total Size).
It’s gone from 513MB to 896MB!  You will need to be aware of the disk space you may need before enabling encryption.

The statistics server is smaller and went from 80MB used to 101MB used:

Snap668 2014-06-26, 11_26_17

image

Again, notice that we’ve got an increased allocation from 320MB to 400MB.

Validate the encryption status using an SQL console:

SELECT * FROM M_PERSISTENCE_ENCRYPTION_STATUS;

image

What are the implications for encryption?

Well, the data volumes have been encrypted, but we’ve not seen anything about encrypting the logs or the backups.
Also, to preserve the performance of the HANA system in memory, the data is decrypted when read from the disk into memory.

Is encrypting the HANA DB transaction logs feasible?  Probably not.  There are various whitepapers detailing the issues of encryption of data stored on SSDs.  Since the majority of high performance appliance resellers use SSDs for the HANA DB transaction logs, the use of software layer encryption on SSDs is not worth the effort and would probably reduce the performance of the database.  Instead, using the SSD hardware layer encryption may prove useable, but this is only worth while if you think that the SSD could be physically stolen.  SSD hardware encryption doesn’t prevent an intruder at the O/S level from seeing the data.

Is encrypting the HANA DB backups (data and logs) feasible?  Yes this is definitely something that should be employed.  There are 3rd party backup tools that will take a copy of the HANA backup files, then encrypt and store elsewhere.  But these tools may not support secure wipe, so the disk location of the backup files would potentially still contain the data.  Using backup tools certified for SAP with the BackInt interface would be better, since the data doesn’t touch the disks, it’s piped straight to the backup device via the “network”.

There is possibly some slight performance impact from encrypting the data volumes.  The data files are written to at least every 5 minutes.  They are also read from during HANA start up.  These I/O operations will be the same speed as they were before, except there will be some additional software routines involved to perform the encryption, which will use slightly more memory and mean a slight delay between the data being read/written to/from the disk.

Be aware of bugs in the backup/restore process.
I’ve seen mention of bugs in other software products where an encrypted backup is void due to a software bug.  Ensure that you test your solution thoroughly.

SAP recommend that you change the Page Encryption Key (used to encrypt the data volumes) regularly as per your organisations standards.  This may yet again increase the allocated size.

Summary:
– Data volume encryption is easy and fast in HANA.  There’s not really any reason to not implement it.  Beware of implementing in an already populated database and ensure you change the keys regularly.
– Backups and transaction log volumes are not encrypted and for the logs there’s a good reason why you may not want to.
– Performance in certain scenarios could be affected slightly.
– You should attempt to implement a supported Backint capable backup product with encryption, since the backups with this method don’t touch the unencrypted disks.
– Encryption can be performed and validated at the SQL command level.
Be aware that data volume encryption in HANA will require more disk space to be allocated to the data volumes, increasing the footprint of the HANA database by as much as 50%.

SAP HANA – SSL Security Essential

The HeartBleed hack exposed the consequences of security holes in software designed to provide encryption of network traffic.
However, this doesn’t mean that all encryption software has holes and it’s certainly better to have some form of encryption than none at all.

I’ve watched numerous online demos, official training videos and worked on real life HANA instances.  All of these systems so far, have not enabled SSL (now called TLS)  between the HANA Studio and the SAP Host Agent or the HANA Studio to the HANA database.
This means that specific communication between the HANA Studio, the SAP Host Agent and the HANA database indexserver, is not encrypted.

The HTTP protocol has been around for a long time now (thanks Tim).
It is inherently insecure when using HTTP BASIC authentication, since the username and password which is passed over HTTP to a server that has requested authentication, is sent in the clear (unencrypted) but encoded in BASE64.
The BASIC authentication is used to authenticate the HANA Studio with the SAP Host Agent.

What does this mean with regards to SAP HANA and the SAP HANA Studio?
Well, it means that any user with a network packet sniffer (such as Wireshark) could intercept one vital password, that of the <sid>adm SUSE Linux user.

In a SAP HANA system, the software is installed and owned by the <sid>adm Linux user.  Usually <sid> is a unique identifier for each HANA system in a SAP landscape.  As an example, H10 or HAN or any other 3 alphanumeric combination (within certain SAP restrictions) can be used.
When the HANA Studio is used to control the HANA database instance (start up and shutdown), the HANA Studio user is prompted to enter the username and password for the <sid>adm user.
This username and password is then sent via HTTP to the SAP Host Agent installed on the HANA server.  The SAP Host Agent uses the username and password to start or stop the HANA database instance.
If the password for the <sid>adm user is obtained, it is possible for a malicious user to establish an SSH connection directly to the SUSE Linux server where the HANA instance is installed, then control the instance, or access the database directly using a command line interface for executing SQL statements.

Here’s a 6-step example which took me 10 minutes to setup, trace, collect the data and then login to the Linux server as an authorised user.

Step 1, Install and open Wireshark (on your PC) and start tracing for TCP connections to the HANA server on the Host Agent TCP port 5<xx>13.
Step 2, Launch HANA Studio (on your PC) and in the navigator right click and choose “Log On”:

HANA  Logon without SSL

Step 3, If you haven’t elected to save the username and password during previous use of the HANA Studio, you will be prompted.  Otherwise, the system will auto-logon to the Host Agent.
Step 4, Analyse the Wireshark capture.  You’re looking for the text “Authorization: Basic” in the TCP packets:

HANA Logon Wireshark trace

The actual string will look something like: 
Authorization: Basic aDEwYWRtOmhhbmFoYW5h
I’ve copied an example HTTP POST out to a text editor for easy viewing:

HANA SAPControl HTTP POST

POST /SAPControl HTTP/1.1
Accept: text/xml, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Authorization: Basic aDEwYWRtOmhhbmFoYW5h
Content-Type: text/xml; charset=utf-8
Cache-Control: no-cache
Pragma: no-cache
User-Agent: Java/1.7.0_45
Host: hana01.fqdn.corp:51013
Connection: keep-alive
Content-Length: 248

Step 5, Decode the username and password in the BASIC authentication string using a base64 decoder.  It’s possible to use an online one:

HANA SAPControl HTTP POST BASE64 decoder

The output includes the username and password in the following format:
USERNAME:PASSWORD

Step 6, With our new found details, log onto the HANA server using an SSH terminal:

HANA Server Logon

From this point onward it’s possible to access any data in the HANA database using command line tools.

SUMMARY:
You MUST enable SSL (TLS) encryption of the HTTP communications between the HANA Studio and the SAP Host Agent.  Without this, you might as well put the password on a post-it note on your screen.
See https://service.sap.com/sap/support/notes/1718944

Another option would be to segregate the HANA Studio users on their own vLAN, or to firewall the SAP HANA Host Agent and HANA database indexserver ports, tying them to specific user PCs only.
Incidentally, the password for the SYSTEM user of the HANA database, is encrypted with SHA256.  The encrypted string is then compared with the already encrypted password in the HANA database in order to authenticate a user.
However, if you have not enabled SSL between the HANA Studio and the HANA database indexserver, then all the of data retrieved from the database is sent in the clear.  You don’t need to authenticate to the database if you can just read the network packets.  This is true of most database connections.

Oracle 11g Transparent Data Encrpytion (TDE) Tablespace Conversion Script

After needing to configure and setup an Oracle tablespace in an 11g database so that it was encrypted with Transparent Data Encryption (TDE), I devised a scripted method of doing this quickly and simply.

The script is hardcoded to only work on a tablespace called “USERS_TDE” at the moment, but if you find and replace this for your specific tablespaces. I’ve even included commented out code showing how to do more than one tablespace at a time.

You should also change the disk locations within the script for the location of the export/import dump files and the wallet file location.
As per the Oracle docs, the wallet file is stored outside of the Oracle home in /etc/ORACLE.

The basic process is:

  • Create wallet dir.
  • Adjust sqlnet.ora.
  • Create dpump dir.
  • Export tablespace data.
  • Export tablespace DDL metadata.
  • Adjust tablespace DDL to include TDE commands.
  • Enable TDE encryption at database level (sets the enc key).
  • Offline & drop existing tablespace.
  • Create TDE version of tablespace.
  • Import tablespace data.

!/bin/bash
#############################################################################
# Author: D.Griffiths
# Script: oracle_TDE_encrypt.sh
# Params: None.
# RunAs:  Oracle home owner.
# Desc:   Script to automatically export a tablespace called USERS_TDE

#         (change as you wish), drop it, re-create it ENCRYPTED with TDE
#         then re-import the data objects back into the tablespace.
#
# HISTORY ###################################################################
# v1.0, Created.
#############################################################################

# Variable definitions.
wallet_dir="/etc/ORACLE/WALLETS/$ORACLE_SID"
walletpw=""
dp_dir="/data/backup/tblspace_export_pre_TDE/$ORACLE_SID" # Datapump export/import dir.

# Check required variables.
if [ -z "$ORACLE_SID" -o `grep -c "^$ORACLE_SID:" /etc/oratab` -ne 1 ] ; then
   echo "ERROR: Invalid ORACLE_SID."
   exit 1;
fi

if [ -z "$ORACLE_HOME" ] ; then
   export ORACLE_HOME = "`awk '/'$ORACLE_SID'/ { split($0,a,":"); print a[2] }' /etc/oratab`"

   if [ -z "$ORACLE_HOME" ] ; then
      echo "ERROR: Failed to set ORACLE_HOME."
      exit 1;
   fi
fi

# Show setup.
echo "------------------------------------------"
echo "Starting TDE setup script."
date
echo "The following has been defined:"
echo " ORACLE_SID : $ORACLE_SID"
echo " ORACLE_HOME : $ORACLE_HOME"
echo " WALLET DIR : $wallet_dir"
echo " DATAPUMP DIR: $dp_dir"
echo "------------------------------------------"
echo " "

# Check if DB already has wallet/encryption.
retval="`sqlplus -s "/ as sysdba" <<EOF
set head off
set newpage none
set feedback off
set trimout on
set tab on
select DECODE(count(status),0,'NONE','SOME') from v\$wallet;
EOF`"

if [ "$retval" != "NONE" ] ; then
   echo "ERROR: Encryption may already be enabled on this database."
   exit 1;
 else
   echo "Encryption is not already enabled on this database."
fi

# Check for existence of Wallet Dir.
if [ ! -d $wallet_dir ] ; then
   mkdir -p $wallet_dir
   if [ $? -ne 0 ] ; then
      echo "ERROR: Failed to create wallet dir $wallet_dir"
      exit 1;
   fi
   

   echo "WALLET directory created: $wallet_dir"
 else
   echo "WARNING: Wallet directory already exists:"
   echo "$wallet_dir"
   echo -n "Continue to use this? {Y|N} : "
   read reply
   if [ "$reply" != "Y" -a "$reply" != "y" ] ; then
      exit 1;
   fi
 

   echo "Using wallet dir: $wallet_dir"
fi
 

if [ ! -d $dp_dir ]; then
   mkdir -p $dp_dir;
   if [ $? -ne 0 ] ; then
      echo "ERROR: Failed to create datapump dir $dp_dir"
      exit 1;
   fi
 else
   echo "ERROR: Datapump dir already exists, files can not be overwritten."
   echo "Check: $dp_dir"
   exit 1;
fi

# Check sqlnet.ora file.
if [ "`grep -c '^ENCRYPTION_WALLET_LOCATION' $ORACLE_HOME/network/admin/sqlnet.ora`" -ne 1 ] ; then
   echo "Adding ENCRYPTION_WALLET_LOCATION to sqlnet.ora in $ORACLE_HOME/network/admin/sqlnet.ora"
   cat <<EOF >>$ORACLE_HOME/network/admin/sqlnet.ora
# Added WALLET location for TDE see 1228046.1.
ENCRYPTION_WALLET_LOCATION =
(SOURCE = (METHOD = FILE)
(METHOD_DATA =
(DIRECTORY = ${wallet_dir}/)))
EOF

   if [ $? -ne 0 ] ; then
      echo "ERROR: Failed to update sqlnet.ora."
      exit 1;
   fi
 else
   echo "Section ENCRYPTION_WALLET_LOCATION already exists in sqlnet.ora"
fi

# Get list of unencrypted non-system tablespaces
list_tblspace="`sqlplus -s "/ as sysdba" <<EOF
ttitle off
btitle off
set newpage none
set feedback off
set head off
set wrap off
set trim on
set tab on
set linesize 30
set pagesize 1000
-- select rpad(UPPER(t.name),30)
-- from v$tablespace t
-- where t.name not in ('SYSTEM','SYSAUX','UNDO','TEMP')
-- and t.ts# not in (select ts# from v$encrypted_tablespaces);
select rpad('USERS_TDE',30) from dual;
EOF`"

# Confirm tablespaces.
echo " "
echo "The following non-system unencrypted tablespaces have been found:"
echo "$list_tblspace"
echo -n "Setup these for TDE? {Y|N} : "
read reply

if [ "$reply" != "Y" -a "$reply" != "y" ] ; then
   exit 1;
fi

# Check dbdump oracle directory doesnt exist.
retval="`sqlplus -s "/ as sysdba"<<EOF
set head off
set newpage none
set feedback off
set trimout on
set tab on
select trim(directory_path) from dba_directories
where directory_name='DPDUMP_BACKUP';
EOF`"

if [ -n "$retval" ] ; then
   echo "WARNING: Oracle directory DPDUMP_BACKUP already exists."
   echo "It is assigned to path: $retval"
   echo "OK to recreate it to $dp_dir"
   echo -n "Enter {Y|N} : "
   read reply
   if [ "$reply" != "Y" -a "$reply" != "y" ] ; then
      exit 1;
   fi
fi

# Create the datapump export/import dir in oracle.
sqlplus -s "/ as sysdba" <<EOF
create or replace directory dpdump_backup as '$dp_dir';
EOF

echo "------------------------------------------"
echo "Exporting tablespaces and generating DDL."

# Export the tablespaces to the backup directory location and generate DDL.
echo "$list_tblspace" | while read tablespace
do
   expdp userid="'/ as sysdba'" dumpfile=$tablespace.dmp directory=dpdump_backup logfile=${tablespace}_exp.log tablespaces="$tablespace"

   # Generate the tablespace creation DDL to the backup directory location.
   sqlplus -s "/ as sysdba" << EOF
SET LONG 10000
set head off
set newpage none
set trimspool on
spool $dp_dir/cre_$tablespace.sql
SELECT dbms_metadata.get_ddl('TABLESPACE','$tablespace') FROM DUAL;
SPOOL OFF
EOF

   # Adjust the DDL file to include the ENCRYPTION string.
   cat $dp_dir/cre_$tablespace.sql | grep -e '^.*[A-Z0-9]' > $dp_dir/cre_TDE_$tablespace.sql

   echo " ENCRYPTION using 'AES256'" >> $dp_dir/cre_TDE_$tablespace.sql
   echo " STORAGE (ENCRYPT)" >> $dp_dir/cre_TDE_$tablespace.sql
 

done

########### ENABLE ENCRYPTION FROM HERE ON IN ################################

while [ -z "$walletpw" ] ; do
   echo -n "Enter the wallet encryption key to use: "
   read walletpw
   echo -n " Re-enter the wallet encryption key: "
   read reply
   if [ "$walletpw" != "$reply" ] ; then
      echo "WARNING: Typed keys do not match."
      echo -n "Try again? {Y|N} : "
      read reply
      if [ "$reply" != "Y" -a "$reply" != "y" ] ; then
         exit 1;
      fi
      walletpw=""
   fi

done

# Alter the DB to enable encryption.
sqlplus "/ as sysdba" << EOF
alter system set encryption key identified by "$walletpw";
EOF

if [ $? -ne 0 ] ; then
   echo "WARNING: Enabling encryption may have failed in the DB."
   exit 1;
fi

# Change permissions on all files in the wallet dir.
chmod 600 "$wallet_dir/*"

# Offline and then drop the tablespace and datafiles.
# WARNING: You should have a backup at this point.
echo "WARNING: About to offline and drop tablespaces:"
echo "$list_tblspace"
echo "##################################################"
echo "You should check the expdp export logs in $dp_dir."
echo "##################################################"

echo -n "Continue? [Y|N]: "
read reply
if [ "$reply" != "Y" -a "$reply" != "y" ]; then
   echo "Cancelling."
   exit 1
fi

# Offline and drop each tablespace (including datafiles) then re-create and import export dump file.
echo "$list_tblspace" | while read tablespace
do
   sqlplus "/ as sysdba" << EOF
alter tablespace $tablespace offline;
drop tablespace $tablespace including contents and datafiles;
@$dp_dir/cre_TDE_$tablespace.sql
-- @$dp_dir/cre_$tablespace.sql <<-- Recreates original non-TDE tablespace.
/
EOF

   impdp userid="'/ as sysdba'" dumpfile=$tablespace.dmp directory=dpdump_backup logfile=${tablespace}_imp.log tablespaces="$tablespace"

done

echo "#######################################################"
echo "You should enable auto-open of the wallet if necessary."
date
echo "End of script."
############################################################################