Why would you need HA
While the previous Cloud HSM article was mostly covering “Why Cloud HSM is Important” topics, this one describes technical details about how Luna HA (High Availability ) cluster can be built in a cloud and provides links to scripts that could help automating and codifying a rather complicated Luna’s setup process.
It’s obvious that when you build an HA system with subsystems that rely heavily on cryptographic services built around Luna, you need to put latter to the same HA category. That’s why a single Luna appliance is not usually sufficient and you need an array (or cluster) of HSM’s that look and behave as a single one from a client’s point of view.
The first few chapters of this document cover manual Luna and Luna array configuration topics, while “Setup Automation” section describes a command line tool that allows automating the whole process by creating a JSON configuration file and running a Python script. You do need to go through manual setup topics if you didn’t have a prior experience with configuring Luna, otherwise it could be very difficult to understand how to create the JSON file and troubleshoot possible issues.
Unfortunately, there is no way to deploy a Luna device to a cloud in the same automated manner (e.g. through CloudFormation) as other pieces of AWS infrastructure. There is a well documented manual process for that, which is not very difficult to follow, but it’s still manual. The process is described here.
The two important things that are worth mentioning are the facts that you’ll need a VPC to deploy a Luna appliance and that you’ll need at least two devices to create an HA array.
After the appliances are provisioned to a VPC, you’ll be given managers passwords that could be used to connect to the devices remotely and perform all necessary configuration jobs.
Configuring Luna Servers
You’ll need to login to a Luna device using SSH from a client machine to perform a configuration job. I would strongly recommend to enable key based authentication on a Luna device, because most likely you’ll need to SSH to the device many times before you’re done with configuration and verification:
scp <public-cert-file-name> manager@<luna-ip>:.
sysc ssh pu esysc ssh pu a -f <public-cert-file-name>
where <public-cert-file-name> is a public certificate generated by a ‘ssh-keygen’ command on a client machine.
The following high level manual steps are required to configure a Luna server:
- ‘hsm init’ command to initialize the device
- ‘sysconf re’ command to regenerate server side certificates
- ‘ntls bind’ command to restart Luna’s network interfaces
- ‘hsm login’ to as admin to Luna
- ‘par cr …’ to create a partition
- ‘c reg …’ to register client
- ‘c a …’ to assign a partition to a client
The Luna configuration process is described in details here.
Configuring Luna Clients
This one is interesting and requires some re-thinking because of differences introduced by AWS’ auto scaling groups (ASG). In a traditional ‘static’ environment each client would require a unique client certificate and an IP (or client’s host name) to be registered on a Luna server. Since EC2 instances can randomly go up and down in ASG that approach would be difficult to implement. Fortunately, there is a way around that allows sharing a single client’s certificate for the whole ASG.
The first step in configuring clients is to download and install Luna’s client tools and libraries that are available for free:
After these two components are installed, ‘vtl’ command line tool used for a client’s setup could be found at the following location on Linux: /usr/lunasa/bin/vtl. A new client certificate and a private key can be generated by running the following command:
vtl createCert -n <cert_name>
A newly generated certificate will be stored at ‘/usr/lunasa/cert/client/<cert_name>.pem’ and will need to be transferred to a Luna server for further registration:
scp /usr/lunasa/cert/client/<cert_name>.pem manager@<luna_server>:.
A trick that allows registering the whole ASG without binding a registration to an IP is to use <cert-name> as a parameter in ‘-hostname’ option and not to use ‘-ip’ option at all. It’s not obvious, but it definitely works. A command on the server will look like this:
c reg -c <client-name> -h <cert-name>
where <client-name> is a logical name that the server will use to refer the new client and <cert-name” is the same cert that we’ve just created on the client using ‘vtl’ command.
To replicate the generated certificate and private key to other ASG members you’ll need to place the generated files to /usr/lunasa/cert/client/ and to make sure that you have two following entries in ‘LunaSA Client’ section of ‘/etc/Chrystoki.conf’ file:
ClientPrivKeyFile = /usr/lunasa/cert/client/<cert-name>Key.pem;
ClientCertFile = /usr/lunasa/cert/client/<cert-name>.pem;
You’ll also need to register a Luna server on the client to be able to connect to that server:
scp manager@<luna-server-host>:server.pem .
vtl addServer -n <luna-server-host> -c server.pem
Configuring Luna HA Cluster
You’ll need at least two Luna servers configured as described in “Configuring Luna Servers” section with the following limitations:
- Admin and partition passwords should be the same.
- Partition names for partitions participating in HA cluster should be the same
- Cloning domain names should be the same
If the conditions above are met, registering the cluster should be easy:
vtl haAdmin -newGroup -serialNum <par-ser-nbr1> -label <group-name> -password <par-pwd>
vtl haAdmin -addMember -serialNum <par-ser-nbr2> -group <group-ser-num> -password <par-pwd>
<par-ser-nbr1> and <par-ser-nbr1> – serial numbers of partitions included to the cluster
<group-name> – is a logical name for the newly created cluster
<par-pwd> – is a partition password
<group-ser-num> – a serial number for the newly created group (it will be displayed after the group is created by ‘newGroup’ command.
To figure out what <par-ser-nbr1> and <par-ser-nbr1> parameters are, you can run the following command on a client:
The output should look as below:
The following Luna SA Slots/Partitions were found:
Slot Serial # Label
==== ======== =====
1 <par-ser-nbr1> <par name>
2 <par-ser-nbr1> <par name>
Configuring Java Client
Luna’s client includes Java classes that implement traditional JCA architecture. You’ll need to include LunaProvider.jar to a classpath and modify java.security file that can be normally found at the following location: <JRE-DIR>/lib/security/java.security. The section that needs to be updated looks as follows:
To enable Luna Provider, add the following line to the list:
Testing HA Luna from a Java application
You can find many Java sample applications under following location: /usr/lunasa/jsp/samples/com/safenetinc/luna/sample. Connecting to an HA cluster is not different from connecting to a single Luna device – you just need to know a correct Slot number that represents the Luna array. ‘vtl haAdmin -show’ command can be used to find out what HA slot number is:
[ec2-user@ip-10-0-1-225 lunasa]$ /usr/lunasa/bin/vtl haAdmin -show
================ HA Group and Member Information ================
HA Group Label: ha_group
HA Group Number: <grp-ser-nbr>
HA Group Slot #: <grp-slot-nbr>
Group Members: <par-ser-nbr1>, <par-ser-nbr1>
Standby members: <none>
Slot # Member S/N Member Label Status
====== ========== ============ ======
1 <par-ser-nbr1> <par_name> alive
2 <par-ser-nbr2> <par_name> alive
The slot number that you’re looking for is <grp-slot-nbr>.
If you look at Java sample found in the KeyStoreLunaDemo.java file, you’ll find the following lines :
ByteArrayInputStream is1 = new ByteArrayInputStream((“slot:1”)
This is the place where you would need to use the slot number displayed by ‘vtl haAdmin -show’ command.
As you’ve probably noticed already, the Luna HA setup process is rather cumbersome and doesn’t fit well to a major cloud concept that assumes a great deal of automation. I’ve tried to address the issue by creating a Python package that would allow setting up a Luna HA cluster by running a single command:
luna_mech -a -g -r <luna-array-config-file>
or, if you want to configure a single Luna appliance, you can run:
luna_mech -l -g -r <luna-config-file>
An idea here is to put all Luna parameters to a JSON file and make the Luna Mechanizer to parse and interpret it. The next step could be to in integrate the mechanizer with a CloudFormation framework.
The code can be found in a git repo @ sf.net and can be downloaded by usual means:
git clone git clone http://git.code.sf.net/p/lunamech/code luna_mech
git clone git://git.code.sf.net/p/lunamech/code luna_mech
Check README file for further instructions.
Since a cloud environment is not commonly considered as trusted, there is still a problem of passing secrets such as an HA partition password and client side private key to ASG members. You definitely don’t want to “bake” secrets like this to an AMI or even store them in an encrypted S3 bucket, let alone putting them to unencrypted EC2’s “user data“.
I’ll try to explore other AWS specific ways of passing secrets from an internal DC to a cloud in the next blog.