Chef 0.10 Preview: Encrypted Data Bags

Chef 0.10 Preview: Encrypted Data Bags

Many server roles, such as databases, require setting up passwords as part of their configuration. A common pattern is to put such passwords in a data bag stored on the Chef server. Recipes can then access the data bag to automate the configuration. A downside of this approach is that passwords are stored in plain text on the Chef server and also within your version control system if you use one to manage your cookbooks, roles, and data bags. In Chef 0.10, you can use Encrypted Data Bags to reduce the exposure of sensitive data stored in data bags to the servers that need it for their configuration and the operators that need to modify it. Here’s what you need to know about encrypted data bags before we jump into a few examples:

1. Only the values of an encrypted data bag are encrypted. The keys remain in plain text so that you can still use search (on the keys) to find items in encrypted data bags.
2. All values will be encrypted. If you have a data bag containing sensitive and non-sensitive information, split it into two separate data bag items.
3. The encryption relies on a shared secret. At this time, you are responsible for creating and distributing the shared key. For example, you will need to devise a mechanism of making the key available to servers that need to decrypt values for their configuration.

Using knife to create an Encrypted Data Bag

Before we can create an Encrypted Data Bag, we need a secret key. We’ll use openssl to generate a random secret.

$ openssl rand -base64 512 > secret_key

Now we can create an encrypted data bag item within the “prod” data bag:

$ knife data bag create --secret-file ./secret_key prod passwords
# edited data bag item in editor and saved
Created data_bag[prod]
Created data_bag_item[passwords]

(Note: there are 2 dashes (-) before secret in the above command)

This will pop open your editor. Here’s what I filled in for the example:

{
  "id": "passwords",
  "mysql": "open-sesame-123",
  "rabbitmq": "open-queue-123"
}

Let’s take a look at the result without decrypting (if you are following along, your values will depend on the secret used):

$ knife data bag show prod passwords
id:        passwords
mysql:     xtSxLvqHqPP1gHsqP5SlytFtDIfpWMJebJ2aZPd0mGU=
rabbitmq:  wVcK/OboqpRcfF5fOKlEHKz2ev7CxSrBsoCwWo9Jcko=

To see the decrypted values, specify the secret key:

$ knife data bag show --secret-file=./secret_key prod passwords
id:        passwords
mysql:     open-sesame-123
rabbitmq:  open-queue-123

(Note: there are 2 dashes (-) before secret in the above command)

Next, we’ll look at how to use encrypted data bags within recipes for node configuration.

Accessing Encrypted Data Bags from a Recipe

Recipes can access decrypted values using Chef::EncryptedDataBagItem.load. If you don’t specify a shared secret when calling the load method, Chef will look for a file based on the value of the Chef::Config[:encrypted_data_bag_secret] config entry (which you can set for a node via /etc/chef/client.rb). The default value is /etc/chef/encrypted_data_bag_secret. To demonstrate the use of encrypted data bags on a node, we’ll start by copying the secret_key file created above to our example node using scp and moving it to /etc/chef/encrypted_data_bag_secret.

scp ./secret_key $MY_NODE_IP:~/
ssh $MY_NODE_IP
sudo mv ./secret_key /etc/chef/encrypted_data_bag_secret

Next, we’ll create a recipe that will log the decrypted values for demonstration purposes (if these were real secrets, you would want to avoid logging them).

knife cookbook create edb_demo

Now edit cookbooks/edb_demo/recipes/default.rb so that it contains the following:

# cookbooks/edb_demo/recipes/default.rb
passwords = Chef::EncryptedDataBagItem.load("prod", "passwords")
mysql = passwords["mysql"]
Chef::Log.info("The mysql password is: '#{mysql}'")

Finally, upload the cookbook and run chef-client on the node. You should see something like this:

knife cookbook upload edb_demo
# output clipped
knife ssh name:i-8a436fe5 -a ec2.public_hostname 'sudo chef-client'
INFO: *** Chef 0.10.0.rc.2 ***
INFO: Run List is [recipe[edb_demo]]
INFO: Run List expands to [edb_demo]
INFO: Starting Chef Run for i-8a436fe5
INFO: Loading cookbooks [edb_demo]
INFO: The mysql password is: 'open-sesame-123'
INFO: Chef Run complete in 3.122228 seconds
INFO: Running report handlers
INFO: Report handlers complete

As you can see, the recipe was able to decrypt the values in the encyrpted data bag. It did so by using the shared secret located in the default location of /etc/chef/encrypted_data_bag_secret.

Wrap Up

Encrypted Data Bags are an easy way to add a layer of security to protect sensitive configuration data like passwords and cloud credentials. You can use knife to create and view encrypted data bags. By placing the shared secret in a known location on a node, you can allow the node to access the protected data via a recipe. You can learn more about Encrypted Data Bags and some of the additional options they support on the Chef Wiki