Chef 11 In-Depth: Client Improvements
On the client side, Chef 11 development focused on a broad range of improvements rather than a few major features. We had two sweet new features from our Co-MVPs, and a handful of improvements from Opscode.
Chef has always been focused on providing a solid way to manage your infrastructure at scale, which means that we have sometimes overlooked opportunities to provide tools for quick and easy hacking. For example, just to get started hacking on a recipe with chef-solo, you need to do something like this:
echo 'cookbook_path "/tmp/cookbooks"' > /tmp/solo.rb mkdir -p /tmp/cookbooks/test/recipes/ $EDITOR /tmp/cookbooks/test/recipes/default.rb chef-solo -c /tmp/solo.rb -o test
In addition to the extra typing, there’s a fair bit of Chef knowledge, like the structure of cookbooks, configuration parameters, and the (newish) override run list feature embedded in these commands.
To make all this a lot easier, Chef 11 ships with a
that allows you to execute a single recipe from the command line.
Instead of the above you just need to do:
$EDITOR hack_the_planet.rb chef-apply hack_the_planet.rb
Beyond being a cool new feature,
chef-apply is also a great story
about the awesomeness of open source.
chef-apply started life as a
quick hack that I posted to twitter.
I’d intended on cleaning it up someday, but life intervened, and the
code went nowhere. Then Bryan Berry
stepped in and did all the dirty work required to make it a part of core
Chef. It’s stuff like this that makes working on open source software
pretty much the best job ever. Thanks, Bryan!
Formatted Output By Default
When we developed why run mode for Chef 10.14, we realized we needed Chef’s output to be more structured so users could easily scan the information it generated. Since it’s such a big change to your day-to-day interaction with Chef, we wanted to give everyone a while to try out the formatted output and offer feedback before making it the default. In Chef 11, we’ve made the switch: whenever running Chef in an interactive terminal, you’ll get formatted output by default. We’re confident that this will be a huge improvement in your experience when running chef to check out changes you’ve made to your cookbooks. Contrarily, the logger output is still better suited for offline analysis or for integrating with log storage tools, so when you’re piping Chef’s output to another program (like your system logger), Chef will switch back to the logger output. If you want to override the automatic output selection, there are two new command line flags:
--force-formatter: Formatters all the time
--force-logger: Logger all the time
Note that the logger is not completely disabled when formatters are
enabled, because some kinds of messages, such as deprecation warnings,
are not yet integrated with output formatters. What we’ve done instead
is add a special log level, “auto”, and made this the default. When an
interactive terminal is detected, “auto” is equivalent to “warn,” and
when one is not detected, it’s equivalent to “info.” If you’re noticing
that you get both logger and formatter output, check your
setting in your config file. It’s likely that nodes originally
knife bootstrap and upgraded from 10.x will need a
configuration update, since the bootstrap in 10.x configures the log
level to info by default.
In Chef 11, just as in 10.14+, you can change the formatter using the
-F FORMATTER option. You can also write your own output formatter if
you don’t like the default ones. For a fun example, have a look at
Andrea Campi’s nyan cat formatter.
If you’re using Chef to manage applications with a huge single configuration file, you’ve probably wished you could break out the one or two sections you care about and manage those individually. Or as Bryan Berry put it in his feature request:
Templating such a large file is a mega PITA. JBoss AS 7 is nicely componentized so that you can easily turn on and off modules. This process would be a billion times easier and more readable if I could include fragments within an erubis template.
<%= render "partial.conf.erb" %>
The render method takes some additional options that allow you to set variables within the partial, lookup partials from disk, or use partials from a different cookbook. Rather than explain all the features here, I’ve created a sample cookbook that shows off the major features.
Chef client and solo now use filesystem-based locking to ensure that only one instance of Chef is actively configuring the system at a time. This will prevent unexpected issues that could occur if, say, you run Chef daemonized but SSH into a box to test cookbook changes or push an urgent update.
Note that the locking mechanism uses
flock. This means that if Chef
dies unexpectedly, the lock will automatically be released, so you don’t
need to deal with stale lockfiles blocking Chef runs. By default, the
lockfile is located at
filesystems (most notably NFS) don’t support
flock, so you’ll need to
manually configure the lockfile location if you’re using such a
filesystem. You also need to customize the lockfile location if you’re
running multiple configurations on a single host. The config setting for
# In client.rb or solo.rb lockfile "/not-nfs/chef-lockfile.pid"
Delayed Notifications Run on Failure
You’re probably familiar with this scenario:
- Chef reconfigures a service
- A delayed notification to restart the service is queued
- An unrelated resource fails and halts the Chef run
- Subsequent Chef runs don’t restart the service because it hasn’t been reconfigured during that run.
We originally felt this was the best design choice because restarting a partially configured service could cause it to fail when it wouldn’t have otherwise. In retrospect, we’ve found that it’s preferable to run the delayed notifications on failure to ensure that all configuration changes are applied, so Chef 11 changes this behavior: when a Chef run fails, all queued notifications are run. If any of those fail, Chef continues to run the remaining notifications and presents all failures at the end.
Get It Now
Before upgrading head over to the breaking changes page so you’re aware of the changes in this release. When you’re ready to upgrade, hit up the Chef install page to get Chef 11 and whip up some awesome.