Provision developer environment with chef server a

Question

I wish to create a developer environment of a web based application which will have an application server and database server installed on a VM using vagrant. I am using open source chef server and vagrant with chef_client provisioner. However each time a developer node is provisioned, chef_client provisioner creates a client node on chef server. This is not desirable because this setup will be used by a lot of developers. Such nodes might get created and destroyed several times. So maintaining the list of such nodes on chef server is not required.

chef_solo is another option but it requires the cookbook to be present in the system. We are using chef server as a repository for cookbooks as well. So a developer need not have the specific cookbook in his/her system. I have looked at berkshelf as well where I can configure chef_api path to the chef server. But that too requires a node. Following is my vagrant file:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "precise64-chef"
  config.vm.network :private_network, ip: "33.33.33.10"
  config.omnibus.chef_version = '11.8.2'
  config.vm.provision "chef_client" do |chef|
    chef.log_level = :debug
    chef.environment = "development"
    chef.delete_node = true
    chef.delete_client = true
    chef.node_name = "sample_app"
    chef.chef_server_url = "https://chef-server-url"
    chef.validation_key_path = "chef-validator.pem"
    chef.add_role "base"
    chef.add_role "pg_client"
    chef.add_role "application_server"
    chef.add_role "db_server"
    chef.add_recipe "deploy_application"
  end
end

I wish to not create sample_app node on chef server. Otherwise this serves my purpose. Any other strategies to provision developer environment using chef server are welcome.

Answer 1

Immediate solution

Butcher plugin

The immediate answer that springs to mind is use the vagrant butcher plugin. This will automtically delete your developer client and node entries on the chef server.

Longer term solution

Since you're aware of the Berkshelf "chef_api" directive I will assume you have the typical Enterprise scenario where several developers are sharing locally developed cookbooks.

Here's what I'm doing with chef....

Chef cookbook repository pattern

Reserve an instance of chef server as a repository of cookbooks. Each released cookbook version is pushed here and made available for consumption by all internal users of chef.

To maintain the repository cookbooks, use Jenkins to implement a CI process for cookbooks. Jenkins jobs pull from local GIT repositories, or from the community cookbook site. The advantage of this approach is that tests can be automated using chef spec and QA rules can be implemented using tools like foodcritic. In this manner build failures help improve the overall quality of the cookbooks in use within the organisation.

I reckon this repository pattern is something we're going to a lot more of when the Berkshelf guys finally launch Berkshelf 3.0 (See also berkshelf-api).

Use Berkshelf to load cookbooks

Berkshelf is used to control the loading of cookbooks into all chef servers (dev or prod). The following is a demo "Berksfile":

chef_api "https://cookbook.repo.myorg.com/", node_name: "dev1", client_key: "/path/to/dev1/private/key/key.pem"

cookbook "apache2"
cookbook "mysql"
...

The "chef_api" directive tells berkshelf to load all cookbooks from the local chef cookbook repository, instead of defaulting to loading from the community repository.

This makes it trivial to create "production-like" copies of your main chef server. It also enables one of the more magical aspects of berkshelf. Each cookbook in your system can have it's own simple Berksfile:

chef_api "https://cookbook.repo.myorg.com/".....

metadata

The "metadata" statement tells Berkshelf to load the cookbooks listed as dependencies in the cookbook's metadata file. This is wonderful, a simple "berks upload" and all the cookbooks I need get automatically loaded into my development chef server.

Use Spiceweasel for loading chef server

Very useful tool. Integrates Berkshelf and generates all the other knife commands for loading a chef server from a chef repository.

https://github.com/mattray/spiceweasel

Development: Chef Zero and Vagrant

Once cookbooks are under control (using Berkshelf) my advise is to adopt Chef zero. Each developer can then have their own locally managed chef server instance.

Here are the vagrant plugins I routinely install to support chef cookbook development:

• Omnibus plugin : Ensures I'm running a specified version of chef-client
• Berkshelf plugin : Downloads and manages my cookbooks
• Chef zero plugin : Runs a local copy of chef server

Once a cookbook is ready, it gets committed and pushed. Jenkins will then pick up the new revision and automatically loads it into the cookbook repository.