Chef convergence fails with ERROR: Unable to deter

Question

My server fails to converge with the following messages. I am unable to figure where this error is coming from.

I am copying and pasting the chef stacktrace AFTER the client.log messages below:

Weird thing is that the client.rb file stacktrace is refering to does not even exist on this box.

Any help will be appreciated!

[root@xyz chef]# tail client.log

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 

[2016-04-21T11:47:24-06:00] INFO: *** Chef 11.12.8 ***
[2016-04-21T11:47:24-06:00] INFO: Chef-client pid: 7309
[2016-04-21T11:47:24-06:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-04-21T11:47:24-06:00] ERROR: Unable to determine node name: configure node_name or configure the system's hostname and fqdn
[2016-04-21T11:47:24-06:00] ERROR: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[2016-04-21T11:47:24-06:00] ERROR: Sleeping for 3600 seconds before trying again
[2016-04-21T11:51:10-06:00] INFO: SIGTERM received, exiting gracefully

---

STACK TRACE:

cat /var/chef/cache/chef-stacktrace.out
Generated at 2016-04-21 11:47:24 -0600
Chef::Exceptions::CannotDetermineNodeName: Unable to determine node name:    configure node_name or configure the system's hostname and fqdn
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-  11.12.8/lib/chef/client.rb:299:in `node_name'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/client.rb:313:in `register'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/client.rb:416:in `do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/client.rb:213:in `block in run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/client.rb:207:in `fork'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/client.rb:207:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/application.rb:217:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/application/client.rb:328:in `block in run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/application/client.rb:317:in `loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/application/client.rb:317:in `run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/application.rb:67:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:23:in `load'
/usr/bin/chef-client:23:in `<main>'[root@xyz chef]# 

Answer 1

As someone else pointed out 11.12.8 is kind of ancient code at this point. You do seem to have the patch that was added to add ohai[:machinename] to the detection of the node_name which means that it should fall back to hostname (which should at least be the empty string I would think? can we set an empty string for the hostname?) which would then avoid the exception.

Maybe your ohai version is also too old to have node[:machinename]? But the patch to ohai and chef-client went in at basically the same time.

I literally was the last person to touch the node_name detection code and I can't see how you would be hitting that exception...

Best I can suggest is to see if you can replicate it on a more recent version of the omnibus install of chef-client because that "should" never happen. Probably in order to debug this we need to know what the output of ohai is on your machine (including hostname, machinename, fqdn and domain values in particular). Also the output of running /bin/hostname and /bin/hostname -f.

Answer 2

You need to configure a node_name in your /etc/chef/client.rb. Sometimes Ohai can figure this out on its own, but it doesn't seem like it in your case. check hostname, hostname -f, and the /etc/hostname file to see what your servers are configured for.

I personally always define my node_name in chef, and then use the node name to set the hostname and fqdn... not the other way around as the Chef doc's recommend, depends on your setup.

For example, my client.rb looks like:

chef_server_url "..."
client_fork true
log_location "/var/log/chef/client.log"
validation_client_name "...-validator"
verify_api_cert true # you might not want this one unless you have a valid cert
node_name "my_node_name" # Add this line
...

From the chef client.rb docs

node_name The name of the node. Determines which configuration should be applied and sets the client_name, which is the name used when authenticating to a Chef server. The default value is the FQDN of the chef-client, as detected by Ohai. In general, Chef recommends that you leave this setting blank and let Ohai assign the FQDN of the node as the node_name during each chef-client run.

Answer 3

As you noticed, the configuration file does not exist. You have to create it and define the mentioned options. See the documentation for details.

Answer 4

Alternatively to the above, fix your server so that hostname -f shows something.