Running an Ansible Playbook using Python API 2.0.0

Question

Ansible version : 2.0.0.1

I've been looking around quite a bit now, and most documentation I find is either incomplete or deprecated (this post is for version 1.8.4, ie)

I'm trying to launch an Ansible playbook through the Python API. Ansible's documentation seem to be showing how to generate and play tasks, but not how to load and run a playbook yml file. I've been digging into the code to try to understand how to launch it, and I think I've done some progress, but I'm really hitting a wall. Here's what I have so far :

def createcluster(region, environment, cluster):
    Options = namedtuple('Options', ['region','env', 'cluster'])

    # initialize needed objects
    variable_manager = VariableManager()
    loader = DataLoader()
    options = Options(region=region, env=environment, cluster=cluster)
    options.listhosts = False
    vault_password = getpass.getpass('Enter vault password :')
    passwords = dict(vault_pass=vault_password)

    #Getting hosts
    hostsread = open('provisioning/inventory/hosts','r')
    hosts =  hostsread.read()
    inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list=hosts)
    variable_manager.set_inventory(inventory)
    #Create and load the playbook file
    playbook = Playbook(loader)
    playbook.load('provisioning/cluster.yml', variable_manager,loader)

    #Create an executor to launch the playbook ? 
    executor = None
    executor = PlaybookExecutor(playbook,inventory,variable_manager,loader,options,passwords)
    try:
        result = executor.run()
    finally:
        if executor is not None:
            executor.cleanup()

I'm not sure at all about the executor part, and I keep getting a "AttributeError: 'Options' object has no attribute 'listhosts'" error when I try to launch the code (weirdly enough as it should just ignore its absence, I think (line 60))

How am I supposed to load a YML file and launch it through the Python API ? Am I on the good path or did I lose myself ? Why isn't Ansible better documented ? Why would 42 be the answer to 7*7 ?

Answer 1

Disclaimer

Posting for completion.
I had trouble setting verbosity for ansible 2.4. I will mainly talk about that.

TL;DR

Ansible use a global Display object in the __main__ file (the one you launch) if it doesn't exist some imports will create it.
This is considered a bad practice and not PEP8 compliant (second bullet point)

---

Explanation part

versions: (I am using python virtualenv)

• ansible = 2.4.2.0 (also tested in 2.4.1.0)
• python = 2.7.13

How is it used inside ansible

try:
    from __main__ import display
except ImportError:
    from ansible.utils.display import Display
    display = Display()

It is called in almost every file (108). Like so you have a new display in your entry point and then all other module will retrieve this first declared display.

Running with another verbosity

You just have to declare a Display object like so:

from ansible.utils.display import Display
display = Display(verbosity=5)

You can alternatively use this after: display.verbosity = 1000

---

Problem

I wanted to be able to completely remove ansible output (negative value = no output)

Solving

I ended up creating a new class like so:

from ansible.utils.display import Display

class AnsibleDisplay(Display):
    ''' 
    This  class override the display.display() function
    '''
    def display(self, *args, **kwargs):
        if self.verbosity >= 0:
            super(AnsibleDisplay, self).display(*args, **kwargs)

Then import it in my __main__ file

# Ansible call this global object for display
sys.path.append(ROOT_DIR + os.sep + 'lib')
from ansible_display import AnsibleDisplay
display = AnsibleDisplay(verbosity=0)

And only after import all other modules

Example

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
ROOT_DIR = os.path.dirname(SCRIPT_DIR)

## For ansible
import json
from collections import namedtuple

# Ansible call this global object for display
sys.path.append(ROOT_DIR + os.sep + 'lib')
from ansible_display import AnsibleDisplay
display = AnsibleDisplay(verbosity=0)

# Load other libs after to make sure they all use the above 'display'
from ansible.parsing.dataloader import DataLoader
from ansible.vars.manager import VariableManager
from ansible.inventory.manager import InventoryManager
from ansible.playbook.play import Play
from ansible.executor.playbook_executor import PlaybookExecutor

def apply_verbosity(args):
    global display
    verb = -1 if args.verbosity is None else args.verbosity
    display.verbosity = verb

def ansible_part():
    playbook_path = "%s/ansible/main_playbook.yml" % (ROOT_DIR)
    inventory_path = "%s/watev/my_inventory.ini"

    Options = namedtuple('Options', ['connection', 'module_path', 'forks', 'become', 'become_method', 'become_user', 'check', 'diff', 'listhosts', 'listtasks', 'listtags', 'syntax'])
    # initialize needed objects
    loader = DataLoader()
    options = Options(connection='local', module_path='%s/' % (ROOT_DIR), forks=100, become=None, become_method=None, become_user=None, check=False,
                    diff=False, listhosts=True, listtasks=False, listtags=False, syntax=False)
    passwords = dict(vault_pass='secret')

    # create inventory and pass to var manager
    inventory = InventoryManager(loader=loader, sources=[inventory_path])
    variable_manager = VariableManager(loader=loader, inventory=inventory)

    pbex = PlaybookExecutor(playbooks=[playbook_path], inventory=inventory, variable_manager=variable_manager, loader=loader, options=options, passwords=passwords)
    results = pbex.run()

def main():
    ansible_part()

Notes

1. I needed to add the 4 options to the namedtuple:
   listhosts=True, listtasks=False, listtags=False, syntax=False
2. The import __main__ makes debugging impractical because when using debugger (in my case pudb), the __main__ file is the debugger file hence from __main__ import display will never work

hth

[Edit1]: added note

Answer 2

Here is an example with Ansible 2:

#!/usr/bin/python2

from collections import namedtuple
from ansible.parsing.dataloader import DataLoader
from ansible.vars import VariableManager
from ansible.inventory import Inventory
from ansible.playbook import Playbook
from ansible.executor.playbook_executor import PlaybookExecutor

Options = namedtuple('Options', ['connection',  'forks', 'become', 'become_method', 'become_user', 'check', 'listhosts', 'listtasks', 'listtags', 'syntax', 'module_path'])

variable_manager = VariableManager()
loader = DataLoader()
options = Options(connection='local', forks=100, become=None, become_method=None, become_user=None, check=False, listhosts=False, listtasks=False, listtags=False, syntax=False, module_path="")
passwords = dict(vault_pass='secret')

inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list='localhost')
variable_manager.set_inventory(inventory)
playbooks = ["./test.yaml"]

executor = PlaybookExecutor(
              playbooks=playbooks,
              inventory=inventory,
              variable_manager=variable_manager,
              loader=loader,
              options=options,
              passwords=passwords)

executor.run()

Tested with Python 2.7.10 and ansible 2.0.1.0

Answer 3

I wrote this without seeing you want version 2. Leaving it, though it isn't the correct answer.

This will work in 1.9. You can modify your createcluster() command to call it.

def run_ansible():
  vaultpass = "password"
  inventory = ansible.inventory.Inventory("provisioning/inventory/hosts", vault_password=vaultpass)

  stats = callbacks.AggregateStats()
  playbook_cb = callbacks.PlaybookCallbacks(verbose=3)

  pb = ansible.playbook.PlayBook(
            playbook=playbook,
            inventory=inventory,
            extra_vars=parsed_extra_vars,
            #private_key_file="/path/to/key.pem",
            vault_password=vaultpass,
            stats=stats,
            callbacks=playbook_cb,
            runner_callbacks=callbacks.PlaybookRunnerCallbacks(stats, verbose=3)
        )
  pb.run()

  hosts = sorted(pb.stats.processed.keys())

  failed_hosts = []
  unreachable_hosts = []
  for h in hosts:
    t = pb.stats.summarize(h)
    if t['failures'] > 0:
      failed_hosts.append(h)
    if t['unreachable'] > 0:
      unreachable_hosts.append(h)

  print("failed hosts: ", failed_hosts)
  print("unreachable hosts: ", unreachable_hosts)

  retries = failed_hosts + unreachable_hosts
  print("retries:", retries)
  if len(retries) > 0:
    return 1
  return 0