Parse an AWS CloudFormation template with the PyYA

Question

I am writing a custom Python application using the PyYAML library that needs to read in AWS CloudFormation YAML templates.

I know the templates are valid CloudFormation templates, because I tested them using validate-template:

▶ aws cloudformation validate-template --template-body file://cloudformation.yml

When I try to read them using the PyYAML library, however, I get errors like:

yaml.scanner.ScannerError: mapping values are not allowed here

and

could not determine a constructor for the tag "!Sub"

and others.

By way of example, I try this AWS example template:

▶ curl -s \
    https://raw.githubusercontent.com/awslabs/aws-cloudformation-templates/master/aws/services/CloudFormation/FindInMap_Inside_Sub.yaml \
    -o FindInMap_Inside_Sub.yaml

And then:

▶ python
Python 2.7.15 (default, Nov 27 2018, 21:40:55) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import yaml
>>> yaml.load(open('FindInMap_Inside_Sub.yaml'))

Which leads to:

yaml.constructor.ConstructorError: could not determine a constructor for the tag '!FindInMap'
  in "FindInMap_Inside_Sub.yaml", line 89, column 45

How can I parse a CloudFormation YAML file using a library like PyYAML or others?

Answer 1

It is possible to use the cfn_tools library that ships with the aws-cfn-template-flip project.

Install the library:

▶ pip install cfn_flip

Then the simplest Python to read in the template might be:

#!/usr/bin/env python

import yaml
from cfn_tools import load_yaml, dump_yaml

text = open('./FindInMap_Inside_Sub.yaml').read()
data = load_yaml(text)

print dump_yaml(data)

This library is not really documented but there are also various methods in there for customising the formatting of the output worth exploring.

Answer 2

Their aws-cfn-template-flip project that converts cfn templates to/from json and yaml is a good starting point. Example check out the yaml_loader.py script. It shows how it's adding yaml constructors. At the bottom, you'll see:

CfnYamlLoader.add_constructor(TAG_MAP, construct_mapping)
CfnYamlLoader.add_multi_constructor("!", multi_constructor)

You'll probably be interested in the construct_mapping method there. From there, you can look how the code works.