How can I generate Hocon conf file dynamically usi

Question

I would like to use automation to create the hocon configuration with python 3 scripting. I read that lightbend (https://github.com/lightbend/config) recommends pyhocon (https://github.com/chimpler/pyhocon).

I am having problems figuring out how to create an Hocon object and write the data to a file as hocon. It is important to me that the syntax for the substitution are in the result.

For example I expect the output of the file myconfig.conf to look something like this:

{
   Environment: "dev"
   JobName: ${Environment}"-hello-bob"
}

So, I assumed that there was a way to do something like this:

config2 = ConfigFactory.parse_string("{}")
config2.put("Environment", "dev")
#Some type of object for references or special syntax for ${Environment}  
config2.put("JobName", "${Environment}")

Then after creating the stuffed object there should be a simple way to write out to a file or files:

filename = "myconfig.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(config2.to_hocon_str)

Has anyone figured a way to do this? It seems odd that the library can only be used for reading data only.

Answer 1

So, I decided to look at documentation for JVM (Java/Scala) library (https://github.com/lightbend/config). After reading the documentation, there was a clear section on hocon examples (https://github.com/lightbend/config#examples-of-hocon). In this documentation, they categorized 7 valid hocon styles. I call these styles because if I was to automate the generation of these files, I would be picking one way to write out and sticking with it.

All of these are valid HOCON.

1.Start with valid JSON:

{
    "foo" : {
        "bar" : 10,
        "baz" : 12
    }
}

2.Drop root braces:

"foo" : {
    "bar" : 10,
    "baz" : 12
}

3.Drop quotes:

foo : {
    bar : 10,
    baz : 12
}

4.Use = and omit it before {:

foo {
    bar = 10,
    baz = 12
}

5.Remove commas:

foo {
    bar = 10
    baz = 12
}

6.Use dotted notation for unquoted keys:

foo.bar=10
foo.baz=12

7.Put the dotted-notation fields on a single line:

foo.bar=10, foo.baz=12

Because I will be using the pyhocon library, I needed to look for write solutions within the library. I found some help from chimpler's git (https://github.com/chimpler/pyhocon). What I found was that they have two hocon styles which can be simply written out. One is json and the other is something that wasn't on the list which was describe above by lightbend.

Style 1: pure JSON, witch can be written out in two ways:

HOCONConverter.to_json

#Using HOCONConverter.to_json
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")

filename = "./json_coverted.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(HOCONConverter.to_json(confTree))

HOCONConverter.to_json Result

{
    "Environment": "Dev",
    "Test": "${Environment}"
}

OR Using json.dump

#Using json.dump
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./json_dumped.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(json.dumps(confTree,indent=4))

Using json.dump Result

{
  "Environment": "Dev",
  "Test": "${Environment}"
}

Pyhocon's other Style, not listed by lightbend

# HOCONConverter.to_hocon
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./hocon_coverted.txt"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(HOCONConverter.to_hocon(confTree))

Pyhocon's other Style, not listed by lightbend Result

Environment = "Dev"
Test = "${Environment}"

So, to answer my own question the only dependable way to generate a hocon conf file dynamically using pyhocon in Python 3 is by using one of the json methods (converter or dumps). But this still leaves an open question. The question being, will reading a json to a pyhocon ConfTree object be able dereference the substitutions when they are in the json?

For example if I read the file

{
    "Environment": "Dev",
    "Test": "${Environment}"
}

Will the ConfTree object get "Dev" as the value for Test?
No, it will not. Here is my test

filename = "json_coverted.conf"
print("Reading file{}".format(filename))
conf = ConfigFactory.parse_file(filename)
key="Test"
value=conf.get(key)
print("Key:{} Value:{}".format(key,value))

Test Result Out to screen

Reading filejson_coverted.conf
Key:Test Value:${Environment}

So, then how does one use pyhocon with substitutions?