How to make a list of pairs unique in Jinja?

Question

I have a dictionary with array values coming from an Ansible vars file.

ipfilter:
  zone1:
    - { application: "app 1", local_ip: 1.1.1.1 }
    - { application: "app 1", local_ip: 1.1.1.1 }
    - { application: "app 2", local_ip: 2.2.2.2 }
  zone2:
    - { application: "app 3", local_ip: 3.3.3.3 }
    - { application: "app 4", local_ip: 4.4.4.4 }
    - { application: "app 4", local_ip: 4.4.4.4 }

The problem is, that the tuple of application and local_ip is not unique, because the objects contain additional data, which I have omitted in the example, because it is not relevant for the following script.

The script is a Jinja template delivered by Ansible to a Solaris server:

#! /bin/bash
set -eu
cat <<';' |
{% for zone, rules in ipfilter.items() %}
{%   for rule in rules %}
 {{zone}} {{rule.application}} {{rule.local_ip}}
{%   endfor %}
{% endfor %}
;
sort | uniq | while read zone application local_ip; do
  ipfcfg delete "$zone" "$application" "$local_ip"
done

The outer loop iterates over the dictionary and the inner loop iterates over the array values.

I am not able to make the list of tuples unique with Jinja. Therefor I had to use the sort | uniq | while read loop in Bash. This is far from perfect and has its own limitations.

How to make the list of tuples unique in Jinja instead?

Answer 1

To get unique list pairs from your example, you can simply use:

ipfilter.values() | sum(start=[]) | unique

But this will not work in your case, because you have omitted other keys from your original data, so unique filter will not work.

You will have to workaround this with a bit of Ansible variables templating magic prior to your task:

# construct list of tuples
- set_fact:
    tmp_app: '{"app":"{{item.application}}","ip":"{{item.local_ip}}"}'
  with_items: "{{ ipfilter.values() | sum(start=[]) }}"
  register: tmp_apps
# pass uniq list to template
- template:
    src: script.j2
    dest: script.sh
  vars:
    uniq_apps: "{{ tmp_apps.results | map(attribute='ansible_facts.tmp_app') | list | unique }}"

and script.j2:

#! /bin/bash
set -eu
{% for app in uniq_apps %}
ipfcfg delete "{{app.app}}" "{{app.ip}}"
{% endfor %}

Answer 2

Thanks to Konstantin Suvorov's and Sean Vieira's answer to another question I was able to do all of my stuff in Jinja.

#! /bin/bash
set -eu
{% set all_rules = [] %}
{% for zone, rules in ipfilter.items() %}
{%   for rule in rules %}
{%     set x = all_rules.extend([{'zone': zone, 'application': rule.application, 'local_ip': rule.local_ip}]) %}
{%   endfor %}
{% endfor %}
{% for item in all_rules|unique %}
ipfcfg delete {{item.zone|quote}} {{item.application|quote}} {{item.local_ip|quote}}
{% endfor %}