September 21, 2017 · network automation ansible configuration change validation set operations difference cisco ios

Networking with Ansible 108.2

This post will look closer into how we can build in configuration validation in our plays. In this case we will be using Ansible Core Modules with some jinja filters to ensure idempotency. The playbook is not only going to push the changes we want, it is also going to ensure that the targets are only configured with these items and nothing else.

Part 1: Configuring changes and identifying our problem

We are going to work with the following files:

group_vars/all.yml

---
name_server1: 8.8.8.8  
name_server2: 8.8.4.4

default_domain: bogus1.com

ntp1: 130.126.24.24  
ntp2: 152.2.21.1  

ntp_cfg_and_validation.yml

---
- name: Idempotent NTP configuration validation Example
  hosts: cisco
  vars:
    cfg_ntp:
          - ntp server {{ ntp1 }}
          - ntp server {{ ntp2 }}
  tasks:
    - name: 1. Check Connectivity
      ios_facts:
        provider: "{{ creds_core_ssh }}"

    - name: 2. Configuration Changes
      ios_config:
        provider: "{{ creds_core_ssh }}"
        lines: "{{ cfg_ntp }}"

As of right now our two cisco ios targets are configured with the following two variables:
ntp1: 130.126.24.24
ntp2: 152.2.21.1

The problem here happens when we decide it is time (pun intended) to update our our NTP servers. If we change the values in our group vars file containing these variables and run our playbook there is nothing that actually removes the old ones.

If we ngrep what is currently configured we can see the following:
$ netmiko-grep 'ntp' cisco
pynet_rtr1.txt:ntp server 130.126.24.24
pynet_rtr1.txt:ntp server 152.2.21.1
pynet_rtr2.txt:ntp server 130.126.24.24
pynet_rtr2.txt:ntp server 152.2.21.1

Let us change the values of our NTP variables in our all.yml file as per below and then run our playbook:
ntp1: 130.111.111.24
ntp2: 152.111.111.1

But that looks kind of alright does it not? Let's verify how our ansible targets look right now:
$ netmiko-grep 'ntp' cisco

pynet\_rtr1.txt:ntp server 130.126.24.24
pynet\_rtr1.txt:ntp server 152.2.21.1
pynet\_rtr1.txt:ntp server 130.111.111.24
pynet\_rtr1.txt:ntp server 152.111.111.1

pynet\_rtr2.txt:ntp server 130.126.24.24
pynet\_rtr2.txt:ntp server 152.2.21.1
pynet\_rtr2.txt:ntp server 130.111.111.24
pynet\_rtr2.txt:ntp server 152.111.111.1

So this is the problem we're facing. We need to make sure that the targets are only configured with what we want, no remains from earlier configurations should exist for this specific playbook.

Part 2: Working towards solving our problem with validation and ensuring there is logic processing declarative intent.

To solve this problem we will create an additional 5 tasks:
3. Examining the configuration changes and registering them as a variable
4. Creating a usable variable of what is currently configured
5. Processing the currently configured values with our desired values
6. Printing a debug msg for when we have no extra ntp servers configured
7. Removing undesired configuration lines

Now we just need to keep on building our playbook according to what has been stipulated above.


3. Examining the configuration changes and registering them as a variable

The first task we create simply registers the current configuration (our "bonus" NTP lines) as a variable we can continue to work with. The output of this one is not pretty so we will have to fix that in our next task.

---
    - name: 3. Examining the configuration changes and registering them as a variable
      ios_command:
        provider: "{{ creds_core_ssh }}"
        commands: "show run | inc ntp"
      register: ntp_servers

Debug example of ntp_servers

    "msg": {
        "changed": false,
        "stdout": [
            "ntp server 130.126.24.24\nntp server 152.2.21.1\nntp server 130.111.111.24\nntp server 152.111.111.1"
        ],
        "stdout_lines": [
            [
                "ntp server 130.126.24.24",
                "ntp server 152.2.21.1",
                "ntp server 130.111.111.24",
                "ntp server 152.111.111.1"
            ]
        ]
    }

4. Creating a usable variable of what is currently configured
We are going to use the set_fact module to dig deeper into the data structure we registered as ntp_servers to work more easily with the data within. We are going to work with the list stdout\lines which contains another list at index 0.

The contents of this we put into the variable ntp_servers_proc with the following line ntp_servers_proc: "{{ ntp_servers.stdout_lines[0] }}".

---
    - name: 4. Creating a usable variable of what is currently configured
      set_fact:
        ntp_servers_proc: "{{ ntp_servers.stdout_lines[0] }}"

Debug example of ntp_servers.stdout_lines[0]

    "msg": [
        "ntp server 130.126.24.24",
        "ntp server 152.2.21.1",
        "ntp server 130.111.111.24",
        "ntp server 152.111.111.1"
    ]

So where are we at now?
We can push changes to our target devices, we have been setting up some processing to actually gather the data relevant to our NTP server play, when then dug into it and created a variable where the data has been made more accessible in the form of a list containing all present configuration lines pertaining NTP servers.

We still have to process the data and work with it to ensure that only what we want to be configured is actually configured after the playbook has executed.

Important
Before continuing, let us change back the values of our NTP variables in our all.yml file as per below:
ntp1: 130.126.24.24
ntp2: 152.2.21.1


5. Processing the currently configured values with our desired values

It is time to work with jinja list and set theory filters again. For reference look at Networking with Ansible 106 or List & Set Theory Filters - Ansible Documentation.

What we want to achieve here is to have a list returned to us containing items in ntp_servers_proc (the actual configuration) that does not exist in cfg_ntp (our declared configuration).

Basically a perfect job for set operations, difference (unique items in x not in y)

We use the set_fact module to have this list returned with the name: extra_ntp_srvs. First we pipe our current list to the difference-filter and the result is then piped/registered into a list, extra_ntp_srvs.

---
    - name: 5. Processing the currently configured values with our desired values
      set_fact:
        extra_ntp_srvs: "{{ ntp_servers_proc | difference(cfg_ntp) | list }}"

In the below debug msg of extra_ntp_srvs we can see that the variable contains ntp servers NOT declared (anymoore) by our group_vars/all.yml file.

ok: [pynet-rtr1] => {  
    "msg": [
        "ntp server 130.111.111.24",
        "ntp server 152.111.111.1"
    ]
}
ok: [pynet-rtr2] => {  
    "msg": [
        "ntp server 130.111.111.24",
        "ntp server 152.111.111.1"
    ]
}

Ok, so we now have a list of configuration lines that should not be present in our configuration anymore. Great stuff!


6. Printing a debug msg for when we have no extra ntp servers configured
Below is just a human readable way of patting ourselves on the back. When the "problem" list is empty it simply prints out a line on the terminal and tells us so.

---
    - name: 6. Printing a debug msg for when we have no extra ntp servers configured
      debug:
        msg: NTP Servers are correct
      when: extra_ntp_srvs == []

7. If there are extra NTP servers configured this task this will remove them Oh, we are finally here! The task to rule them all and help us sleep at night. Ok what are we doing here then?

We're using the ios_config ansible core module have it loop through our list of undesired NTP servers, prepending every line in that list with no which in cisco ios syntax means we remove the line from our cisco ios configuration.

---
    - name: 7. If there are extra NTP servers configured this task this will remove them
      ios_config:
        provider: "{{ creds_core_ssh }}"
        lines: "no {{ item }}"
      with_items: "{{ extra_ntp_srvs }}"

Part 3: Basking in the lights of Testing

Our first proper run

TASK [6. Printing a debug msg for when we have no extra ntp servers configured] ************************************************************************  
skipping: [pynet-rtr1]  
skipping: [pynet-rtr2]  

Since we know we are NOT OK, task 6 will be skipped.

TASK [7. If there are extra NTP servers configured this task this will remove them] **********************************************************************  
changed: [pynet-rtr2] => (item=ntp server 130.111.111.24)  
changed: [pynet-rtr1] => (item=ntp server 130.111.111.24)  
changed: [pynet-rtr2] => (item=ntp server 152.111.111.1)  
changed: [pynet-rtr1] => (item=ntp server 152.111.111.1)

PLAY RECAP **********************************************************************  
pynet-rtr1                 : ok=7    changed=1    unreachable=0    failed=0  
pynet-rtr2                 : ok=7    changed=1    unreachable=0    failed=0  

Our second proper run

TASK [6. Printing a debug msg for when we have no extra ntp servers configured] **********************************************************************  
ok: [pynet-rtr1] => {  
    "msg": "NTP Servers are correct"
}
ok: [pynet-rtr2] => {  
    "msg": "NTP Servers are correct"
}

Oh, would you look at that? Since all was great this time around no items were in our "problem list" and it just exited our play since since task 7 never needed to be processed. That is how we build validation and ensuring that our declarative intentions are being followed by our little skynet+anisble hybrid creation.