August 10, 2017 · network automation ansible napalm merge replace facts

Networking with Ansible 105

1.1 NAPALM Ping

We start off with creating a new working folder, then we create our group_vars folder that we will be using for this post.

workingdir$ tree  
├── group_vars
│   ├── arista.yml
│   ├── cisco.yml
│   └── juniper.yml
├── napalm_ping_1.yml
└── napalm_ping.yml

arista.yml should contain the following:

creds:  
  hostname: "{{ ansible_host }}"
  username: "{{ username }}"
  password: "{{ password }}"
  dev_os: "eos"
  optional_args: {}

cisco.yml should contain the following:

creds:  
  hostname: "{{ ansible_host }}"
  username: "{{ username }}"
  password: "{{ password }}"
  dev_os: "ios"

creds_secret:  
  hostname: "{{ ansible_host }}"
  username: test9
  password: "{{ password }}"
  dev_os: "ios"
  optional_args:
    secret: "{{ password }}"

cisco.yml should contain the following:

---
creds:  
  hostname: "{{ ansible_host }}"
  username: "{{ username }}"
  password: "{{ password }}"
  dev_os: "junos"
  optional_args: {}

The onyl real difference here is the dev_os: "ios"/"eos"/"junos" lines which is how NAPALM handles the targeting of different vendor devices.

napalm_ping.yml

We write a playbook that will be using the napalm-ansible module :

- name: NAPALM Ping napalm_ansible module
  hosts: cisco:arista:juniper
  tasks:
    - name: NAPALM ping
      napalm_ping:
        provider: "{{ creds }}"
        destination: 8.8.8.8

The destination argument is the target for our PING we will be running. We then test our playbook with the verbose flag set to -vv.

ansible-playbook napalm_ping.yml -vv

The main takeaway here is the ease of use working across infrastructure consisting of multiple vendor devices. Remember how the ansible networking modules have a 1-1 relationship when it came to a playbook using them in tasks. To ping both ios, eos and junos devices we would need a playbook with 1 task for the ios devices, another for the eos devices and a third for the junos device.

NAPALM abstracts this necessity away, notice that we are only using one task to run the playbook targeting the ios, eos and junos devices.

1.2 NAPALM Getters

We expand upon our group_vars and add a nxos.yml file containing:

nxos.yml

---
creds:  
  hostname: "{{ ansible_host }}"
  username: "{{ username }}"
  password: "{{ password }}"
  dev_os: "nxos"
  optional_args:
    port: 8443

napalm_get.yml

This will be our initial playbook for the napalm_ansible get module.

---
- name: NAPALM GETTERS (get_facts)
  hosts: cisco:juniper:arista:nxos
  tasks:
    - name: NAPALM FACTS
      napalm_get_facts:
        provider: "{{ creds }}"
        filter: "facts,interfaces,config"
    - debug:
        var: "napalm_os_version,napalm_fqdn,napalm_model,napalm_serial_number"

After running this (not including the verbose output, since it would be a bit too much information too handle...) we get the debug variables printed to our terminal screen:

In this case we're just printing an inventory listing of our devices with hostname, serial no and uptime included.

By using the filter key we can get a bit more granular with what facts we are collecting and these arguments correlates to the NAPALM documentation: Getters support matrix where we simply delete the prefixed get_\ from, for example get_arp_table to only have facts about the current state of the arp table collected.

Let's try to see if we can find more data about our devices

napalm_get_all_the_things.yml example

We can get even more data out, let's create a new playbook containing:

---
- name: THE NAPALM - GET ALL THE THINGS Example.
  hosts: cisco:arista
  tasks:
    - name: ARP
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "arp_table,mac_address_table"
    - name: BGP
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "bgp_config,bgp_neighbors,bgp_neighbors_detail"
    - name: Config
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "config,environment,facts"
    - name: Interfaces
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "interfaces,interfaces_counters,interfaces_ip"
    - name: LLDP
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "lldp_neighbors,lldp_neighbors_detail"
    - name: Network Instances
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "network_instances"
    - name: NTP
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "ntp_servers,ntp_stats"
    - name: Users
      napalm_get_facts: 
        provider: "{{ creds }}"
        filter: "users"

Above is an example on how to create a structure where the tasks are operationally/functionally grouped. As we can see there exists endless possibilities of getting information, and since we are getting the data as a standardised, vendor agnostic output we have many means of further processing this data. One example could be to use these filters in running validate post-change validation tasks. Validation is something we will examine more closely in the next section.

1.3 NAPALM Validate

The primary use case for the NAPALM validate method would be to if we wanted to validate operational data, think of it as a way of ensuring that the configuration intention leads to a operationally working environment. The validation we want to see performed can be stated and specified in a NAPALM-specific .yml file.

The data for achieving state inspection we get from the NAPALM getter methods that we looked at in the previous section and with this data we can build out validation reports that is relevant to our infrastructure.

More documentation about NAPALM Validate can be found at https://napalm.readthedocs.io/en/latest/validate/index.html and there is an additional example for napalm-ansible at https://github.com/napalm-automation/napalm-ansible.

In group_vars/ we declare the following variables:

group_vars/cisco.yml validate_file: "validate_ios.yml"
group_vars/arista.yml: validate_file: "validate_eos.yml"

Then we create the files in our playbook directory and enter the following:

for validate_eos.yml:

---
- get_facts:
    os_version: "4.15.4F-2923910.4154F"
    model: vEOS
    vendor: Arista
    interface_list:
      - Ethernet1
      - Ethernet2
      - Ethernet3
      - Ethernet4
      - Ethernet5
      - Ethernet6
      - Ethernet7
      - Management1
      - Vlan1

- get_arp_table:
    list:
        - ip: 10.220.88.1

and for validate_ios.yml:

---
- get_facts:
    model: "881"
    os_version: "Version 15.4.2.T1"
    vendor: Cisco
    serial_number: FTX1512038X
    hostname: pynet-rtr1

- get_arp_table:
    list:
        - ip: 10.220.88.1

So at first we're using the get_facts method to make sure that the devices are what they should be and for the arista switch, that it contains the interfaces we have specified.

Then for both validation files we want to make sure that our arp table contains an entry that has an IP of 10.220.88.1.

Let us build out playbook for this validation scenario:

---
- name: NAPALM Validation
  hosts: pynet-sw6:pynet-rtr1
  tasks:
    - name: NAPALM validate (IOS and EOS)
      napalm_validate:
        provider: "{{ creds }}"
        validation_file: "{{ validate_file }}"

The playbook in itself is quite simple, which makes since the validation specification is decoupled from the playbook into the file we call via group_vars. After running it we can see that everything checks out OK.

Now let's make up an IP (130.130.88.1) and replace the ip in our validate_ios.yml file and have the cisco IOS validation fail and we will extract and look at the json formatted return data:

While our pynet-sw6 was successfully validated in regards to our policy, pynet-rt1 was not since get_arp_table did not contain an entry with the ip 130.130.88.1 but as we can see we are actually getting the return code of true for the the hostname, model, osversion, serialnumber and vendor.

Policies are hard and the NAPALM validation is strict.

1.4 NAPALM Merge

The NAPALM merge method is one of the abstracted configuration methods that napalm-ansible will be using to give us greater control over the change process.

A merge uses a data source which will be merged into the full (existing) configuration. We will look at how to deal with the replace (all configuration) at a later section.

Underneath for an IOS device a merge operation will use SCP to transfer a file named merge_config.txt to flash:/ and upon us running a commit it will be executing the following CLI command: copy flash:/mergeconfig.txt system:/running-config. The dope thing about NAPALM is we really don't have to know, but think of the description as something extra to treat ourselves with.

Setting up our working environment

Folder and file structure:

    $ tree
    .
    ├── CFGS
    │   ├── eos-vlans.txt
    │   ├── nxos1-merge.txt
    │   ├── nxos-vlans.txt
    │   ├── pynet-rtr1-merge.txt
    │   ├── pynet-sw5-merge.txt
    │   └── srx1-merge.txt
    ├── DIFFS
    │   ├── nxos1.txt
    │   ├── nxos2.txt
    │   ├── pynet-rtr1.txt
    │   ├── pynet-sw5.txt
    │   ├── pynet-sw6.txt
    │   ├── pynet-sw7.txt
    │   ├── pynet-sw8.txt
    │   └── srx1.txt
    ├── group_vars
    │   ├── arista.yml
    │   ├── cisco.yml
    │   ├── juniper.yml
    │   └── nxos.yml
    └── napalm_merge.yml

Playbook setup (napalm_merge.yml):

---
- name: Test napalm merge
  hosts: pynet-sw5
  tasks:
    - napalm_install_config:
        provider: "{{ creds }}"
        config_file: "CFGS/{{ inventory_hostname }}-merge.txt"
        commit_changes: False
        replace_config: False
        diff_file: "DIFFS/{{ inventory_hostname }}.txt"

Ok, so we have some new configuration items to discuss:

config_file: "CFGS/{{ inventory_hostname }}-merge.txt"
The configuration file we will load from our CFGS directory

commit_changes: False
This line means that we will not commit our changes neither merge nor replace operations. Think of it as having a test run performed where the only change really will be that we locally create our diff file.

replace_config: False
Having this set to False will enable us to perform a merge operation instead of a replace operation

diff_file: "DIFFS/{{ inventory_hostname }}.txt" is where the diff files will be created upon running our playbook.

Making a test run (only creating a diff file)

We enter the folloing in CFGS/pynet-sw5-merge.txt:

logging buffered   50000

Then we run our playbook:

$ ansible-playbook napalm_merge.yml -vv

Due to how we have setup our playbook, no actual configuration merge has been performed, however, we should be able too locate a newly created diff file under DIFFS/:

$ cat DIFFS/pynet-sw5.txt

@@ -3,6 +3,8 @@
! boot system flash:/vEOS-lab.swi
!
transceiver qsfp default-mode 4x10G  
+!
+logging buffered 50000
!
hostname pynet-sw5  

The logging buffered 50000 configuration line does not yet exist on our switch, let us change the commit_changes value to True and run our playbook again.

$ ansible-playbook napalm\_merge.yml -vv

The sad part is that it actually looks identical to our previous run, with one big exception: It now merged our configuration line with the running-configuration.

$ netmiko-grep 'logging' pynet_sw5
logging buffered 50000

If we were to do a new run we would discover the idempotent nature of NAPALM.

It does not even create our DIFF since what we are requesting is already present on the target device.

Targetting multiple systems

We start off with creating some new *merge.txt-files as per below:

nxos1-merge.txt

logging history size 400

pynet-rtr1-merge.txt

logging buffered 50000

pynet-sw5-merge.txt

logging buffered 50000

srx1-merge.txt

system {
   syslog {
         archive size 240k files 3;
  }
}

Then we change our napalm_merge.yml playbook file

  1 ---
  2
  3 - name: Test napalm merge
  4   hosts: pynet-sw5:nxos1:pynet-rtr1:srx1
  5   tasks:
  6     - napalm_install_config:
  7         provider: "{{ creds }}"
  8         config_file: "CFGS/{{ inventory_hostname }}-merge.txt"
  9         commit_changes: False
 10         replace_config: False
 11         diff_file: "DIFFS/{{ inventory_hostname }}.txt"

Here we change two lines, we add additional devices to test towards and we are setting commit_changes to False again.

First we just want to see that we are getting our DIFFS created properly after running our playbook.

Ok, as we can see we only have two devices that are not configured correctly and subsequently we now have two diff files that actually contains configuration in our DIFFS/ directory.

$ tree DIFFS -s
DIFFS
├── [          0]  nxos1.txt
├── [         23]  pynet-rtr1.txt
├── [          0]  pynet-sw5.txt
└── [         84]  srx1.txt 

Setting commit\_changes to True again and running our playbook would merge the configuration for our pynet-rtr1 and srx1 devices. Running the playbook with -vvv gives us some extra information and an excerpt from this for the pynet-rtr1 can be seen below.

changed: [pynet-rtr1] => {  
    "changed": true,
    "invocation": {
        "module_args": {
            "archive_file": null,
            "commit_changes": true,
            "config": null,
            "config_file": "CFGS/pynet-rtr1-merge.txt",
            "dev_os": "ios",
            "diff_file": "DIFFS/pynet-rtr1.txt",
            "get_diffs": true,
            "hostname": "hostname.domain.url",
            "optional_args": null,
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "provider": {
                "dev_os": "ios",
                "hostname": "hostname.domain.url",
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "username": "username"
            },
            "replace_config": false,
            "timeout": 60,
            "username": "username"
        }
    },
    "msg": "+logging buffered 50000"
}
Targeting vendor device groups

So far we have been targeting singular devices and now we will just go through a quick example on how to setup vlans across different vendors with NAPALM.

eos-vlans.txt

vlan 300
  name blue0
vlan 301
  name blue1
vlan 302
  name blue2
vlan 303
  name blue3
vlan 304
  name blue4

nxos-vlans

vlan 300
  name blue0
vlan 301
  name blue1
vlan 302
  name blue2
vlan 303
  name blue3
vlan 304
  name blue4

We then have to change our playbook slightly since we are now using a different type of source file.

---
- name: VLANS configuration merge for Nexus and Arista switches.
  hosts: arista:nxos
  tasks:
    - napalm_install_config:
        provider: "{{ creds }}"
        config_file: "CFGS/{{ creds['dev_os'] }}-vlans.txt"
        commit_changes: True
        replace_config: False
        diff_file: "DIFFS/{{ inventory_hostname }}.txt"

To start with we are stating the configuration file to be called upon using our variable dev_os which we have set previously in our group_vars (refer to a previous section if you need a reminder) and we're also suffixing the source filename with -vlans.txt.

This should result in that both nxos-vlans.txt and eos-vlans.txt will be used as source files for this play.

Let us run this playbook and see what happens:

Amd we can see that two of these devices already had the desired vlans configured, but as for the rest, they really had their coonfiguration handed to them.

The shorthand verison of what is being done here is basically that we are staging the configuration (CFGS/) we then use NAPALM to generate a diff file for us which we can verify.

After the verification has been approved (best of worlds) we can then commit our changes to the device (with the possibility of doing a rollback since that configuration file is created automatically for us)

And finally we have done this without needing to resort to using conditional logic in order for us to target different vendor products since that necessity has been abstracted away for us by the NAPALM developers.

1.5 NAPALM Replace

Setting up our environment

We start with making sure our files and folders are setup correctly in our working directory.

napalm_replace/  
├── CFGS
│   ├── nxos1.txt
│   ├── pynet-rtr1.txt
│   ├── pynet-sw5.txt
│   └── srx1.txt
├── DIFFS
├── group_vars
│   ├── arista.yml
│   ├── cisco.yml
│   ├── juniper.yml
│   └── nxos.yml
└── napalm_replace.yml

In the previous section pertaining the NAPALM Merge method we were staging our merges with pieces of configuration, for the replace method we are using full configurations that we are replacing. To extract the full configurations do not simply copy the configuration from show commands, use SCP and grab it off the device.

Adding juniper.yml to our group_vars

We expand upon our group_vars and create a juniper.yml file containing:

---
creds:  
  hostname: "{{ ansible_host }}"
  username: "{{ username }}"
  password: "{{ password }}"
  dev_os: "junos"
  optional_args: {}

Setting up the playbook

---
- name: NAPALM REPLACE METHOD
  hosts: pynet-rtr1
  tasks:
    - napalm_install_config:
        provider: "{{ creds }}"
        config_file: "CFGS/{{ inventory_hostname }}.txt"
        commit_changes: False
        replace_config: True
        diff_file: "DIFFS/{{ inventory_hostname }}.txt"

Since we are just handling full configurations we simply call upon them by inventory_hostname variable and append .txt to it. We are not commiting changes at this stage, hence commit_changes set to False, also the replace_config: key has been set to true since this deals with NAPALM replace method.

Test run (Generating some diffs)

$ ansible-playbook napalm_replace.yml -vvv

Ok, we can see that NAPALM picked up on some stuff that needs to be changed.

$ tree DIFFS -s
DIFFS
├── [        485]  nxos1.txt
├── [        558]  pynet-rtr1.txt
├── [        914]  pynet-sw5.txt
└── [         83]  srx1.txt

Let's look at an example of what these diffs can contain:

cat DIFFS/nxos1.txt
no interface loopback0
interface Ethernet2/3
  no ip address 10.99.99.1/24
  exit
interface Ethernet2/2
  no ip address 10.10.100.1/24
  exit
interface Ethernet2/1
  no ipv6 address 2001::db8:1:200c:1/64
  no ip address 10.10.10.1/24
  exit
no vlan 304
no vlan 303
no vlan 302
no vlan 301
no vlan 300

The above section is what what needs to be performed in order for the configuration of nxos1 to be compliant with what we say (via CFGS/nxos1.txt) the configuration should look like.

So we now have quite excellent materials to bring to our lab change control meeting discussions instead of someone just saying "I am gonna be doing some stuff with the network come this weekend", (this change was approved by the way...).

Running our playbook with COMMIT set to TRUE

We change commit_changes: to True and execute our playbook again:

And since we're big fans of idempotence we run it again:

Look at the filesizes on thos DIFF files!

tree DIFFS -s
DIFFS
├── [          0]  nxos1.txt
├── [          0]  pynet-rtr1.txt
├── [          0]  pynet-sw5.txt
└── [          0]  srx1.txt

We're still a long way from turning this lab into skynet, but let's just be happy that the idempotent behaviour of NAPALM makes repeated runs create predictable returns. No configuration was messed up, nothing that shouldn't happen, happened.

These configuration replacements used different transport protocols in order to achieve the state we specified, we just used NXAPI, eAPI, NETCONF and SSH to accomplish these configuration operations.

Could we just debunk the myth that network engineers will be unemployable if they don't learn code? It would seem choosing a tool and being able to read YAML can get us quite a long way in the world of network automation and still be rather employable.