Network Device Configuration Standardization with Trigger

#Network Device Configuration Standardization with Trigger

Presented by John Marrett <br>
Director of Managed Service  <br>
Stack8 Technologies <br>

.s8-logo-box-title[
  <a href="http://www.stack8.com/en/products-services/managed-services/"><img width=50% src="images/Stack8-Logo-white-PNG-trans.png"></a>
]

???

- How would you like to be able to reconfigure 1000 sites in an hour vs 20 days of manual work?
- This is what I'm going to show you in this presentation
- Started with a Migration project for 1000 sites, each site in ten minutes
- Manage Increasing numbers of devices, improvded consistency

---

.right-column[
- About Me
- About the Networking Industry
- Network Configuration Management Approach
- Adapting to your Environment
- Improvements
- Questions
]

???

- Audience - Networking Professional with some Programming Ability

---

.right-column[
- Programming background 
- Transitioned into system and network administration 
- Moved into managed service environment 
]

---

.left-column[
<img src="images/gear74.png">
]
.right-column[
- Director of Managed Services at Stack8
- Network and Security work 
- Large Managed Service environments 
- Manage approximately 2600 CPE routers in a number of different environments 
]

.s8-logo-box[
  <a href="http://www.stack8.com/en/products-services/managed-services/"><img width=50% src="images/Stack8-Logo-white-PNG-trans.png"></a>
]

---
.right[
#State of our Industry
]
.left-column[
<img src="images/computers3.png">
]
.right-column[
- It sucks 
- Lots of manual processes 
- Many vendor proprietary tools  
- Expensive commercial tools 
- Lack of evolution in networking 
]

---

.left-column[
<img src="images/server30.png">
]
.right-column[
- Identify issues or ongoing requirements 
- Generate reports to assess scope and impact  
- Apply configuration changes 
]

---

.right[
# Approaches I have Tried
]
.left-column[
<img src="images/delete96.png">
]
.right-column[
- RANCID clogin scripts (TCL) 
- Perl generated TCL, logic in Perl 
- Cut back on (ab)using Perl; switched to Python 
- Discovered Trigger
]

---
.right[
# Steps to Network Configuration
]

.left-column[
<img src="images/list4.png">
]
.right-column[
- Approach 
- Implementation - Normalization 
- Implementation - Reporting 
- Customizing to your Environment 
- Improvements 
]

---

.right[
# Approach
]
.left-column[
<img src="images/numbered5.png">
]
.right-column[
- Collect configuration and device state 
- Parse information and structure it 
- Analyze parsed information 
- Make changes to the devices 
- Store device information 
- Generate reports 
]

---
.right[
#Structure of Example Code
]
.left-column[
<img src="images/data45.png">
]
.right-column[
- .blue-text[router.py] - common code for interacting with Cisco Routers 
- .blue-text[normalize.py] - normalization script 
- .blue-text[report.py] - reporting script 
- .blue-text[test-units.csv] - device list 
]

---
.right[
#Implementation - Normalization
]
Normalization Output
```
johnf@pstanadm1:~/TriggerOpen$ ./Normalize.py
Processing all sites
Are you certain you want to normalize 3 devices? [y/N]
y
Processing 3 devices (r1 r2 r3)
Failed to ping host r1
Validating router details
r2: Will normalized trigger-test acl on device
Normalizing 1 devices (r2)
Device r2: Configuration Saved
```
This is the ACL that the code validates
```
ip access-list standard trigger-test-1
  permit 1.1.1.1
```
---
.right[
# Core Callback Processing
]
.blue-text[Normalize.py]
```python
d = getRouterDetails(device_list).run()
d.addCallback(validateRouterDetails)
d.addCallback(initiateRouterNormalization)
```
---
.right[
#Getting Router Details
]
.blue-text[Normalize.py]
```python
class getRouterDetails(ReactorlessCommando):
    # Commands to be executed
    commands = Router.showCommands
    # Selection of Available devices
    def select_next_device(self, jobs=None):
        # If I ping, return me.
        if next_device.is_reachable():
            log.msg('PING [SUCCESS]: %s' % next_device)
            return next_device

```
.blue-text[Router.py]
```python
class Router(object):
    showCommands = ['show run | i ip access-list', 'show ver']
```
---
.right[
# Parsing Details
]
.blue-text[Normalize.py]

```python
def validateRouterDetails(result):
# [...]
    for device, results in result.items():
        routers[device].validate()
```
.blue-text[Router.py]
```python
    def validate(self):
        self.lastAccess = datetime.now()
        self.validateAcl()
    def validateAcl(self):
        self.acls = []
        for line in self.results["show run | i ip access-list"].splitlines():
            line = line.strip()
            # ip access-list standard trigger-test-1
            # ip access-list extended outside-in
            m = re.search("ip access-list \S+ (\S+)", line)
            if m is not None:
                self.acls.append(m.group(1))
```
---
.right[
# Normalizing Configuration
]
.blue-text[Normalize.py]
```python
def validateRouterDetails(result):
# [...]
    for device, results in result.items():
        routers[device].normalize()
        if routers[device].normalizeRequired:
            devicesToCorrect.append(device)
```
.blue-text[Router.py]
```python
    def normalize(self):
        self.commands = []
        self.normalizeRequired = False
        if "trigger-test-1" not in self.acls:
            self.normalizeRequired = True
            self.commands += [
                "ip access-list standard trigger-test-1",
                "permit 1.1.1.1"
            ]
```
---
.right[
# Executing Normalization
]
.blue-text[Normalize.py]
```python
def initiateRouterNormalization(devices):
    if devices is not None:
        devicesToNormalize = []
        for device in devices:
            if routers[device].normalizeRequired:
                devicesToNormalize.append(device)
        return normalizeRouters(devicesToNormalize).run()

class normalizeRouters(ReactorlessCommando):
    def to_cisco(self, dev, commands=None, extra=None):
        dev_commands = routers[dev.nodeName].commands
        return dev_commands
```

---
.right[
# Validating Results
]
.blue-text[Normalize.py]
```python
    if d.result is not None:
        for device in d.result.keys():
            m = re.search(r"\[OK\]", d.result[device]["write mem"])
            if m is not None:
                print "Device {}: Configuration Saved".format(device)
            else:
                print "Device {}: Warning no [OK] in Output".format(device)
```

---
.right[
# Storing Router State Objects Results
]
.blue-text[Normalize.py]
```python
    stateFile = open("routerstate.json", "w")
    jsonpickle.set_encoder_options('json', sort_keys=True, indent=4)
    data = jsonpickle.encode(routers)
    stateFile.write(data)
```

---
.right[
# Implementation - Reporting
]
Reporting Output

```
    johnf@pstanadm1:~/TriggerOpen$ ./Report.py 
    Processing all sites
    Failed to ping host r1
    johnf@pstanadm1:~/TriggerOpen$ cat report.csv 
    Device,Last Access,Version
    r1,Never,Unknown
    r2,2015-08-20 09:00,15.2(4)M6
    r3,2015-08-20 09:00,15.2(4)M6
    johnf@pstanadm1:~/TriggerOpen$ ./Report.py r2
    johnf@pstanadm1:~/TriggerOpen$ cat report.csv 
    Device,Last Access,Version
    r1,Never,Unknown
    r2,2015-08-20 09:01,15.2(4)M6
    r3,2015-08-20 09:00,15.2(4)M6
```

---

.right[
# Reporting Process
]
.left-column[
<img src="images/file148.png">
]
.right-column[
- Load state information 
- Identify devices to update 
- Connect to devices   
- Update information 
- Generate report 
]

---

It's really almost exactly the same as the normalization process

.blue-text[Report.py]
```python
def main():
	# [...]
    d = getRouterDetails(device_list, timeout=120, max_conns=30).run()
    d.addCallback(validateRouterDetails)
    d.addBoth(stop_reactor)

def validateRouterDetails(result):
	# [...]
    for device, results in result.items():
        routers[device].results = results
        routers[device].validate()
```

---

.right[
# Output a CSV
]
```python
    with open("report.csv", "w") as f:
        fieldNames = ["Device", "Last Access", "Version"]
        writer = csv.DictWriter(f, fieldNames)
        writer.writeheader()
    
        for router in routers.itervalues():
            try:
                last_access = "{:%Y-%m-%d %H:%M}".format(router.lastAccess)
            except AttributeError:
                last_access = "Never"
            try:
                version = router.version
            except AttributeError:
                version = "Unknown"
            writer.writerow({
                "Device": router.name,
                "Last Access": last_access,
                "Version": version
                })
```

---

.right[
#Adapting to your Environment
]
.left-column[
<img src="images/home168.png">
]
.right-column[
- Build on this framework 
- Create your own specific tests and resolutions 
- Begin with analysis and small changes 
- Bring in other sources of information to drive changes 
]

---
.right[
# ACL Interface Parser
]

In .blue-text[Router.py] we add:

```python
class Router(object):
    showCommands = ['show run | i ip access-list', 'show ver',
        'show run | s ^interface']

def validate(self):
    [...]
    self.validateInterface()

def validateInterface(self):
        self.interfaces={}
        interface=None
        for line in self.results["show run | s ^interface"].splitlines():
            line = line.strip()
            m = re.search("^interface (\S+)", line)
            if m is not None:
                interface = m.group(1)
                self.interfaces[interface]={}
            m = re.search("^ip access-group (\S+) (\S+)", line)
            if m is not None:
                if m.group(2)=="out":
                    self.interfaces[interface]["acl out"]=m.group(1)
                if m.group(2)=="in":
                    self.interfaces[interface]["acl in"]=m.group(1)
        print self.interfaces
```

---
.right[
# ACL Interface Parser Output
]

Resulting in this output:
```
johnf@pstanadm1:~/TriggerOpen$ ./Report.py r2
{'Vlan10': {}, 'Cellular0': {}, 'Vlan1': {}, 'Tunnel801': {}, 'Tunnel800': {},
'GigabitEthernet0': {'acl in': 'test'}, 'Tunnel805': {}, 'Tunnel804': {},
'Dialer2': {}, 'Serial0': {}, 'Loopback0': {}, 'FastEthernet1': {},
'FastEthernet2': {}, 'FastEthernet3': {}, 'FastEthernet0': {}}

```

---

.right[
#Improvements
]
.left-column[
<img src="images/add83.png">
]
.right-column[
- Generic parsing of configurations (ciscoconfparse?)
- Domain Specific Language 
- Community
]

---

- My wife, children and family
- Jathan McCollum for creating Trigger as an open source project
- Stack8 for providing a great place to work and develop my skills
- Henrik Sørensen for suffering through my python (sadly he was unable to get me to write to his standards)
- Sharlene Dozois for turning this into an intelligible presentation
- \#trigger on freenode
- \#linux

.s8-logo-box[
  <a href="http://www.stack8.com/en/products-services/managed-services/"><img width=50% src="images/Stack8-Logo-white-PNG-trans.png"></a>
]

---

#Questions

---

- This Presentation - http://z-c.ca/trigger14x
- Example Code - http://z-c.ca/trigger14xcode
- Trigger Documentation - http://trigger.rtfd.org
- Trigger Project - https://github.com/trigger/trigger
- Learn Python the Hard Way - http://learnpythonthehardway.org/
- My Website - https://zioncluster.ca/
- My email - <a href="mailto:johnf@zioncluster.ca">johnf@zioncluster.ca</a>
- Stack8 Managed Services - http://www.stack8.com

.s8-logo-box[
  <a href="http://www.stack8.com/en/products-services/managed-services/"><img width=50% src="images/Stack8-Logo-PNG-transpatent.png"></a>
]

---

- <a href="http://www.freepik.com" title="Freepik">Freepik</a> from <a href="http://www.flaticon.com" title="Flaticon">www.flaticon.com</a> is licensed under <a href="http://creativecommons.org/licenses/by/3.0/" title="Creative Commons BY 3.0">CC BY 3.0</a>
- <a href="http://www.google.com" title="Google">Google</a> from <a href="http://www.flaticon.com" title="Flaticon">www.flaticon.com</a> is licensed under <a href="http://creativecommons.org/licenses/by/3.0/" title="Creative Commons BY 3.0">CC BY 3.0</a>
- <a href="http://www.amitjakhu.com" title="Amit Jakhu">Amit Jakhu</a> from <a href="http://www.flaticon.com" title="Flaticon">www.flaticon.com</a> is licensed under <a href="http://creativecommons.org/licenses/by/3.0/" title="Creative Commons BY 3.0">CC BY 3.0</a>
- <a href="http://icon-works.com" title="Icon Works">Icon Works</a> from <a href="http://www.flaticon.com" title="Flaticon">www.flaticon.com</a> is licensed under <a href="http://creativecommons.org/licenses/by/3.0/" title="Creative Commons BY 3.0">CC BY 3.0</a>