Need for Speed: Network Automation
The best kind of job is where your intellect is constantly challenged to provide simple solutions to complex problems. One such instance was when I had to push CLI commands to configure over 500 devices in the production environment, for a financial services client, with near-zero downtime. Logging in to each device one by one using an SSH/Telnet terminal would’ve taken ages so automating the entire process chain using a Python script was the only way to solve this!
This required two major tasks that had to be performed — pushing the CLI to the device using the netmiko library and exploring multi-threading using the concurrent.futures library.
Netmiko vs Paramiko
Both these libraries provide the same functions however, Netmiko can support many more platforms and is significantly faster in processing in comparison to paramiko. Generally, paramiko can be used for simple SSH/Telnet to PCs or servers, and netmiko is more for executing commands and configurations on network devices. I used Python 3.7 since the higher versions are not compatible with these libraries at the moment.
It is quite simple to establish a connection to the required devices using Netmiko. The connection is established through a ConnectHandler where you can pass the following parameters based on the requirement:
If you have more than one IP, you can simply iterate over a list or a file that contains all your IPs and store all the information in a list of dictionaries.
Using ConnectHandler, the connection can be established using the enable module. The send_command_timing module is used to push the command string to the device. The output can be stored in a variable as below.
strip_prompt and strip_command are cosmetic parameters used to remove the router prompt and remove the command string respectively from the output.
This brings us to our second and most important task: establishing parallel connections and running the command simultaneously on multiple devices. I was able to achieve this task using the ThreadPoolExecutor module of the concurrent.futures library.
ThreadPoolExecutor vs ProcessPoolExecutor
All tasks can be classified as either I/O intensive or CPU intensive. A process refers to any program in execution and a thread is a segment of a process; a process can have multiple threads. If your task is I/O intensive (performs input/output operations), you can use the ThreadPoolExecutor, and for CPU-bound tasks, using ProcessPoolExecutor. In my case, using the ThreadPoolExecutor gave me the desired results.
You can monitor your CPU and Memory using Task Manager. If your CPU shoots up while the code is running, it is a CPU-intensive task and if your Memory shoots up, it is an I/O bound task.
Using the following code snip, the library can be used. The task_function is what contains the code to establish SSH connectivity and execute the command string on the device. The host_list is the list of dictionaries containing login information for all the IP addresses (passed as an argument to the task_function).
The two python libraries mentioned have several functionalities and I have just discussed a few of the features I found useful! Like concurrent.futures, multiprocessing is another python module that can be leveraged for parallel execution of tasks. I haven’t had the chance to explore it yet but if anyone worked with this module, I’m happy to hear your thoughts! 😊
