If debugging is the process of removing software bugs, then programming must be the process of putting them in. - Edsger Dijkstra

Challenges With Scripting

Irrespective of our role in software development we often work with Linux/Unix(*nix). While working with command line(CLI) on *nix we need to write shell scripts. We automate something with scripts or we do number crunching on some data using shell commands. Most of the time we already know frequently used commands else we google it. But it becomes tedious if the task is long or complex and it becomes more difficult to find appropriate command or combine more than two commands.

Now writing scripts are not that easy as using one or two commands with some regex and pipes. It becomes painful and we find some workarounds or we give up. But either way it is time as well as mind consuming task so we tend avoid it because it kills your actual working time. Unless you are full time command line script-er.

Sometimes you take some of our time out and finish writing that script and feel really proud of yourself (which you should be). But after a month or so when you visit that same script and try to figure out a way tweaking it for new/changed requirement and you find yourself completely lost in the process and because it becomes really hard to make sense out of your code (after visiting it after long time) when it comes to scripting. Now one may argue that it is not only limited to scripting but also happens with other programming task too. True! but I will say with scripting it is more likely to happen and the reason is that with scripting a process of building abstraction is complex as compared to other fancy programming languages and hence we often end up writing context sensitive code when it comes to scripting.

So lets take a look at how using python helps to overcome some of this problems.

Why python is good choice for scripting?

  • Python being a very high level language it has features similar to all other typical high level programming languages.

Python handles a lot of complexity for you and syntax are more user friendly than other classic scripting languages.

  • Python also provide a nice REPL (interpreter) which becomes handy for trial and error strategy.

  • Rich set of library support and huge developer community for question and answers on IRC, mailing list and stack-overflow.

  • Libraries written in python to interact with OS are the wrappers around OS specific library. So in this way those library are generic.

So you can easily write python script on one specific OS and can use same on another OS without any changes.

  • To write script using python you don’t have to have full fledged IDE, you just need an editor (vim or any other) and a python binary.

So I can say it is this much lightweight in this sense.

  • And one of the most important reason which I believe to choose python for scripting over classic scripting languages is that

python gives you a better programming environment than shell. Meaning, you call pull out data by firing shell commands through python code and bring that output to better programming arena and you can mold that data, extract some pattern from data, compute something on that data, write it to another file, send that data over the network with much easiness with python code. All of this becomes easy because python provides features of typical programming language and the process of building abstraction becomes much more easier and that results into keeping your code more readable, maintainable and extendable.

Let me show you short example.

Lets assume we have one file which has some text containing IP address (similar to output of ‘ip addr’ command). Now the task is first extract all the IP addresses from the file and then check one by one that IP address is up or down. Task may sound simple but I urge you to take a pause here and try to write script for the same using shell commands or at least just try to think that how it could be done.

file: sample_text_file.txt

~]# ip address add dev eth1
~]# ip address add dev eth0
~]# ip addr
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:fb:77:9e brd ff:ff:ff:ff:ff:ff
    inet scope global eth1
    inet scope global eth1


Now lets do this with python script.

import sys
import os
import re

# function to extract all ip address from a file.
def get_all_ip_from_file(file_path):

    file_data = open(file_path, "r").readlines()
    ip_address_list = [] # List to store ip address
    for line in data:
        ips_on_line = re.findall( r'[0-9]+(?:\.[0-9]+){3}', line ) 
        ip_address_list = ip_adress_list + ips_on_line
    return ip_address_list

# function to check if the ip is Up or Down
def check_if_ip_up_or_down(ip):

    response = os.system("ping -c 1 " + ip) 
    if response == 0:
        print ip, 'is up!'
        return True
        print ip, 'is down!'
        return False

def main():
    ip_list = get_all_ip_from_file('sample_text_file.txt')
    for ip in ip_list:
        status = check_if_ip_up_or_down(ip)
        print ip + " --> "+ str(status)


I hope you get the point with this little demo. Please feel free to put your comment/opinion/critisim in comment section down below. I will be happy to discuss more. Happy coding!!!