How to leverage command line options parsing modules in Python

Here in this article we will try to understand about command line options and arguments. We will also try to see how we can leverage python modules such as getopt, optparse and argparse to parse and process these options and arguments in python scripts.

Table of Contents

Test Environment

Python 3.12.7

What are Options and Arguments

Linux one of the popular operation system comes with an interactive terminal known as sh or bash more commonly. This terminal is a place where in the users can run commands or programs. Most of these programs that are provided by the system or custom programs, scripts, tools or utilities built by the administrator accept some form inputs. These inputs can be either options or arguments.

These options and arguments are distinct components that modify or provide data to a command’s execution.

Options (or Flags):

Options are used to modify the behavior of a command. They typically start with a hyphen (-) for short options (single letter) or two hyphens (–) for long options (full word).
Short options can often be combined (e.g., ls -la is equivalent to ls -l -a).
Long options are generally more descriptive and easier to understand.
Some options take a value, which can be provided directly after the option (e.g., command -o value) or with an equals sign for long options (e.g., command –option=value).

Examples:

ls -l               ### The -l option tells the ls command to display a long listing format.
grep -i "pattern"   ### The -i option makes grep perform a case-insensitive search.

Arguments:

Arguments are the data or targets that a command operates on. They are typically placed after the command and its options.
Arguments do not have leading hyphens.
The type and number of arguments a command expects depend on the specific command.

ls /home/user/documents     ### /home/user/documents is an argument specifying the directory to list.
cp file1.txt file2.txt      ### file1.txt and file2.txt are arguments specifying the source and destination files for the cp command.

In linux, particularly the C programming getopt() is a standard library function that provides with a set of functionalities and tools primarily used for parsing command-line options and arguments in Unix-like systems and programming languages. It provides a standardized way to handle user input provided when executing a program or script from the command line.

Here we will be looking at similarly library function in python programming language which are provided by the following three modules.

getopt: This module closely mirrors the C getopt API, providing basic C-style parsing of short and long options
optparse: This module provides declarative and user-friendly alternative to getopt, offering improved option parsing and automatic generation of help messages.
argparse: This module provides functional enhancements for command line parameter processing and is recommended one for Python 2.7+ and Python 3.2+.

Let’s try to see the following modules in action using a sample python script.

If you are interested in watching the video. Here is the YouTube video on the same step by step procedure outlined below.

Procedure

Step1: Create a Python script to transform file content

Let’s create a “sample.txt” file with some content which will act as an input to the “transformfile.py” file.

admin@fedser:optionparser$ cat sample.txt 
Hi All,

Gud morning, 

Here is a sample file which will be provided as an input to the transformfile.py script.
This script takes the file names and action as input arguments. Based on the action if its "lower" and "upper", it will
transform the file content to either lowercase or uppercase.

Thanks
novicejava1

The following python script takes two input arguments. Based on the action, it will transform the file content to lowercase or uppercase and print the results to console.

admin@fedser:optionparser$ cat transformfile.py 
### This script is used to transform a file content to either upper or lower case
import sys

def usage():
    print("Run the script as shown below:")
    print("python transformfile.py <action> <filename>")

def transform(args, action):
    for filename in args:
        with open(filename, 'r') as file:
            if action == 'lower':
                for line in file.readlines():
                    print(line.strip().lower())
            elif action == 'upper':
                for line in file.readlines():
                    print(line.strip().upper())
            else:
                for line in file.readlines():
                    print(line)                

if __name__ == '__main__':

    if len(sys.argv) < 3:
        usage()
    else:
        action = sys.argv[1]
        filenames = sys.argv[2:]
        print(action)
        print(filenames)
        transform(filenames, action)

Here are the commands that we can execute with the following script.

admin@fedser:optionparser$ python transformfile.py --help
admin@fedser:optionparser$ python transformfile.py upper sample.txt
admin@fedser:optionparser$ python transformfile.py lower sample.txt

Though this script works perfectly fine, we need to manually parse each arguments and pass it to the function. For an enduser who is running the script there is no way to define exactly what each input argument is meant for or used for within the script. Also, we have to manually handle the incorrect arguments that are passed to the script.

All of these pitfalls can be addressed using the options and argument parsing modules provided by the python programming language.

Step2: Update Python script with getopt module

As defined previously this module closely mirrors the C getopt API, providing basic C-style parsing of short and long options.

We need to import the getopt module and pass the sys.argv string of arguments except the program file name, also we need to define the shorthand options that are accepted by this script and if any shorthand option accepts argument as well it needs to be appended with a “:”.

Finally we can define the longhand options as list of strings with any longhand option accepting argument appended with a “=”.

getopts also provides us with a exception object “GetoptError” that can be handled incase of any incorrect argument that are passed to the script.

With the correct options are arguments that are passed to the script they are captured as a tuple in “opts, args” and further verified and passed to the transform function for processing.

admin@fedser:optionparser$ cat transformfile_with_getopt.py 
### This script is used to transform a file content to either upper or lower case
import sys
import getopt

def usage():
    print("Run the script as shown below:")
    print("python transformfile.py -a <action> <filename>")

def transform(filenames, action):
    for filename in filenames:
        with open(filename, 'r') as file:
            if action == 'lower':
                for line in file.readlines():
                    print(line.strip().lower())
            elif action == 'upper':
                for line in file.readlines():
                    print(line.strip().upper())
            else:
                for line in file.readlines():
                    print(line)                

if __name__ == '__main__':
    try:
        opts, args = getopt.getopt(sys.argv[1:], "ha:", ["help", "action="])
    except getopt.GetoptError as err:
        print(err)  # print exception to incorrect options that are passed
        usage()
        sys.exit(2)

    if not opts and not args:
        usage()
        sys.exit()
    elif not opts:
        action = None
        filenames = args
        transform(filenames, action)
    else:
        for o, a in opts:
            if o in ('-a', '--action'):
                action = a
                filenames = args
                transform(filenames, action)
            elif o in ("-h", "--help"):
                usage()
                sys.exit()
            else:
                assert False, "unhandled option"

Here are the commands that we can execute with the following script.

admin@fedser:optionparser$ python transformfile_with_getopt.py --help
admin@fedser:optionparser$ python transformfile_with_getopt.py -a upper sample.txt
admin@fedser:optionparser$ python transformfile_with_getopt.py -a lower sample.txt

Step3: Update Python script with optparse module

Here in this step we are using the optparse module. In this updated script we are explicitly defining the option that we want to pass to the script (ie. -a or –action). Also we are customing the OptionParser usage parameter with program name, options and arguments that can be passed to the script.

Finally we are using “parser.parse_args()” to identify the options and arguments that are passed to the script and saving it to a tuple (opts, args). These are options and arguments are further passed to the transform() function for processing.

With optparse, we get an inbuilt help option functionality and also its easy to declare the options and parse them.

admin@fedser:optionparser$ cat transformfile_with_optparse.py 
import optparse

def transform(args, action):
    for filename in args:
        with open(filename, 'r') as file:
            for line in file.readlines():
                if action == 'lower':
                    print(line.strip().lower())
                elif action == 'upper':
                    print(line.strip().upper())
                else:
                    print(line)  

if __name__ == '__main__':
    parser = optparse.OptionParser(usage="usage: %prog [options] [-a|--action|-h|--help] <filenames>")
    parser.add_option('-a', '--action')
    opts, args = parser.parse_args()

    print(opts.action)
    print(args)
    transform(args, action=opts.action)

Here are the commands that we can execute with the following script.

admin@fedser:optionparser$ python transformfile_with_optparse.py --help
admin@fedser:optionparser$ python transformfile_with_optparse.py -a upper sample.txt
admin@fedser:optionparser$ python transformfile_with_optparse.py -a lower sample.txt

Step4: Update Python script with argparse module

The argparse module makes it easy to write user-friendly command-line interfaces. The program defines what arguments it requires, and argparse will figure out how to parse those out of sys.argv. The argparse module also automatically generates help and usage messages. The module will also issue errors when users give the program invalid arguments.

Here is the updated script using the argparse module.

admin@fedser:optionparser$ cat transformfile_with_argparse.py 
import argparse

def transform(args, action):
    for filename in args:
        with open(filename, 'r') as file:
            for line in file.readlines():
                if action == 'lower':
                    print(line.strip().lower())
                elif action == 'upper':
                    print(line.strip().upper())
                else:
                    print(line)  

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('-a', '--action')
    parser.add_argument('filenames', nargs='*')
    args = parser.parse_args()
    print(args.action)
    print(args.filenames)
    transform(args.filenames, action=args.action)

Here are the commands that we can execute with the following script.

admin@fedser:optionparser$ python transformfile_with_argparse.py --help
admin@fedser:optionparser$ python transformfile_with_argparse.py -a upper sample.txt
admin@fedser:optionparser$ python transformfile_with_argparse.py -a lower sample.txt

Hope you enjoyed reading this article. Thank you..

How to leverage command line options parsing modules in Python