“Perfection is achieved not when there is nothing more to add, but rather when there is nothing more to take away.” – Antoine de Saint-Exupery

Recently I encountered with very fascinating concept which is “Meta-programming in Python”. I would like to share my findings about this topic in this article. I hope it may help you to wrap your head around this because they say it is tough nut to crack.

What is Meta-programming?

So in one line:

“Meta-programming is a act of writing code which manipulates code.”

Wait what? yes you read it right. Code which manipulate code. Isn’t it sound fascinating and powerful? Well actually it is.

In the context of Python, meta-programming can be stated as:

“Meta-programming is an act of building functions and classes who can manipulate code by modifying, wrapping existing code or generating code.”

Meta-programming in python can be achieved by:

  1. Decorators
  2. Meta-classes

Lets get familiar with them one by one.

Decorators

Decorator is way to add new functionality to existing function without modifying its original structure.

For instance we have this 3 following functions:

def add(x, y):
    return x + y    

def sub(x, y):
    return x - y
    
def mul(x, y):
    return x * y

Now we need to print function name and parameter values when function get called. This should be applicable to all three function above. Naive way is to add print/log statement to all three functions. But this sounds very repetitive work and also need to modify each function body.

def add(x, y):
    print("add is called with parameter {0},{1}".format(x,y))
    return x + y    

def sub(x, y):
    print("sub is called with parameter {0},{1}".format(x,y))
    return x - y
    
def mul(x, y):
    print("mul is called with parameter {0},{1}".format(x,y))
    return x * y    

print(add(5,3))
print(sub(5,3))
print(mul(5,3))

*********************** output *********************

add is called with parameter 5, 3
8
sub is called with parameter 5, 3
2
mul is called with parameter 5, 3
15

Can we do better? Of course we can because by the grace of god we are programmers and programmers are intelligent. We can achieve this by writing decorator function and by NOT modifying any of existing function body.

def my_decorator(func):
    def wrapper_function(*args):
        print("{0} is called with parameter {1}".format(func.__name__, args))
        return func(*args)
    return wrapper_function

@my_decorator
def add(x, y):
    return x + y
    
@my_decorator
def sub(x, y):
    return x - y

@my_decorator    
def mul(x, y):
    return x * y 

*********************** output *********************

add is called with parameter (5, 3)
8
sub is called with parameter (5, 3)
2
mul is called with parameter (5, 3)
15

Bingo! In above code snippet my_decorator is decorator function and we decorate all three functions with @my_decorator and we have not touch existing function body to add this print functionality.

So basically decorators are higher order function which takes function as argument and returns another function. Here my_decorator takes function as argument and return wrapper_function as result where wrapper_function adds our print functionality to func.

There is more to decorators but this a brief introduction to decorators in python.

Meta-classes

Now we seen decorators, those are for decorating functions. But there is more to meta-programming than decorators, “Meta-classes”.

Meta-classes are special type of classes than ordinary classes in python. Where ordinary class defines behaviour of its own instance, meta-class defines behaviour of “ordinary” class and its instance. Meta-class can add or subtract method or field to ordinary class. Python has one special class “type” class which is default meta-class. All custom type class must inherit from type class.

For instance if we have class “Calc” having three class methods and we want to provide debug functionality to all the methods in one class then we can use meta-class for this.

class Calc():
    def add(self, x, y):
        return x + y
    
    def sub(self, x, y):
        return x - y
    
    def mul(self, x, y):
        return x * y

First we need to create a meta class “MetaClassDebug” having debug functionality and make Calc class inherit from MetaClassDebug. And when we call any method from Calc class it will get invoke with our debug_function.

def debug_function(func):

   def wrapper(*args, **kwargs):
       print("{0} is called with parameter {1}".format(func.__qualname__, args[1:]))
       return func(*args, **kwargs)
   
   return wrapper


def debug_all_methods(cls):

   for key, val in vars(cls).items():
       if callable(val):
           setattr(cls, key, debug_function(val))
   return cls


class MetaClassDebug(type):

   def __new__(cls, clsname, bases, clsdict):
       obj = super().__new__(cls, clsname, bases, clsdict)
       obj = debug_all_methods(obj)
       return obj


class Calc(metaclass=MetaClassDebug):
   def add(self, x, y):
       return x + y

   def sub(self, x, y):
       return x - y

   def mul(self, x, y):
       return x * y


calc = Calc()
print(calc.add(2, 3))
print(calc.sub(2, 3))
print(calc.mul(2, 3))


**************** output ****************

Calc.add is called with parameter (2, 3)
5
Calc.sub is called with parameter (2, 3)
-1
Calc.mul is called with parameter (2, 3)
6

Bingo! In above snippet we create a meta-class MetaClassDebug and write new method which is responsible of creating instance of class and applied our decorator function debug_function to the object(instance) which will get created of every class which inherit MetaClassDebug. As Calc is inherited from MetaClassDebug hence every method has been decorated by debug_function from debug_all_methods.

This way we can add new behaviour to all methods within a class and also control instance creation of a class using meta-class. There can be much achieved by meta-class like adding a method or field to class or removing a method or field from a class and many more.

I wanted you to give glance of meta-programming in python so I could not cover all the things in this post. I hope this article would have helped you to get familiarize with the concept of meta-programming. Comment as well as criticism is always welcome!