The biggest difference between dynamic and static languages is that the definitions of functions and classes are not created at compile time, but dynamically at runtime.
For example, to define a Hello class, we create a hello.py module:
class Hello(object):
def hello(self, name='world'):
print('Hello, %s.' % name)
When the Python interpreter loads the hello module, it executes all statements in the module in sequence. The result is the dynamic creation of a Hello class object, which we can test as follows:
>>> from hello import Hello
>>> h = Hello()
>>> h.hello()
Hello, world.
>>> print(type(Hello))
<class 'type'>
>>> print(type(h))
<class 'hello.Hello'>
The type() function can check the type of a type or variable: Hello is a class, so its type is type; h is an instance, so its type is class Hello.
We say that class definitions are dynamically created at runtime, and the method to create a class is to use the type() function.
The type() function can both return the type of an object and create new types. For example, we can create the Hello class using type() instead of defining it with class Hello(object)...:
>>> def fn(self, name='world'): # Define the function first
... print('Hello, %s.' % name)
...
>>> Hello = type('Hello', (object,), dict(hello=fn)) # Create Hello class
>>> h = Hello()
>>> h.hello()
Hello, world.
>>> print(type(Hello))
<class 'type'>
>>> print(type(h))
<class '__main__.Hello'>
To create a class object, the type() function takes three parameters in sequence:
fn to the method name hello).Classes created with the type() function are identical to those defined with the class keyword. When the Python interpreter encounters a class definition, it simply parses the syntax of the class definition and then calls the type() function to create the class.
Under normal circumstances, we define classes using class Xxx.... However, the type() function also allows us to dynamically create classes at runtime. This is a significant difference from static languages—creating classes at runtime in static languages requires constructing source code strings and calling the compiler, or generating bytecode with tools. Essentially, this is dynamic compilation and is extremely complex.
In addition to using type() to dynamically create classes, we can control class creation behavior using metaclasses.
Metaclass (literally translated as “meta-class”) can be simply explained as:
Putting it all together: define the metaclass first → create the class → create instances.
Thus, metaclasses allow you to create or modify classes. In other words, you can think of a class as an “instance” created by a metaclass.
Metaclasses are the most difficult to understand and use among Python’s object-oriented “magic” features. Under normal circumstances, you will never need to use metaclasses—so it’s okay if you don’t understand the following content, as you will almost certainly never use it.
Let’s start with a simple example: this metaclass adds an add() method to our custom MyList class.
First, define ListMetaclass. By convention, metaclass names always end with Metaclass to clearly indicate they are metaclasses:
# A metaclass is a template for classes, so it must inherit from `type`:
class ListMetaclass(type):
def __new__(cls, name, bases, attrs):
attrs['add'] = lambda self, value: self.append(value)
return type.__new__(cls, name, bases, attrs)
With ListMetaclass, we also need to specify using it to customize the class when defining MyList, by passing the keyword argument metaclass:
class MyList(list, metaclass=ListMetaclass):
pass
When we pass the metaclass keyword argument, the magic happens: it instructs the Python interpreter to create MyList via ListMetaclass.__new__(). Here, we can modify the class definition (e.g., add new methods) and return the modified definition.
The __new__() method receives the following parameters in sequence:
Test if MyList can call the add() method:
>>> L = MyList()
>>> L.add(1)
>>> L
[1]
A regular list does not have the add() method:
>>> L2 = list()
>>> L2.add(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'add'
What’s the point of dynamic modification? Wouldn’t it be simpler to directly write the add() method in the MyList definition? Under normal circumstances, yes—modifying via metaclass is purely “perverse”.
However, there are scenarios where modifying class definitions via metaclasses is necessary. ORM (Object Relational Mapping) is a classic example.
ORM (short for “Object Relational Mapping”) maps a row in a relational database to an object—i.e., one class corresponds to one table. This makes coding simpler, as you don’t need to operate SQL statements directly.
To write an ORM framework, all classes must be defined dynamically, because only the user can define the corresponding classes based on table structures.
Let’s attempt to write a simple ORM framework.
The first step in writing underlying modules is to define the calling interface. For example, if a user wants to define a User class to operate the corresponding database table User using this ORM framework, we expect them to write code like this:
class User(Model):
# Define mapping from class attributes to table columns:
id = IntegerField('id')
name = StringField('username')
email = StringField('email')
password = StringField('password')
# Create an instance:
u = User(id=12345, name='Michael', email='test@orm.org', password='my-pwd')
# Save to database:
u.save()
The parent class Model and attribute types (StringField, IntegerField) are provided by the ORM framework. All remaining “magic methods” (e.g., save()) are automatically implemented by the parent class Model. Although writing the metaclass is complex, using the ORM is extremely simple for users.
Now, let’s implement this ORM according to the above interface.
First, define the Field class, which is responsible for storing the field name and type of a database table:
class Field(object):
def __init__(self, name, column_type):
self.name = name
self.column_type = column_type
def __str__(self):
return '<%s:%s>' % (self.__class__.__name__, self.name)
Based on Field, further define various types of Field (e.g., StringField, IntegerField):
class StringField(Field):
def __init__(self, name):
super(StringField, self).__init__(name, 'varchar(100)')
class IntegerField(Field):
def __init__(self, name):
super(IntegerField, self).__init__(name, 'bigint')
Next, write the most complex part: ModelMetaclass:
class ModelMetaclass(type):
def __new__(cls, name, bases, attrs):
if name=='Model':
return type.__new__(cls, name, bases, attrs)
print('Found model: %s' % name)
mappings = dict()
for k, v in attrs.items():
if isinstance(v, Field):
print('Found mapping: %s ==> %s' % (k, v))
mappings[k] = v
for k in mappings.keys():
attrs.pop(k)
attrs['__mappings__'] = mappings # Save attribute-column mappings
attrs['__table__'] = name # Assume table name matches class name
return type.__new__(cls, name, bases, attrs)
And the base class Model:
class Model(dict, metaclass=ModelMetaclass):
def __init__(self, **kw):
super(Model, self).__init__(**kw)
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError(r"'Model' object has no attribute '%s'" % key)
def __setattr__(self, key, value):
self[key] = value
def save(self):
fields = []
params = []
args = []
for k, v in self.__mappings__.items():
fields.append(v.name)
params.append('?')
args.append(getattr(self, k, None))
sql = 'insert into %s (%s) values (%s)' % (self.__table__, ','.join(fields), ','.join(params))
print('SQL: %s' % sql)
print('ARGS: %s' % str(args))
When a user defines class User(Model), the Python interpreter first looks for metaclass in the current User class definition. If not found, it continues searching in the parent class Model. Once found, it uses ModelMetaclass (defined in Model) to create the User class. In other words, metaclasses are implicitly inherited by subclasses, even though the subclasses themselves are unaware of it.
In ModelMetaclass, the following tasks are performed:
Model class itself;User). If a Field attribute is found, save it to a __mappings__ dictionary and remove the Field attribute from the class attributes (otherwise, runtime errors may occur—instance attributes will override class attributes with the same name);__table__ (simplified here to use the class name as the table name by default).In the Model class, we can define various methods for database operations (e.g., save(), delete(), find(), update).
We implemented the save() method to save an instance to the database. With the table name, attribute-column mappings, and collection of attribute values, we can construct an INSERT statement.
Test the code:
u = User(id=12345, name='Michael', email='test@orm.org', password='my-pwd')
u.save()
The output is as follows:
Found model: User
Found mapping: email ==> <StringField:email>
Found mapping: password ==> <StringField:password>
Found mapping: id ==> <IntegerField:uid>
Found mapping: name ==> <StringField:username>
SQL: insert into User (password,email,username,id) values (?,?,?,?)
ARGS: ['my-pwd', 'test@orm.org', 'Michael', 12345]
As you can see, the save() method prints an executable SQL statement and parameter list. We only need to connect to a real database and execute this SQL statement to complete the actual functionality.
With fewer than 100 lines of code, we implemented a streamlined ORM framework using metaclasses—isn’t that amazing?
Metaclasses are highly “magical” objects in Python that can alter the behavior of class creation. This powerful functionality must be used with extreme caution.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
def fn(self, name="world"):
print("Hello, %s." % name)
Hello = type("Hello", (object,), dict(hello=fn))
h = Hello()
print("call h.hello():")
h.hello()
print("type(Hello) =", type(Hello))
print("type(h) =", type(h))
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# metaclass
class ListMetaclass(type):
def __new__(cls, name, bases, attrs):
attrs["add"] = lambda self, value: self.append(value)
return type.__new__(cls, name, bases, attrs)
class MyList(list, metaclass=ListMetaclass):
pass
L = MyList()
L.add(1)
L.add(2)
L.add(3)
L.add("END")
print(L)
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
" Simple ORM using metaclass "
class Field(object):
def __init__(self, name, column_type):
self.name = name
self.column_type = column_type
def __str__(self):
return "<%s:%s>" % (self.__class__.__name__, self.name)
class StringField(Field):
def __init__(self, name):
super(StringField, self).__init__(name, "varchar(100)")
class IntegerField(Field):
def __init__(self, name):
super(IntegerField, self).__init__(name, "bigint")
class ModelMetaclass(type):
def __new__(cls, name, bases, attrs):
if name == "Model":
return type.__new__(cls, name, bases, attrs)
print("Found model: %s" % name)
mappings = dict()
for k, v in attrs.items():
if isinstance(v, Field):
print("Found mapping: %s ==> %s" % (k, v))
mappings[k] = v
for k in mappings.keys():
attrs.pop(k)
attrs["__mappings__"] = mappings
attrs["__table__"] = name
return type.__new__(cls, name, bases, attrs)
class Model(dict, metaclass=ModelMetaclass):
def __init__(self, **kw):
super(Model, self).__init__(**kw)
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError(r"'Model' object has no attribute '%s'" % key)
def __setattr__(self, key, value):
self[key] = value
def save(self):
fields = []
params = []
args = []
for k, v in self.__mappings__.items():
fields.append(v.name)
params.append("?")
args.append(getattr(self, k, None))
sql = "insert into %s (%s) values (%s)" % (self.__table__, ",".join(fields), ",".join(params))
print("SQL: %s" % sql)
print("ARGS: %s" % str(args))
# testing code:
class User(Model):
id = IntegerField("id")
name = StringField("username")
email = StringField("email")
password = StringField("password")
u = User(id=12345, name="Michael", email="test@orm.org", password="my-pwd")
u.save()