All data in Python resides in objects. And all Python objects have a type with associated attributes and methods. Many of the native data types come with their own methods to create and initialize new instances and work with various operators by default. Your custom classes aren't so lucky.
Fortunately, you can ensure that your classes and objects behave like these native data types using something known as dunder methods or magic methods. You can quickly recognize these operations by the double underscore or “dunder” format: __methodname__(). These methods add special functionality to your classes—constructor methods and overloaded operators—so that they perform some functionality with your custom classes.
Every Python programmer should know how to use dunder methods to give native data type functionalities to your user-defined objects. Dunder methods allow you to provide your own classes with enriched functionality, streamlining your code and avoiding complicated, messy attempts to manually replicate the functions available for built-in Python data types.
Understanding the Python native data types
To fully understand dunder methods, you first need to understand the basic built-in Python data types and some of the functions associated with them that you can replicate for your own classes.
Python object types have several characteristics. First, they may be single value object types (e.g., numbers or Boolean values) or container/collection object types (e.g., lists and dictionaries).
Second, Python types can be mutable or immutable. Mutable object types have contents that can be modified. On the other hand, immutable object types remain fixed and cannot be altered.
Each built-in data type in Python has its own set of dunder methods. You can quickly see a list of all dunder methods for a data type using the dir() function. Most of the time, you will not call these methods directly or override their definition, as they are associated with certain specific functions. Until, of course, you need to use them with your objects.
While it might seem overly simple to go back over common data types, it's just as important to know the basics as it is to know more advanced information like how to identify vulnerabilities in your code for security purposes.
Numeric
While numeric types are fairly self-explanatory, you should be aware of three subsets:
TypeDescriptionExampleConstructorIntegerPositive or negative whole numbers1, 2, 3, -1int()Floating-pointRational numbers, i.e. decimals and fractions1.5, 2.75, 3.5float()ComplexNumbers with a real and imaginary part, each of which itself is a floating-point number.10 + 5j complex()
Numeric types allow for a wide range of mathematical operations associated with dunder methods. For example, the addition operator + corresponds to the dunder method __add__(). Additional mathematical dunders include __sub__, __mul__, __divmod__, __abs__, __lt__, __gt__, __le__, etc. (which also have associated operator symbols, for example, -, *, / <, >).
Typically, you just use standard operators instead of these functions. For example, if you want to add 10 to a number, you would use the standard operator num + 10 rather than num.__add__(10). However, if you want to make these standard operators work with user-defined objects, you need to implement dunder methods in the data type.
Sequences
Python has several native data types that deal with sequences, whether of alphanumeric characters, binary numbers, or items. Sequence types have a wide range of uses, including sorting, storage, and looping.
TypeDescriptionExampleConstructorListThe broadest category of sequence data types includes several subtypes: Lists are mutable (i.e. modifiable) sequences, normally of similar or homogeneous items Tuples are immutable (i.e. they cannot change state or content) sequences, particularly useful for different or heterogeneous items Ranges are immutable sequences of numbers, most often used for managing loopsList: a, b, c, d, e Tuple: 1, 2, 3 Range: 0 (start value), 100 (stop value), 10 (iterative step) list() tuple() range()TextA text-specific sequence type, this type encompasses sequences of Unicode characters.Hello worldstr()BinaryPython has three different binary-specific sequence data types: Bytes are immutable sequences of single bytes. Byte arrays are the mutable version of bytes. memoryview objects allow Python to access and manipulate the content of a byte-type object without first copying it. These types allow the manipulation of binary objects and the conversion of string data to binary for machine storage.Byte: 0xff Bytearray: [0x13, 0x00, 0x00, 0x00, 0x08, 0x00] Memoryview: 64 or [10, 20, 30] b’123’b’abc’b'\x20\xC2\xA9’ byte() bytearray() memoryview()
Sequence dunders include __contains__, __getitem__, __delitem__ and more.
One of the more widely used functions for sequence data is len(), which returns the length of the entire sequence. Attempting to use the built-in len() function on a custom class returns an attribute error. However, by implementing the __len__ dunder in your class, you can replicate built-in functionality. You can then use these objects anywhere you would a native sequence type without changing code.
Boolean
Part native data type, part operator, Booleans are single value types that evaluate the truth or falsity of a statement. The Boolean data type uses the bool() constructor and has only two possible values: true and false. The corresponding Boolean dunder is __bool__().
You can use __bool__() to redefine when an object will return true versus false. In truth value testing, the default state of an object is always true unless the coder includes a specific set of conditions for truth in the class definition. However, if you only want the object to be true if a certain state of the object is within a given range, you would use __bool__() to set the conditions for a true result.
Sets, Mappings, and Dictionaries
As with other types, the native set data types (which are containers) include mutable and immutable variants, using the set() and frozen set() constructors, respectively. Sets differ from lists in that sets are unordered collections, where the individual elements must be hashable (i.e. they must have a constant hash value across their lifespan). Note, however, that the set type itself is not hashable, even though it contains hashable items, because it is mutable.
Examples of sets include {"dog", "cat", "bird"} and {"Alice", "Bob", "Carol"}.
Set types are particularly useful when comparing groups of items. For example, you can use sets when you want to determine membership in a group, identify common elements in groups (i.e. intersection) or elements not in common (i.e. difference and symmetric difference), or combine groups (i.e. union).
The dictionary type is a container type that uses the dict() constructor. It creates one or more key:value pairs between a unique, hashable value (key) and an arbitrary item (value). For example, a dictionary type might have the following key:value pairs: 1:’Hello’, 2:’world’.
Sets, mappings, and dictionaries, like sequences, can use a variety of dunders for accessing specific portions of the mapping. Such dunders include __setitem__, __contains__, ___getitem__, etc.
There are many more advanced data types in Python, including classes, iterators, and exceptions. But as we will see, classes and instances do not necessarily come along with the same dunder methods as other data types.
Using dunder methods in Python
The most well-known dunder method in Python is __init__(), which initializes the state of a new instance. Most python programmers focus overrides on __init__ so that changes take place on instantiation of a new object, while __new__ typically only creates subclasses of immutable data types. The syntax for the init method varies by data type, but typically it includes self (the object being initialized) and one or more variables.
So, as an example, you just got an email promising loads of Bitcoin if you first just provide a fraction of a Bitcoin to the mail sender, which is a classic example of a phishing scam. You want to start tracking how much bitcoin you have in your various accounts. You could start by defining a bitcoin wallet object like this:
class btc_wallet(object):
def __init__(self, wallet_name, amount):
self.wallet_name = wallet_name
self.amount = amount
You can then initialize an instance for each of your bitcoin wallets and show us how much you have:
btcwallet1 = btc_wallet('Coinbase', 5)
print("I have", btcwallet1.amount, "BTC in my", btcwallet1.wallet_name, "wallet (lucky me!)")This would output “I have 5 BTC in my Coinbase wallet (lucky me!).”
The btc_wallet class is custom, so many standard operations are not available to use with it. If we try to do so, we will get an error message. But now, let’s look at how we would use dunder methods to add functionality to our btc_wallet object type.
First, let’s create two more bitcoin wallets.
btcwallet2 = btc_wallet('Binance', 2)
btcwallet3 = btc_wallet('Huobi', 25)If you want to determine how much total bitcoin you have in your first two wallets it would be tempting to write:
total_btc = btcwallet1 + btcwallet2But if you use the dir() function on btc_wallet, you would see that __add__ is not present. So we need to modify the class definition with a dunder method:
def __add__(self, other):
return self.amount + other.amountNow we can calculate and print our total:
total_btc = btcwallet1 + btcwallet2
print("I have a total of", total_btc, "BTC in my wallets (lucky me!)")And our output is “I have a total of 7 BTC in my wallets (lucky me!)”
Now, what if we want to total only our wallets that have three or more BTC? We could use a standard if then statement, checking the amount for each wallet and including only the ones where btcwallet.amount > 3. But we could also return to our discussion about Booleans above. You can define btcwallet so that it is only true if the amount is greater than three by adding the following to the class definition:
def __bool__(self):
if self.amount > 3:
return True
return FalseNow you can set up the loop to calculate the total of your wallets:
total_btc = 0
for x in range (0,3):
loopname = locals()['btcwallet' + (str(x+1))]
if bool(loopname):
total_btc = total_btc + loopname.amountThe output will now be “I have a total of 30 BTC in my wallets (lucky me!)”. And you are indeed lucky as this is approximately $1.2 million.
We can use this dunder method overloading practice to build out functionality for our classes, whether mathematical operations, comparisons, or even constrained Boolean operations.
[Note: code operation was verified using the ExtendsClass Python checker.]
Conclusion
Understanding how native classes and their functions work is crucial for any Python programmer. And it is equally important to dunder-stand how to extend those native type methods to user-defined classes and objects.
Ed note: Thanks to SuperStormer and richardec for their feedback on this article.