Python's __all__: Packages, Modules, and Wildcard Imports

Python's __all__: Packages, Modules, and Wildcard Imports

by Leodanis Pozo Ramos Mar 04, 2024 intermediate python

Python has something called wildcard imports, which look like from module import *. This type of import allows you to quickly get all the objects from a module into your namespace. However, using this import on a package can be confusing because it’s not clear what you want to import: subpackages, modules, objects? Python has the __all__ variable to work around this issue.

The __all__ variable is a list of strings where each string represents the name of a variable, function, class, or module that you want to expose to wildcard imports.

In this tutorial, you’ll:

  • Understand wildcard imports in Python
  • Use __all__ to control the modules that you expose to wildcard imports
  • Control the names that you expose in modules and packages
  • Explore other use cases of the __all__ variable
  • Learn some benefits and best practices of using __all__

To get the most out of this tutorial, you should be familiar with a few Python concepts, including modules and packages, and the import system.

Importing Objects in Python

When creating a Python project or application, you’ll need a way to access code from the standard library or third-party libraries. You’ll also need to access your own code from the multiple files that may make up your project. Python’s import system is the mechanism that allows you to do this.

The import system lets you get objects in different ways. You can use:

  • Explicit imports
  • Wildcard imports

In the following sections, you’ll learn the basics of both strategies. You’ll learn about the different syntax that you can use in each case and the result of running an import statement.

Explicit Imports

In Python, when you need to get a specific object from a module or a particular module from a package, you can use an explicit import statement. This type of statement allows you to bring the target object to your current namespace so that you can use the object in your code.

To import a module by its name, you can use the following syntax:

Python
import module [as name]

This statement allows you to import a module by its name. The module must be listed in Python’s import path, which is a list of locations where the path based finder searches when you run an import.

The part of the syntax that’s enclosed in square brackets is optional and allows you to create an alias of the imported name. This practice can help you avoid name collisions in your code.

As an example, say that you have the following module:

Python calculations.py
def add(a, b):
    return float(a + b)

def subtract(a, b):
    return float(a - b)

def multiply(a, b):
    return float(a * b)

def divide(a, b):
    return float(a / b)

This sample module provides functions that allow you to perform basic calculations. The containing module is called calculations.py. To import this module and use the functions in your code, go ahead and start a REPL session in the same directory where you saved the file.

Then run the following code:

Python
>>> import calculations

>>> calculations.add(2, 4)
6.0
>>> calculations.subtract(8, 4)
4.0
>>> calculations.multiply(5, 2)
10.0
>>> calculations.divide(12, 2)
6.0

The import statement at the beginning of this code snippet brings the module name to your current namespace. To use the functions or any other object from calculations, you need to use fully qualified names with the dot notation.

You can also use a similar syntax to import a Python package:

Python
import package [as name]

In this case, Python loads the content of your package’s __init__.py file into your current namespace. If that file exports objects, then those objects will be available to you.

Finally, if you want to be more specific in what you import into your current namespace, then you can use the following syntax:

Python
from module import name [as name]

With this import statement, you can import specific names from a given module. This approach is recommended when you only need a few names from a long module that defines many objects or when you don’t expect name collisions in your code.

To continue with the calculations module, you can import the needed function only:

Python
>>> from calculations import add

>>> add(2, 4)
6.0

In this example, you only use the add() function. The from module import name syntax lets you import the target name explicitly. In this case, the rest of the functions and the module itself won’t be accessible in your namespace or scope.

Wildcard Imports on Modules

When you’re working with Python modules, a wildcard import is a type of import that allows you to get all the public names from a module in one go. This type of import has the following syntax:

Python
from module import *

The name wildcard import derives from the asterisk at the end of the statement, which denotes that you want to import all the objects from module.

Go back to your terminal window and restart your REPL session. Then, run the following code:

Python
>>> from calculations import *

>>> dir()
[
    ...
    'add',
    'divide',
    'multiply',
    'subtract'
]

In this code snippet, you first run a wildcard import. This import makes available all the names from the calculations modules and brings them to your current namespace. The built-in dir() function allows you to see what names are available in the current namespace. As you can confirm from the output, all the functions that live in calculations are now available.

When you’re completely sure that you need all the objects that a given module defines, using a wildcard import is a quick solution. In practice, this situation is rare, and you just end up cluttering your namespace with unneeded objects and names.

Using wildcard imports is explicitly discouraged in PEP 8 when they say:

Wildcard imports (from <module> import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn’t known in advance). (Source)

The main drawback of wildcard import is that you don’t have control over the imported objects. You can’t be specific. Therefore, you can confuse the users of your code and clutter their namespace with unnecessary objects.

Even though wildcard imports are discouraged, some libraries and tools use them. For example, if you search for applications built with Tkinter, then you’ll realize that many of the examples use the form:

Python
from tkinter import *

This import gives you access to all the objects defined in the tkinter module, which is pretty convenient if you’re starting to learn how to use this tool.

You may find many other tools and third-party libraries that use wildcard imports for code examples in their documentation, and that’s okay. However, in real-world projects, you should avoid this type of import.

In practice, you can’t control how the users of your code will manage their imports. So, you better prepare your code for wildcard imports. You’ll learn how to do this in the upcoming sections. First, you’ll learn about using wildcard imports on packages.

Wildcard Import and Non-Public Names

Python has a well-established naming convention that allows you to tell the users of your code when a given name in a module is for internal or external use.

If an object’s name starts with a single leading underscore, then that name is considered non-public, so it’s for internal use only. In contrast, if a name starts with a lowercase or uppercase letter, then that name is public and, therefore, is part of the module’s public API.

When you have non-public names in a given module, you should know that wildcard imports won’t import those names. Say that you have the following module:

Python shapes.py
from math import pi as _pi

class Circle:
    def __init__(self, radius):
        self.radius = _validate(radius)

    def area(self):
        return _pi * self.radius**2

class Square:
    def __init__(self, side):
        self.side = _validate(side)

    def area(self):
        return self.side**2

def _validate(value):
    if not isinstance(value, int | float) or value <= 0:
        raise ValueError("positive number expected")
    return value

In this module, you have two non-public objects _pi and _validate(). You know this because they have a leading underscore in their names. If someone runs a wildcard import on this module, then the non-public names won’t be imported:

Python
>>> from shapes import *

>>> dir()
[
    'Circle',
    'Square',
    ...
]

If you take a look at the output of dir(), then you’ll note that only the Circle and Square classes are available in your current namespace. The non-public objects, _pi and _validate(), aren’t available. So, wildcard imports won’t import non-public names.

Wildcard Import on Packages

Up to this point, you know how wildcard imports work with modules. You can also use this type of import with packages. In that case, the syntax is the same, but you need to use a package name rather than a module name:

Python
from package import *

Now, what happens when you run this type of import? You may expect that this import causes Python to search the file system, find the modules and subpackages that are present in the package, and import them.

However, doing this file system search could take a long time. Additionally, importing modules might have unwanted side effects, because when you import a module, all the executable code in that module runs.

Because of these potential issues, Python has the __all__ special variable, which will allow you to explicitly define the list of modules that you want to expose to wildcard import in a given package. You’ll explore the details in the next section.

Preparing Your Packages for Wildcard Imports With __all__

Python has two different behaviors when dealing with wildcard imports on packages. Both behaviors depend on whether the __all__ variable is present in the package’s __init__.py file.

  • If __init__.py doesn’t define __all__, then nothing happens when you run a wildcard import on the package.
  • If __init__.py defines __all__, then the objects listed in it will be imported.

To illustrate the first behavior, go ahead and create a new folder called shapes/. Inside the folder, create the following files:

shapes/
├── __init__.py
├── circle.py
├── square.py
└── utils.py

Leave the __init__.py file empty for now. Take the code of your shapes.py file and split it into the rest of the files. Click the collapsible section below to see how to do this:

Python shapes/circle.py
from math import pi as _pi

from shapes.utils import validate

class Circle:
    def __init__(self, radius):
        self.radius = validate(radius)

    def area(self):
        return _pi * self.radius**2
Python shapes/square.py
from shapes.utils import validate

class Square:
    def __init__(self, side):
        self.side = validate(side)

    def area(self):
        return self.side**2
Python shapes/utils.py
def validate(value):
    if not isinstance(value, int | float) or value <= 0:
        raise ValueError("positive number expected")
    return value

In this sample package, the __init__.py file doesn’t define the __all__ variable. So, if you run a wildcard import on this package, then you won’t import any name into your namespace:

Python
>>> from shapes import *

>>> dir()
[
    '__annotations__',
    '__builtins__',
    ...
]

In this example, the dir() function reveals that the wildcard import didn’t bring any name to your current namespace. The circle, square, and utils modules aren’t available in your namespace.

If you don’t define __all__ in a package, the statement from package import * doesn’t import all the modules from the target package into the current namespace. In that case, the import statement only ensures that the package was imported and runs any code in __init__.py.

If you want to prepare a Python package for wildcard imports, then you need to define the __all__ variable in the package’s __init__.py file. The __all__ variable should be a list of strings containing those names that you want to export from your package when someone uses a wildcard import.

Go ahead and add the following line to the file:

Python shapes/__init__.py
__all__ = ["circle", "square"]

By defining the __all__ variable in the __init__.py file, you establish the module names that a wildcard import will bring into your namespace. In this case, you only want to export the circle and square modules from your package.

Now, run the following code in your interactive session:

Python
>>> from shapes import *

>>> dir()
[
    ...
    'circle',
    'square'
]

Now, when you run a wildcard import on your shapes package, the circle and square modules become available in your namespace. Note that the utils module isn’t available because you didn’t list it in __all__.

It’s up to you as the package author to build this list and keep it up-to-date. Maintaining the list up-to-date is crucial when you release a new version of your package. In this case, it’s also important to note that you’ll get an AttributeError exception if __all__ contains undefined names.

Finally, if you define __all__ as an empty list, then nothing will be exported from your package. It’s like not defining __all__ in the package.

Exposing Names From Modules and Packages With __all__

You already know that when you run a wildcard import on a module, then you’ll import all the public constants, variables, functions, classes, and other objects in the module. Sometimes, this behavior is okay. However, in some cases, you need to have fine control over what the module exports. You can also use __all__ for this goal.

Another interesting use case of __all__ is when you need to export specific names or objects from a package. In this case, you can also use __all__ in a slightly different way.

In the following sections, you’ll learn how to use __all__ for controlling what names a module exports and how to export specific names from a package.

Names From a Module

You can use the __all__ variable to explicitly control what names a module exposes to wildcard imports. In this sense, __all__ allows you to establish a module’s public interface or API. This technique is also a way to explicitly communicate what the module’s API is.

If you have a large module with many public names, then you can use __all__ to create a list of exportable names so that wildcard imports don’t pollute the namespace of your code’s users.

In general, modules can have a few different types of names:

  • Public names are part of the module’s public interface.
  • Non-public names are for internal use only.
  • Imported names are names that the module imports as public or non-public names.

As you already know, public names are those that start with a lowercase or uppercase letter. Non-public names are those that start with a single leading underscore.

Finally, imported names are those that you import as public names in a module. These names are also exported from that module. So, that’s why you’ll see imports like the following in many codebases:

Python
import sys as _sys

In this example, you import the sys module as _sys. The as specifier lets you create an alias for the imported object. In this case, the alias is a non-public name. With this tiny addition to your import statement, you prevent sys from being exported when someone uses a wildcard import on the module.

So, if you don’t want to export imported objects from a module, then use the as specifier and a non-public alias for the imported objects.

Ideally, the __all__ list should only contain public names that are defined in the containing module. As an example, say that you have the following module containing functions and classes that allow you to make HTTP requests:

Python webreader.py
import requests

__all__ = ["get_page_content", "WebPage"]

BASE_URL = "http://example.com"

def get_page_content(page):
    return _fetch_page(page).text

def _fetch_page(page):
    url = f"{BASE_URL}/{page}"
    return requests.get(url)

class WebPage:
    def __init__(self, page):
        self.response = _fetch_page(page)

    def get_content(self):
        return self.response.text

In this sample module, you import the requests library. Next, you define the __all__ variable. In this example, __all__ includes the get_page_content() function and the WebPage class, which are public names.

Note that the helper function _fetch_page() is for internal use only. So, you don’t want to expose it to wildcard imports. Additionally, you don’t want the BASE_URL constant or the imported requests module to be exposed to wildcard imports.

Here’s how the module responds to a wildcard import:

Python
>>> from webreader import *

>>> dir()
[
    'WebPage',
    ...
    'get_page_content'
]

When you run a wildcard import on the webreader module, only the names listed in __all__ are imported. Now go ahead and comment out the line where you define __all__, restart your REPL session, and run the import again:

Python
>>> from webreader import *

>>> dir()
[
    'BASE_URL',
    'WebPage',
    ...
    'get_page_content',
    'requests'
]

A quick look at the output of dir() shows that now your module exports all the public names, including BASE_URL and even the imported requests library.

The __all__ variable lets you have full control over what a module exposes to wildcard imports. However, note that __all__ doesn’t prevent you from importing specific names from a module using an explicit import:

Python
>>> from webreader import _fetch_page

>>> dir()
[
    ...
    '_fetch_page'
]

Note that you can use explicit import to bring any name from a given module, even non-public names as _fetch_page() in the example above.

Names From a Package

In the previous section, you learned how to use __all__ to define which objects are exposed to wildcard imports. Sometimes, you want to do something similar but at the package level. If you want to control the objects and names that a package exposes to wildcard imports, then you can do something like the following in the package’s __init__.py file:

Python package/__init__.py
from module_0 import name_0, name_1, name_2, name_3
from module_1 import name_4, name_5, name_6

__all__ = [
    "name_0",
    "name_1",
    "name_2",
    "name_3",
    "name_4",
    "name_5",
    "name_6",
]

The import statements tell Python to grab the names from each module in the package. Then, in __all__, you list the imported names as strings. This technique is great for those cases where you have a package with many modules, and you want to provide a direct path for imports.

As an example of how this technique works in practice, get back to the shapes package and update the __init__.py file as in the code below:

Python shapes/__init__.py
from shapes.circle import Circle
from shapes.square import Square

__all__ = ["Circle", "Square"]

In this update, you’ve added two explicit imports to get the Circle and Square classes from their respective module. Then, you add the class names as strings to the __all__ variable.

Here’s how the package responds to wildcard imports now:

Python
>>> from shapes import *

>>> dir()
[
    'Circle',
    'Square',
    ...
]

Your shapes package exposes the Circle and Square classes to wildcard imports. These classes are what you’ve defined as the public interface of your package. Note how this technique facilitates direct access to names that otherwise you would have to import through qualified names.

Exploring Alternative Use Cases of __all__ in Python

Besides allowing you to control what your modules and packages expose to wildcard imports, the __all__ variable may serve other purposes. You can use __all__ to iterate over the names and objects that make up the public interface of a package or module. You can also take advantage of __all__ when you need to expose dunder names.

Iterating Over a Package’s Interface

Because __all__ is typically a list object, you can use it to iterate over the objects that make up a module’s interface. The advantage of using __all__ over dir() is that the package author has explicitly defined which names they consider to be part of the public interface of their package. If you iterate over __all__, you won’t need to filter out non-public names as you’d have to when you iterate over dir(module).

For example, say that you have a module with a few similar classes that share the same interface. Here’s a toy example:

Python vehicles.py
__all__ = ["Car", "Truck"]

class Car:
    def start(self):
        print("The car is starting")

    def drive(self):
        print("The car is driving")

    def stop(self):
        print("The car is stopping")

class Truck:
    def start(self):
        print("The truck is starting")

    def drive(self):
        print("The truck is driving")

    def stop(self):
        print("The truck is stopping")

In this module, you have two classes that represent vehicles. They share the same interface, so you can use them in similar places. You’ve also defined the __all__ variable, listing the two classes as strings.

Now say that you want to use these classes in a loop. How can you do this? You can use __all__ as in the code below:

Python
>>> import vehicles

>>> for v in vehicles.__all__:
...     vehicle = getattr(vehicles, v)()
...     vehicle.start()
...     vehicle.drive()
...     vehicle.stop()
...
The car is starting
The car is driving
The car is stopping
The truck is starting
The truck is driving
The truck is stopping

In this example, you first import the vehicles module. Then, you start a for loop over the __all__ variable. Because __all__ is a list of strings, you can use the built-in getattr() function to access the specified objects from vehicles. This way, you’ve iterated over the classes that make up the module’s public API.

Accessing Non-Public and Dunder Names

When you’re writing modules and packages, sometimes you use module-level names that start and end with double underscores. These names are typically called dunder names. There are a few dunder constants, such as __version__ and __author__, that you may need to expose to wildcard imports.

Remember that the default behavior is that these names aren’t imported because they start with a leading underscore. To work around this issue, you can explicitly list these names in your __all__ variable.

To illustrate this practice, get back your webreader.py file and update it as in the code below:

Python webreader.py
import requests

__version__ = "1.0.0"
__author__ = "Real Python"

__all__ = ["get_page_content", "WebPage", "__version__", "__author__"]

BASE_URL = "http://example.com"

def get_page_content(page):
    return _fetch_page(page).text

# ...

In this update, you define two module-level constants that use dunder names. The first constant provides information about the module’s version, and the second constant holds the author’s name.

Here’s how a wildcard import works on this module:

Python
>>> from webreader import *

>>> dir()
[
    'WebPage',
    ...
    '__author__',
    ...
    '__version__',
    'get_page_content'
]

Now, when someone uses a wildcard import on the webreader module, they get the dunder variables imported into their namespace.

Using __all__ in Python: Benefits and Best Practices

Up to this point, you’ve learned a lot about the __all__ variable and how to use it in your code. While you don’t need to use __all__, it gives you complete control over what your packages and modules expose to wildcard imports.

The __all__ variable is also a way to communicate to the users of your packages and modules which parts of your code they’re supposed to be using as the public interface.

Here’s a quick summary of the main benefits that __all__ can provide:

  • Control over what you expose to wildcard imports: Using __all__ allows you to explicitly specify the public interface of your packages and modules. This practice prevents accidental usage of objects that shouldn’t be used from outside the module. It provides a clear boundary between the module’s internal implementation and its public API.
  • Enhance readability: Using __all__ allows other developers to quickly learn which objects make up the code’s API without examining the entire codebase. This improves code readability and saves time, especially for larger projects with multiple modules.
  • Reduce namespace cluttering: Using __all__ allows you to list the names to be exposed to wildcard imports. This way, you prevent other developers from polluting their namespace with unnecessary or conflicting names.

Even though wildcard imports are discouraged in Python, you have no way to control what the users of your code will do while using it. So, using __all__ is a good way to limit wrong uses of your code.

Here’s a quick list of best practices for using __all__ in your code:

  • Try to always define __all__ in your packages and modules. This variable provides you with explicit control over what other developers can import with wildcard imports.
  • Take advantage of __all__ as a tool for explicitly defining the public interface of your packages and modules. This practice makes it clear to other developers which objects are intended for external use and which are for internal use only.
  • Keep __all__ focused. The __all__ variables shouldn’t include every object in your module, just the ones that are part of the public API.
  • Use __all__ in conjunction with good documentation. Clear documentation about the intended use and behavior of each object in the public API is the best complement to __all__.
  • Be consistent in using __all__ across all your packages and modules. This practice allows other developers to better understand how to use your code.
  • Regularly review and update __all__. The __all__ variable should always reflect the latest changes in your code’s API. Regularly maintaining __all__ ensures that your code remains clean and usable.

Finally, remember that __all__ only affects the wildcard imports. If a user of your code imports a specific object from a package or module, then that object will be imported even if you don’t have it listed in __all__.

Conclusion

Now you know what wildcard imports are in Python. You’ve learned that these imports allow you to quickly get all the public objects from modules and packages. To control the process, Python has the __all__ variable, which you can define in your modules and packages as a list of objects that are available for wildcard imports.

The __all__ variable is a list of strings where each string represents the name of a variable, function, class, or module.

In this tutorial, you’ve learned how to:

  • Work with wildcard imports in Python
  • Control the modules that you expose to wildcard imports with __all__
  • Control the names that you expose in modules and packages.
  • Explore other use cases for the __all__ variable
  • Understand the benefits and best practices of using __all__

With this knowledge, you can now write more robust, readable, and reliable modules and packages with explicit behavior against wildcard imports on your code.

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About Leodanis Pozo Ramos

Leodanis is an industrial engineer who loves Python and software development. He's a self-taught Python developer with 6+ years of experience. He's an avid technical writer with a growing number of articles published on Real Python and other sites.

» More about Leodanis

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Rate this article:

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.


Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!

Keep Learning

Related Topics: intermediate python