Using Environment Variables in Python for App Configuration and Secrets

As a developer, you’ve likely used environment variables in the command line or shell scripts, but have you used them as a way of configuring your Python applications?

This guide will show you all the code necessary for getting, setting, and loading environment variables in Python, including how to use them for supplying application config and secrets.

Not familiar with environment variables? Check out our ultimate guide for using environment variables in Linux and Mac.

Why use environment variables for configuring Python applications?

Before digging into how to use environment variables in Python, it's important to understand why they're arguably the best way to configure applications. The main benefits are:

• Deploy your application in any environment without code changes
• Ensures secrets such as API keys are not leaked into source code

Environment variables have the additional benefit of abstracting from your application how config and secrets are supplied.

Finally, environment variables enable your application to run anywhere, whether it's for local development on macOS, a container in a Kubernetes Pod, or platforms such as Heroku or Vercel.

Here are some examples of using environment variables to configure a Python script or application:

• Set FLASK_ENV environment variable to "development" to enable debug mode for a Flask application
• Provide the STRIPE_API_KEY environment variable for an Ecommerce site
• Supply the DISCORD_TOKEN environment variable to a Discord bot app so it can join a server
• Set environment specific database variables such as DB_USER and DB_PASSWORD so database credentials are not hard-coded

How are environment variables in Python populated?

When a Python process is created, the available environment variables populate the os.environ object which acts like a Python dictionary. This means that:

• Any environment variable modifications made after the Python process was created will not be reflected in the Python process.
• Any environment variable changes made in Python do not affect environment variables in the parent process.

Now that you know how environment variables in Python are populated, let's look at how to access them.

How to get a Python environment variable

Environment variables in Python are accessed using the os.environ object.

The os.environ object seems like a dictionary but is different as values may only be strings, plus it's not serializable to JSON.

You've got a few options when it comes to referencing the os.environ object:

1# 1. Standard way
2import os
3# os.environ['VAR_NAME']
4
5# 2. Import just the environ object
6from os import environ
7# environ['VAR_NAME']
8
9# 3. Rename the `environ` to env object for more concise code
10from os import environ as env
11# env['VAR_NAME']

I personally prefer version 3 as it's more succinct, but will stick to using os.environ for this article.

Accessing a specific environment variable in Python can be done in one of three ways, depending upon what should happen if an environment variable does not exist.

Let's explore with some examples.

Option 1: Required with no default value

If your app should crash when an environment variable is not set, then access it directly:

1print(os.environ['HOME']
2# >> '/home/dev'
3
4print(os.environ['DOES_NOT_EXIST']
5# >> Will raise a KeyError exception

For example, an application should fail to start if a required environment variable is not set, and a default value can't be provided, e.g. a database password.

If instead of the default KeyError exception being raised (which doesn't communicate why your app failed to start), you could capture the exception and print out a helpful message:

1import os
2import sys
3
4# Ensure all required environment variables are set
5try:  
6  os.environ['API_KEY']
7except KeyError: 
8  print('[error]: `API_KEY` environment variable required')
9  sys.exit(1)

Option 2: Required with default value

You can have a default value returned if an environment variable doesn't exist by using the os.environ.get method and supplying the default value as the second parameter:

1# If HOSTNAME doesn't exist, presume local development and return localhost
2print(os.environ.get('HOSTNAME', 'localhost')

If the variable doesn't exist and you use os.environ.get without a default value, None is returned

1assert os.environ.get('NO_VAR_EXISTS') == None

Option 3: Conditional logic if value exists

You may need to check if an environment variable exists, but don't necessarily care about its value. For example, your application can be put in a "Debug mode" if the DEBUG environment variable is set.

You can check for just the existence of an environment variable:

1if 'DEBUG' in os.environ:
2  print('[info]: app is running in debug mode')

Or check to see it matches a specific value:

1if os.environ.get('DEBUG') == 'True':
2  print('[info]: app is running in debug mode')

How to set a Python environment variable

Setting an environment variable in Python is the same as setting a key on a dictionary:

1os.environ['TESTING'] = 'true'

What makes os.environ different to a standard dictionary, is that only string values are allowed:

1os.environ['TESTING'] = True
2# >> TypeError: str expected, not bool

In most cases, your application will only need to get environment variables, but there are use cases for setting them as well.

For example, constructing a DB_URL environment variable on application start-up using DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, and DB_NAME environment variables:

1os.environ['DB_URL'] = 'psql://{user}:{password}@{host}:{port}/{name}'.format(
2  user=os.environ['DB_USER'],
3  password=os.environ['DB_PASSWORD'],
4  host=os.environ['DB_HOST'],
5  port=os.environ['DB_PORT'],
6  name=os.environ['DB_NAME']
7)

Another example is setting a variable to a default value based on the value of another variable:

1# Set DEBUG and TESTING to 'True' if ENV is 'development'
2if os.environ.get('ENV') == 'development':
3  os.environ.setdefault('DEBUG', 'True') # Only set to True if DEBUG not set
4  os.environ.setdefault('TESTING', 'True') # Only set to True if TESTING not set

How to delete a Python environment variable

If you need to delete a Python environment variable, use the os.environ.pop function:

To extend our DB_URL example above, you may want to delete the other DB_ prefixed fields to ensure the only way the app can connect to the database is via DB_URL:

Another example is deleting an environment variable once it is no longer needed:

1auth_api(os.environ['API_KEY']) # Use API_KEY
2os.environ.pop('API_KEY') # Delete API_KEY as it's no longer needed

How to list Python environment variables

To view all environment variables:

The output of this command is difficult to read though because it's printed as one huge dictionary.

A better way, is to create a convenience function that converts os.environ to an actual dictionary so we can serialize it to JSON for pretty-printing:

1import os
2import json
3
4def print_env():
5  print(json.dumps({**{}, **os.environ}, indent=2))

Why default values for environment variables should be avoided

You might be surprised to learn it's best to avoid providing default values as much as possible. Why?

Default values can make debugging a misconfigured application more difficult, as the final config values will likely be a combination of hard-coded default values and environment variables.

Relying purely on environment variables (or as much as possible) means you have a single source of truth for how your application was configured, making troubleshooting easier.

Using a .env file for Python environment variables

As an application grows in size and complexity, so does the number of environment variables.

Many projects experience growing pains when using environment variables for app config and secrets because there is no clear and consistent strategy for how to manage them, particularly when deploying to multiple environments.

A simple (but not easily scalable) solution is to use a .env file to contain all of the variables for a specific environment.

Then you would use a Python library such as python-dotenv to parse the .env file and populate the os.environ object.

To follow along, create and activate a new virtual environment, then install the python-dotenv library:

1# 1. Create
2python3 -m venv ~/.virtualenvs/doppler-tutorial
3
4# 2. Activate
5source ~/.virtualenvs/doppler-tutorial/bin/activate
6
7# 3. Install dotenv package
8pip install python-dotenv

Now save the below to a file named .env (note how it's the same syntax for setting a variable in the shell):

1API_KEY="357A70FF-BFAA-4C6A-8289-9831DDFB2D3D"
2HOSTNAME="0.0.0.0"
3PORT="8080"

Then save the following to dotenv-test.py:

1# Rename `os.environ` to `env` for nicer code
2from os import environ as env
3
4from dotenv import load_dotenv
5load_dotenv()
6
7print('API_KEY:  {}'.format(env['API_KEY']))
8print('HOSTNAME: {}'.format(env['HOSTNAME']))
9print('PORT:     {}'.format(env['PORT']))

Then run dotenv-test.py to test the environment variables are being populated:

1python3 dotenv-test.py
2# >> API_KEY:  357A70FF-BFAA-4C6A-8289-9831DDFB2D3D
3# >> HOSTNAME: 0.0.0.0
4# >> PORT:     8080

While .env files are simple and easy to work with at the beginning, they also cause a new set of problems such as:

• How to keep .env files in-sync for every developer in their local environment?
• If there is an outage due to misconfiguration, accessing the container or VM directly in order to view the contents of the .env may be required for troubleshooting.
• How do you generate a .env file for a CI/CD job such as GitHub Actions without committing the .env file to the repository?
• If a mix of environment variables and a .env file is used, the only way to determine the final configuration values could be by introspecting the application.
• Onboarding a developer by sharing an unencrypted .env file with potentially sensitive data in a chat application such as Slack could pose security issues.

These are just some of the reasons why we recommend moving away from .env files and using something like Doppler instead.

Doppler provides an access-controlled dashboard to manage environment variables for every environment with an easy-to-use CLI for accessing config and secrets that work for every language, framework, and platform.

Centralize application config using a Python data structure

Creating a config specific data structure abstracts away how the config values are set, what fields have default values (if any), and provides a single interface for accessing config values instead of os.environ being littered throughout your codebase.

Below is a reasonably full-featured solution that supports:

• Required fields
• Optional fields with defaults
• Type checking and typecasting

To try it out, save this code to config.py:

1import os
2from typing import get_type_hints, Union
3from dotenv import load_dotenv
4
5load_dotenv()
6
7class AppConfigError(Exception):
8    pass
9
10def _parse_bool(val: Union[str, bool]) -> bool:  # pylint: disable=E1136 
11    return val if type(val) == bool else val.lower() in ['true', 'yes', '1']
12
13# AppConfig class with required fields, default values, type checking, and typecasting for int and bool values
14class AppConfig:
15    DEBUG: bool = False
16    ENV: str = 'production'
17    API_KEY: str
18    HOSTNAME: str
19    PORT: int
20
21    """
22    Map environment variables to class fields according to these rules:
23      - Field won't be parsed unless it has a type annotation
24      - Field will be skipped if not in all caps
25      - Class field and environment variable name are the same
26    """
27    def __init__(self, env):
28        for field in self.__annotations__:
29            if not field.isupper():
30                continue
31
32            # Raise AppConfigError if required field not supplied
33            default_value = getattr(self, field, None)
34            if default_value is None and env.get(field) is None:
35                raise AppConfigError('The {} field is required'.format(field))
36
37            # Cast env var value to expected type and raise AppConfigError on failure
38            try:
39                var_type = get_type_hints(AppConfig)[field]
40                if var_type == bool:
41                    value = _parse_bool(env.get(field, default_value))
42                else:
43                    value = var_type(env.get(field, default_value))
44
45                self.__setattr__(field, value)
46            except ValueError:
47                raise AppConfigError('Unable to cast value of "{}" to type "{}" for "{}" field'.format(
48                    env[field],
49                    var_type,
50                    field
51                )
52            )
53
54    def __repr__(self):
55        return str(self.__dict__)
56
57# Expose Config object for app to import
58Config = AppConfig(os.environ)

The Config object exposed in config.py is then used by app.py below:

1from config import Config
2
3print('ENV:      {}'.format(Config.ENV))
4print('DEBUG:    {}'.format(Config.DEBUG))
5print('API_KEY:  {}'.format(Config.API_KEY))
6print('HOSTNAME: {}'.format(Config.HOSTNAME))
7print('PORT:     {}'.format(Config.PORT))

Make sure you have the .env file still saved from earlier, then run:

1python3 app.py 
2# >> ENV:      production
3# >> DEBUG:    False
4# >> API_KEY:  357A70FF-BFAA-4C6A-8289-9831DDFB2D3D
5# >> HOSTNAME: 0.0.0.0
6# >> PORT:     8080

You can view this code on GitHub and if you're after a more full-featured typesafe config solution, then check out the excellent Pydantic library.

Summary

Awesome work! Now you know how to use environment variables in Python for application config and secrets.

Although we're a bit biased, we encourage you to try using Doppler as the source of truth for your application environment variables, and it's free to get started with our Community plan (unlimited projects and secrets).

We also have a tutorial for building a random Mandalorion GIF generator in Python that puts into practice the techniques shown in this article.

Hope you enjoyed the post and if you have any questions or feedback, we'd love to chat with you over in our Community forum.

Big thanks to Stevoisiak, Olivier Pilotte, Jacob Kasner, and Alex Hall for their input and review!

If you're new to Python, check out Cory Althoff's Python environment variables primer which covers entry-level concepts in more detail.

Python Environment Variables FAQ

What are the benefits of using environment variables in Python?

The benefits of using environment variables in Python include the ability to deploy applications in any environment without code changes and the safeguarding of secrets like API keys from being exposed in the source code. This approach also abstracts how configurations and secrets are supplied, allowing applications to run anywhere, from local development setups to cloud platforms.

Why can't I access an environment variable in my Python code?

If you can't access an environment variable in your Python code, it might be because the environment variable was not set before the Python process started or there's a typo in the variable name. Python's os.environ object, which stores environment variables, only reflects variables available at the start of the process, and changes made afterward are not visible to it.

How to protect your credentials using environment variables with Python?

To protect your credentials using environment variables with Python, avoid hardcoding them in your source code. Instead, use environment variables to store sensitive information. This method keeps your credentials safe by not including them in your application's source code, reducing the risk of exposing them through version control or other means.