Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 704 – Require virtual environments by default for package installers

Author:
Pradyun Gedam <pradyunsg at gmail.com>
Sponsor:
Brett Cannon <brett at python.org>
PEP-Delegate:
Paul Moore <p.f.moore at gmail.com>
Discussions-To:
Discourse thread
Status:
Withdrawn
Type:
Standards Track
Topic:
Packaging
Created:
16-Jan-2023
Post-History:
16-Jan-2023

Table of Contents

Abstract

This PEP recommends that package installers like pip require a virtual environment by default on Python 3.13+.

PEP Withdrawal

During discussion of this PEP, it became clear that changes to pip’s UX are not controlled by PEPs as proposed. It also became clear that a significant number of users rely on being able to mix managed-by-pip dependencies with managed-by-some-other-tool dependencies (most prominently, when using Conda).

Further, a significant subset of the benefits of the proposed change are achievable via PEP 668 (accepted and implemented at the time of writing). It provides redistributors of the Python interpreter control over whether users should be required to use a virtual environment, what the messaging presented to the user is and how the rollout of that change should happen for their users.

Since enforcement of virtual environments with pip was the primary focus of this PEP, it felt more appropriate to withdraw this PEP than to refocus it on a different topic.

A future PEP to resolve the virtual environment naming convention question/issues would still be appropriate, but it’s worth starting any such effort as a fresh PEP focused on the benefits of such a convention, rather than on the enforcement of it.

Motivation

Python virtual environments are an essential part of the development workflow for Python. However, they require extra effort since they are an opt-in feature, and requires users to either:

  • take explicit steps to activate/deactivate a virtual environment
  • use <path-to-venv>/<bin-path>/<executable> to run files

For new users, things will seemingly work correctly when virtual environments are not used -— until they don’t. Further, activating a virtual environment uses slightly different syntax and mechanisms on different platforms. This complicates the introduction to virtual environments, since now information and context about how/why they are useful needs to explained to justify adding additional steps to the workflow.

It also creates a scope for mistakes, since users need to remember to activate the virtual environment before running an installer like pip or configure those installers to error out. On certain Linux distributions, forgetting to do so can result in the installer modifying files that are owned by the operating system (which is partially mitigated by PEP 668 for distributions that opt-in to marking their environments accordingly).

Rationale

Changing the default behaviour of installers like pip to require a virtual environment to be active would:

  • make it easier for new users to get started with Python (since there’s a consistent experience and virtual environments are understood as a thing you’re required to use)
  • reduce the scope for accidental installation issues for all users by default (by explicitly flagging when you’re not using a virtual environment).

Setting up a convention of placing the virtual environment in-tree in a directory named .venv removes a decision point for common workflows and creates a clear convention within the ecosystem.

Specification

Requiring a virtual environment by default

When a user runs an installer without an active virtual environment, the installer SHOULD print an error message and exit with a non-zero exit code.

The error message SHOULD inform the user that a virtual environment is required, SHOULD provide shell-specific instructions on how to create and activate a virtual environment named .venv, and SHOULD provide a link to a documentation page that explains how to create and activate a virtual environment.

See Implementation Notes for more details.

Opting out of virtual environments

The installer SHOULD also provide a explicit opt-in to disable this requirement, allowing an end user to use it outside of a virtual environment. If the installer does not provide this functionality, it SHOULD mention this in the error message and documentation page.

Consistent timeline for the change

Installers MAY choose to implement this default behaviour on any Python versions, but SHOULD implement it on Python 3.13 or newer.

Backwards Compatibility

This PEP is backwards incompatible with workflows where users are using installers outside of virtual environments. Such users will be prompted with an error message and will need to either:

  • explicitly opt-in to running the installer outside of a virtual environment, or
  • create and use a virtual environment

Users who are already using virtual environments will not be affected by this change.

Workflow tools (which manage virtual environments for the user, under the hood) should be unaffected, since they should already be using a virtual environment for running the installer.

Security Implications

This PEP does not introduce any new security implications.

How to Teach This

This PEP requires that new users create and use a virtual environment to get started with using Python packages. This is, however, a best practice, as demonstrated by the section on “basics of how to install Python packages” in the Python Packaging User Guide, which explains how/what virtual environments are before discussing using pip.

Reference Implementation

There is no reference implementation for this PEP. However, the proposed behaviour is largely already implemented in pip and can be activated by setting the PIP_REQUIRE_VENV environment variable to 1. (Leaving it unset results in the proposed opt-in behaviour of not requiring a virtual environment for installation.)

Implementation Notes

Detecting an active virtual environment

As discussed in PEP 668, the logic for robustly detecting a virtual environment is something like:

def is_virtual_environment():
    return sys.base_prefix != sys.prefix or hasattr(sys, "real_prefix")

Documentation on using a virtual environment

Package installers are expected to provide a link to a documentation page in the error message.

Ideally, such a documentation page would explain what virtual environments are, why they are required, and how to create and activate a virtual environment using venv. It should include instructions for the most common shells and platforms.

Such a documentation page should be made available in the Python Packaging User Guide to reduce duplicated effort across installers for covering this topic.

Rejected Ideas

Do not specify a name for the virtual environment directory

Using a consistent name for the virtual environment directory is important for a few reasons:

  1. It makes it easier for users to find the virtual environment directory, and to activate it.
  2. It removes a decision point for new users, since they do not need to decide on a name for the virtual environment directory.
  3. It creates a clear convention within the ecosystem, which makes it easier for users to find documentation.
  4. It ensures consistency across different tools, so that differences in the error messages do not confuse users.

Use a different name for the virtual environment directory

Functionally, the directory name does not matter much as long as there is a single consistent suggestion.

The name .venv was picked since it:

  1. does not conflict with any valid Python import name
  2. does not conflict venv module in the standard library
  3. has pre-existing usage in the Python community
  4. has support for auto-detection in common text editors
  5. can be typed without modifier keys on common keyboard layouts

Do not couple tooling behaviour with a Python version

This PEP creates a coupling between the behaviour of installers and the Python version.

This is already a rollout mechanism being used for behaviour changes in the installation tooling. For example, pip on Python 3.11 will use importlib.metadata instead of pkg_resources for parsing/fetching package metadata, and sysconfig instead of distutils.sysconfig for getting the paths to unpack wheels into.

The difference with those cases is that they’re supposed to be largely transparent to end users. This PEP is proposing a behaviour change that is not transparent to end users, and requires them to take action.

The primary benefit of this is that it allows for redistributors to adapt their tooling in time for the new Python version and provides a clear and consistent point for change across the ecosystem. It also puts a clear deadline on when the default behaviour will consistently require a virtual environment by default (once Python 3.12 goes end-of-life).

The primary issue with this approach is that it enforces a behaviour change on users when they upgrade to a new Python version, which can hamper the adoption of a new Python version. However, this is a migration/upgrade for existing users and it is a common expectation that some changes will be needed for migration/upgrades.

The author of this PEP believes that the benefits of applying this consistently throughout the ecosystem with a deadline outweigh the drawbacks of enforcing a best-practice on users when they upgrade.

Open Issues

None.


Source: https://github.com/python/peps/blob/main/peps/pep-0704.rst

Last modified: 2023-09-09 17:39:29 GMT