poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions. It does so by parsing pyproject.toml and poetry.lock and converting them to Nix derivations on the fly.

For more information, see the announcement post on the Tweag blog.

• API
• FAQ
• How-to guides
• Using the flake
• Contributing
• Contact
• Acknowledgements
• License

The poetry2nix public API consists of the following attributes:

• mkPoetryApplication: Creates a Python application.
• mkPoetryEnv: Creates a Python environment with an interpreter and all packages from poetry.lock.
• mkPoetryPackages: Creates an attribute set providing access to the generated packages and other artifacts.
• mkPoetryScriptsPackage: Creates a package containing the scripts from tool.poetry.scripts of the pyproject.toml.
• mkPoetryEditablePackage: Creates a package containing editable sources. Changes in the specified paths will be reflected in an interactive nix-shell session without the need to restart it.
• defaultPoetryOverrides: A set of bundled overrides fixing problems with Python packages.
• overrides.withDefaults: A convenience function for specifying overrides on top of the defaults.
• overrides.withoutDefaults: A convenience function for specifying overrides without defaults.
• cleanPythonSources: A function to create a source filter for python projects.

Creates a Python application using the Python interpreter specified based on the designated poetry project and lock files. mkPoetryApplication takes an attribute set with the following attributes (attributes without default are mandatory):

• projectDir: path to the root of the project.
• src: project source (default: cleanPythonSources { src = projectDir; }).
• pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
• poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
• overrides: Python overrides to apply (default: defaultPoetryOverrides).
• meta: application meta data (default: {}).
• python: The Python interpreter to use (default: pkgs.python3).
• preferWheels : Use wheels rather than sdist as much as possible (default: false).
• groups: Which Poetry 1.2.0+ dependency groups to install (default: [ ]).
• checkGroups: Which Poetry 1.2.0+ dependency groups to run unit tests (default: [ "dev" ]).
• extras: Which Poetry extras to install (default: [ "*" ], all extras).

Other attributes are passed through to buildPythonApplication.

Make sure to add in your pyproject.toml the py-object for your main(). Otherwise, the result is empty.

[tool.poetry.scripts]
poetry = "poetry.console.application:main"

poetry2nix.mkPoetryApplication {
    projectDir = ./.;
}

The resulting derivation also has the passthru attribute dependencyEnv, which is an environment with a python interpreter, all non-development dependencies and your application. This can be used if your application doesn't provide any binaries on its own and instead relies on dependencies binaries to call its modules (as is often the case with celery or gunicorn). For example, if your application defines a CLI for the module admin and a gunicorn app for the module web, a working default.nix would contain

let
    app = poetry2nix.mkPoetryApplication {
        projectDir = ./.;
    };
in app.dependencyEnv

After building this expression, your CLI and app can be called with these commands

$ result/bin/python -m admin
$ result/bin/gunicorn web:app

If you prefer to build a single binary that runs gunicorn web:app, use pkgs.writeShellApplication for a simple wrapper.

Note: If you need to perform overrides on the application, use app.dependencyEnv.override { app = app.override { ... }; }. See ./tests/dependency-environment/default.nix for a full example.

Creates an environment that provides a Python interpreter along with all dependencies declared by the designated poetry project and lock files. Also allows package sources of an application to be installed in editable mode for fast development. mkPoetryEnv takes an attribute set with the following attributes (attributes without default are mandatory):

• projectDir: path to the root of the project.
• pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
• poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
• overrides: Python overrides to apply (default: defaultPoetryOverrides).
• python: The Python interpreter to use (default: pkgs.python3).
• editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode. Note that path dependencies with develop = true will be installed in editable mode unless explicitly passed to editablePackageSources as null. (default: {}).
• extraPackages: A function taking a Python package set and returning a list of extra packages to include in the environment. This is intended for packages deliberately not added to pyproject.toml that you still want to include. An example of such a package may be pip. (default: (ps: [ ])).
• preferWheels : Use wheels rather than sdist as much as possible (default: false).
• groups: Which Poetry 1.2.0+ dependency groups to install (default: [ "dev" ]).
• checkGroups: Which Poetry 1.2.0+ dependency groups to run unit tests (default: [ "dev" ]).
• extras: Which Poetry extras to install (default: [ "*" ], all extras).

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
}

See ./tests/env/default.nix for a working example.

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
        my-app = ./src;
    };
}

See ./tests/editable/default.nix for a working example of an editable package.

The env attribute of the attribute set created by mkPoetryEnv contains a shell environment.

{ pkgs ? import <nixpkgs> {} }:
let
  myAppEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
      my-app = ./src;
    };
  };
in myAppEnv.env

For a shell environment including external dependencies, override the env to add dependency packages (for example, pkgs.hello) as build inputs.

{ pkgs ? import <nixpkgs> {} }:
let
  myAppEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
      my-app = ./src;
    };
  };
in myAppEnv.env.overrideAttrs (oldAttrs: {
  buildInputs = [ pkgs.hello ];
})

Creates an attribute set of the shape { python, poetryPackages, pyProject, poetryLock }. Where python is the interpreter specified, poetryPackages is a list of all generated python packages, pyProject is the parsed pyproject.toml and poetryLock is the parsed poetry.lock file. mkPoetryPackages takes an attribute set with the following attributes (attributes without default are mandatory):

• projectDir: path to the root of the project.
• pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
• poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
• overrides: Python overrides to apply (default: defaultPoetryOverrides).
• python: The Python interpreter to use (default: pkgs.python3).
• editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).
• preferWheels : Use wheels rather than sdist as much as possible (default: false).
• groups: Which Poetry 1.2.0+ dependency groups to install (default: [ ]).
• checkGroups: Which Poetry 1.2.0+ dependency groups to run unit tests (default: [ "dev" ]).
• extras: Which Poetry extras to install (default: [ "*" ], all extras).

poetry2nix.mkPoetryPackages {
    projectDir = ./.;
    python = python35;
}

Creates a package containing the scripts from tool.poetry.scripts of the pyproject.toml:

• projectDir: path to the root of the project.
• pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
• python: The Python interpreter to use (default: pkgs.python3).

poetry2nix.mkPoetryScriptsPackage {
    projectDir = ./.;
    python = python35;
}

Creates a package containing editable sources. Changes in the specified paths will be reflected in an interactive nix-shell session without the need to restart it:

• projectDir: path to the root of the project.
• pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
• python: The Python interpreter to use (default: pkgs.python3).
• editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

poetry2nix.mkPoetryEditablePackage {
    projectDir = ./.;
    python = python35;
    editablePackageSources = {
        my-app = ./src;
    };
}

poetry2nix bundles a set of default overrides that fix problems with various Python packages. These overrides are implemented in overrides.

Returns a list containing the specified overlay and defaultPoetryOverrides.

Takes an attribute set with the following attributes (attributes without default are mandatory):

• src: project source directory

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetry2nix.overrides.withDefaults (self: super: { foo = null; });
}

See ./tests/override-support/default.nix for a working example.

Returns a list containing just the specified overlay, ignoring defaultPoetryOverrides.

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetry2nix.overrides.withoutDefaults (self: super: { foo = null; });
}

Provides a source filtering mechanism that:

• Filters gitignore's
• Filters pycache/pyc files
• Uses cleanSourceFilter to filter out .git/.hg, .o/.so, editor backup files & nix result symlinks

poetry2nix.cleanPythonSources {
    src = ./.;
}

Sometimes when it can be convenient to create a custom instance of poetry2nix with a different set of default overrides.

let
  # self & super refers to poetry2nix
  p2nix = poetry2nix.overrideScope' (self: super: {

    # pyself & pysuper refers to python packages
    defaultPoetryOverrides = super.defaultPoetryOverrides.extend (pyself: pysuper: {

      my-custom-pkg = super.my-custom-pkg.overridePythonAttrs (oldAttrs: { });

    });

  });

in
p2nix.mkPoetryApplication {
  projectDir = ./.;
}

or as a nixpkgs overlay:

let
  pkgs = import <nixpkgs> {
    overlays = [
      # self & super refers to nixpkgs
      (self: super: {

        # p2self & p2super refers to poetry2nix
        poetry2nix = super.poetry2nix.overrideScope' (p2nixself: p2nixsuper: {

          # pyself & pysuper refers to python packages
          defaultPoetryOverrides = p2nixsuper.defaultPoetryOverrides.extend (pyself: pysuper: {

            my-custom-pkg = super.my-custom-pkg.overridePythonAttrs (oldAttrs: { });

          });

        });
      })

    ];
  };

in pkgs.poetry2nix.mkPoetryApplication {
  projectDir = ./.;
}

Q. Does poetry2nix install wheels our sdists?

A. By default, poetry2nix installs from source. If you want to give precedence to wheels, look at the preferWheel and preferWheels attributes.

Q. Does poetry2nix use package definitions from nixpkgs' Python package set?

A. poetry2nix overlays packages taken from the poetry.lock file on top of nixpkgs, in such a way that overlaid packages in nixpkgs are completely ignored. Any package that is used, but isn't in the poetry.lock file (most commonly build dependencies) is taken from nixpkgs.

Q. How to prefer wheel installation for a single package?

A. Override it and set preferWheel = true in that single package:

poetry2nix.mkPoetryApplication {
  projectDir = ./.;
  overrides = poetry2nix.overrides.withDefaults (final: prev: {
    # Notice that using .overridePythonAttrs or .overrideAttrs wont work!
    some-dependency = prev.some-dependency.override {
      preferWheel = true;
    };
  });
}

Q. I'm experiencing one of the following errors, what do I do?

• ModuleNotFoundError: No module named 'setuptools'
• ModuleNotFoundError: No module named 'pdm'
• ModuleNotFoundError: No module named 'setuptools-scm'
• ModuleNotFoundError: No module named 'poetry-core'
• ModuleNotFoundError: No module named 'flit'
• ModuleNotFoundError: No module named 'flit-core'
• ModuleNotFoundError: No module named 'flit-scm'

A. Have a look at the following document edgecase.md

• Package and deploy Python apps faster with Poetry and Nix This is a short (11 minutes) video tutorial by Steve Purcell from Tweag walking you through how to get started with a small web development project.

For the experimental flakes functionality we provide poetry2nix as a flake providing an overlay to use with nixpkgs. Additionally, the flake provides a flake template to quickly start using poetry2nix in a project:

nix flake init --template github:nix-community/poetry2nix

Contributions to this project are welcome in the form of GitHub PRs. Please consider the following before creating PRs:

• This project uses nixpkgs-fmt for formatting the Nix code. You can use nix-shell --run "nixpkgs-fmt ." to format everything.
• If you are planning to make any considerable changes, you should first present your plans in a GitHub issue so it can be discussed.
• If you add new features please consider adding tests. You can run them locally as follows:

nix-build --keep-going --show-trace tests/default.nix

To list test names:

nix eval --impure --expr 'let pkgs = import <nixpkgs> {}; in pkgs.lib.attrNames (import ./tests/default.nix {})'

To run specific tests, add --attr NAME to the nix-build command above. For example, to run the bcrypt and jq tests:

nix-build --attr bcrypt --attr jq --keep-going --show-trace tests/default.nix

To test with a specific channel:

nix-build --expr 'with import <unstable> {}; callPackage ./tests/default.nix {}'

To sort overrides/build-systems.json according to the sort-build-systems job, patch the source with the output of the "Check format" step, like this: nix-shell [omitted] | patch -p0.

We have a Matrix room at #poetry2nix:blad.is.

Development of poetry2nix has been supported by Tweag.

poetry2nix is released under the terms of the MIT license.