Which toml package to use in python?

See also: toml-lang and PEP 680

The verions of the packages tested in this report.

How the package dumps None value in python

Literally <package>.dumps(None)

How the package dumps key-value pair with value None

Literally <package>.dumps({"key": None})

How the package dumps a list with None value in it.

Literally <package>.dumps({"key": [1, 2, 3, None, 5]})

How the package loads None-like value in string

Literally <package>.loads('v1 = "null" v2 = "None"')

How the package dumps a python dictionary with a heterogenous array.

Literally <package>.dumps({"v": [1, 1.2, True, "string"]})

How the package loads a toml string with a heterogenous array.

Literally <package>.loads('v = [1, 1.2, True, "string"]')

How the package dumps a python dictionary with a nested array.

Literally <package>.dumps({"v": [[1], [1, 2]]})

How the package loads a toml string with a nested array.

Literally <package>.loads('v = [[1], [1, 2]]')

Whether the package preserves the order of the keys while dumps a python dictionary.

Thus, whether <package>.dumps({"c": 1, "a": 2, "b": 3}) yields a string like c = 1\na = 2\nb = 3\n.

Whether the package preserves the order of the keys while loads a TOML string.

Thus, whether <package>.loads('c = 1\na = 2\nb = 3\n') yields a dictionary with keys in the order of ['c', 'a', 'b'].

How the package dumps Unicode in python

Literally, <package>.dumps({"你好": "世界"})

How the package loads a file with unicode.

The file was created by:

## Create a file with unicode content
with open(self.datafile, "w", encoding="utf-8") as f:
    f.write('"你好" = "世界"\n')

## Use `<package>.load()` to load the file
with open(self.datafile, "r", encoding="utf-8") as f:
    loaded = <package>.load(f)

Test the compliance with the standard test suites for valid toml files here:

https://github.com/BurntSushi/toml-test/

The tests come up with a JSON counterpart that can be used to valid whether loading the toml file yields the same result as the JSON counterpart.

Test the compliance with the standard test suites for invalid toml files here:

https://github.com/BurntSushi/toml-test/

• Not OK: The toml file is parsed without error, but expected to fail.
• OK: All files are failed to parse, as expected. Showing the last parsing error.

Test the compliance with python tomllib test data (since python 3.11) for valid toml files here:

https://github.com/python/cpython/tree/3.11/Lib/test/test_tomllib/data/valid

The tests come up with a JSON counterpart that can be used to valid whether loading the toml file yields the same result as the JSON counterpart.

Test the compliance with python tomllib test data (since python 3.11) for invalid toml files here:

https://github.com/python/cpython/tree/main/Lib/test/test_tomllib/data/invalid

• Not OK: The toml file is parsed without error, but expected to fail.
• OK: All files are failed to parse, as expected. Showing the last parsing error.

Test the speed of loading and dumping the loaded using data provided by pytomlpp

https://github.com/bobfang1992/pytomlpp/raw/master/benchmark/data.toml

Test the speed of loading and dumping the loaded using data provided by rtoml

https://github.com/samuelcolvin/rtoml/raw/main/tests/data.toml

Test the speed of loading and dumping the loaded using data provided by tomli

https://github.com/hukkin/tomli/raw/master/benchmark/data.toml

• Tests with toml-test v1.2.0
• Tests with toml-test v1.1.0
• Tests with toml-test v1.0.0
• Tests with python 3.11 (tomllib included)

pip install -U toml-bench

toml-bench

toml-bench --datadir /tmp/toml-bench

toml-bench --report ./README.md

toml-bench --comver 1.0.0

toml-bench --iter 5000

git clone https://github.com/pwwang/toml-bench.git
cd toml-bench
# See https://python-poetry.org/docs/cli/#add
# for how to specify a version constraint
poetry add "tomli=2.0.0"
poetry update
poetry install
poetry run toml-bench