Benchmarks

Introduction

Benchmarks allow to track performance from release to release and verify that latest changes haven not affected it drastically. Benchmarks are based on the pyperf:index.

How to run

requirements/dev.txt should be installed before we can proceed with benchmarks. Please also make sure that you have configured your OS to have reliable results.

To run benchmarks next command can be executed:

$ python benchmarks/benchmark.py

This would run benchmarks for both classes (MultiDict and CIMultiDict) of both implementations (pure-Python and C).

To run benchmarks for a specific class of specific implementation please use --impl option:

$ python benchmarks/benchmark.py --impl multidict_c

would run benchmarks only for MultiDict implemented in C.

Please use --help to see all available options. Most of the options are described at perf’s Runner documentation.

How to compare implementations

--impl option allows to run benchmarks for a specific implementation of class. Combined with the compare_to command of pyperf:index we can get a good picture of how implementation performs:

$ python benchmarks/benchmark.py --impl multidict_c -o multidict_cy.json
$ python benchmarks/benchmark.py --impl multidict_py -o multidict_py.json
$ python -m perf compare_to multidict_cy.json multidict_py.json