Usage

Installation

This plugin can be installed using pip:

pip install pytest-memray

pytest-memray is a pytest plugin. It is enabled when you pass --memray to pytest:

pytest tests/ --memray

Allocation tracking

By default, the plugin will track allocations at the high watermark in all tests. This information is reported after tests run ends:

$ pytest --memray /w/demo 
Traceback (most recent call last):
  File "/v/bin/pytest", line 8, in <module>
    sys.exit(_console_main())
             ~~~~~~~~~~~~~^^
  File "/v/share/python3.13/site-packages/_pytest/config/__init__.py", line 253, in _console_main
    code = _main(prog=_get_prog_name(sys.argv))
  File "/v/share/python3.13/site-packages/_pytest/config/__init__.py", line 223, in _main
    config = _prepareconfig(new_args, plugins, prog=prog)
  File "/v/share/python3.13/site-packages/_pytest/config/__init__.py", line 410, in _prepareconfig
    config: Config = pluginmanager.hook.pytest_cmdline_parse(
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        pluginmanager=pluginmanager, args=args
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/v/share/python3.13/site-packages/pluggy/_caller.py", line 197, in __call__
    return self._hookexec(
           ~~~~~~~~~~~~~~^
        self.name, self._hookimpls.copy(), call_kwargs, firstresult
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/v/share/python3.13/site-packages/pluggy/_manager.py", line 183, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/v/share/python3.13/site-packages/pluggy/_execution.py", line 195, in _multicall
    raise exception
  File "/v/share/python3.13/site-packages/pluggy/_execution.py", line 167, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/v/share/python3.13/site-packages/_pytest/helpconfig.py", line 124, in pytest_cmdline_parse
    config = yield
             ^^^^^
  File "/v/share/python3.13/site-packages/pluggy/_execution.py", line 147, in _multicall
    res = hook_impl.function(*args)
  File "/v/share/python3.13/site-packages/_pytest/config/__init__.py", line 1232, in pytest_cmdline_parse
    self.parse(args)
    ~~~~~~~~~~^^^^^^
  File "/v/share/python3.13/site-packages/_pytest/config/__init__.py", line 1583, in parse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/v/share/python3.13/site-packages/pluggy/_manager.py", line 521, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/v/lib64/python3.13/importlib/metadata/__init__.py", line 179, in load
    module = import_module(match.group('module'))
  File "/v/lib64/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1404, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1369, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1312, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1404, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1369, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1340, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "/v/share/python3.13/site-packages/_pytest/assertion/rewrite.py", line 188, in exec_module
    exec(co, module.__dict__)
    ~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/v/share/python3.13/site-packages/case/__init__.py", line 12, in <module>
    from . import skip
  File "/v/share/python3.13/site-packages/_pytest/assertion/rewrite.py", line 188, in exec_module
    exec(co, module.__dict__)
    ~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/v/share/python3.13/site-packages/case/skip.py", line 7, in <module>
    from nose import SkipTest
  File "/v/share/python3.13/site-packages/nose/__init__.py", line 2, in <module>
    from nose.core import collector, main, run, run_exit, runmodule
  File "/v/share/python3.13/site-packages/nose/core.py", line 8, in <module>
    from nose.loader import defaultTestLoader
  File "/v/share/python3.13/site-packages/nose/loader.py", line 17, in <module>
    from nose.importer import Importer, add_path, remove_path
  File "/v/share/python3.13/site-packages/nose/importer.py", line 14, in <module>
    from importlib._bootstrap import _ERR_MSG, _builtin_from_name
ImportError: cannot import name '_ERR_MSG' from 'importlib._bootstrap' (/v/lib64/python3.13/importlib/_bootstrap.py)

Markers

This plugin provides markers that can be used to enforce additional checks and validations on tests.

pytest.mark.limit_memory(memory_limit: str, current_thread_only: bool = False)

Fail the execution of the test if the test allocates more peak memory than allowed.

When this marker is applied to a test, it will cause the test to fail if the execution of the test allocates more memory (at the peak/high watermark) than allowed. It takes a single argument with a string indicating the maximum memory that the test can allocate.

The format for the string is <NUMBER> ([KMGTP]B|B). The marker will raise ValueError if the string format cannot be parsed correctly.

If the optional keyword-only argument current_thread_only is set to True, the plugin will only track memory allocations made by the current thread and all other allocations will be ignored.

Warning

As the Python interpreter has its own object allocator it’s possible that memory is not immediately released to the system when objects are deleted, so tests using this marker may need to give some room to account for this.

Example of usage:

@pytest.mark.limit_memory("24 MB")
def test_foobar():
    pass  # do some stuff that allocates memory
pytest.mark.limit_leaks(location_limit: str, filter_fn: LeaksFilterFunction | None = None, current_thread_only: bool = False)

Fail the execution of the test if any call stack in the test leaks more memory than allowed.

Important

To detect leaks, Memray needs to intercept calls to the Python allocators and report native call frames. This is adds significant overhead, and will slow your test down.

When this marker is applied to a test, the plugin will analyze the memory allocations that are made while the test body runs and not freed by the time the test body function returns. It groups them by the call stack leading to the allocation, and sums the amount leaked by each distinct call stack. If the total amount leaked from any particular call stack is greater than the configured limit, the test will fail.

Important

It’s recommended to run your API or code in a loop when utilizing this plugin. This practice helps in distinguishing genuine leaks from the “noise” generated by internal caches and other incidental allocations.

The format for the string is <NUMBER> ([KMGTP]B|B). The marker will raise ValueError if the string format cannot be parsed correctly.

The marker also takes an optional keyword-only argument filter_fn. This argument represents a filtering function that will be called once for each distinct call stack that leaked more memory than allowed. If it returns True, leaks from that location will be included in the final report. If it returns False, leaks associated with the stack it was called with will be ignored. If all leaks are ignored, the test will not fail. This can be used to discard any known false positives.

If the optional keyword-only argument current_thread_only is set to True, the plugin will only track memory allocations made by the current thread and all other allocations will be ignored.

Tip

You can pass the --memray-bin-path argument to pytest to specify a directory where Memray will store the binary files with the results. You can then use the memray CLI to further investigate the allocations and the leaks using any Memray reporters you’d like. Check the memray docs for more information.

Example of usage:

@pytest.mark.limit_leaks("1 MB")
def test_foobar():
    # Run the function we're testing in a loop to ensure
    # we can differentiate leaks from memory held by
    # caches inside the Python interpreter.
    for _ in range(100):
        do_some_stuff()

Warning

It is very challenging to write tests that do not “leak” memory in some way, due to circumstances beyond your control.

There are many caches inside the Python interpreter itself. Just a few examples:

  • The re module caches compiled regexes.

  • The logging module caches whether a given log level is active for a particular logger the first time you try to log something at that level.

  • A limited number of objects of certain heavily used types are cached for reuse so that object.__new__ does not always need to allocate memory.

  • The mapping from bytecode index to line number for each Python function is cached when it is first needed.

There are many more such caches. Also, within pytest, any message that you log or print is captured, so that it can be included in the output if the test fails.

Memray sees these all as “leaks”, because something was allocated while the test ran and it was not freed by the time the test body finished. We don’t know that it’s due to an implementation detail of the interpreter or pytest that the memory wasn’t freed. Morever, because these caches are implementation details, the amount of memory allocated, the call stack of the allocation, and even the allocator that was used can all change from one version to another.

Because of this, you will almost certainly need to allow some small amount of leaked memory per call stack, or use the filter_fn argument to filter out false-positive leak reports based on the call stack they’re associated with.

pytest.mark.limit_leaked_objects(filter_fn: LeakedObjectsFilterFunction | None = None)

Fail the execution of the test if any Python objects created while the test body runs are still alive at the end of the test.

This is only possible for Python 3.13.3 and newer. Using this marker with older Python versions will cause an error during test collection. You can use pytest.mark.skipif to prevent the test from running if the Python version is too old, or you can conditionally add the limit_leaked_objects marker only if the Python version is new enough.

The marker takes an optional keyword-only argument filter_fn that can be used to ignore certain leaked objects. This argument represents a filtering function that will be called once for each object leaked by the test body. If it returns True, leaks of that object will be included in the final report. If it returns False, that leaked object will be ignored. If all leaks are ignored, the test will not fail. This can be used to discard any known false positives.

Warning

It is very challenging to write tests that do not “leak” memory in some way, due to circumstances beyond your control.

There are many caches inside the Python interpreter itself. Just a few examples:

  • The re module caches compiled regexes.

  • The logging module caches whether a given log level is active for a particular logger the first time you try to log something at that level.

There are many more such caches. Also, within pytest, any message that you log or print is captured, so that it can be included in the output if the test fails.

Memray sees these all as “leaks”, because something was allocated while the test ran and it was not freed by the time the test body finished. We don’t know that it’s due to an implementation detail of the standard library or pytest that the memory wasn’t freed. Morever, because these caches are implementation details, they can change from one Python version to another.

Because of this, you will need to very carefully design your test to avoid objects being cached, or use the filter_fn argument to filter out false-positive leak reports.