line_profiler.explicit_profiler module

New in line_profiler version 4.1.0, this module defines a top-level profile decorator which will be disabled by default unless a script is being run with kernprof, if the environment variable LINE_PROFILE is set, or if --line-profile is given on the command line.

In the latter two cases, the atexit module is used to display and dump line profiling results to disk when Python exits.

If none of the enabling conditions are met, then line_profiler.profile is a no-op. This means you no longer have to add and remove the implicit profile decorators required by previous version of this library.

Basic usage is to import line_profiler and decorate your function with line_profiler.profile. By default this does nothing, it’s a no-op decorator. However, if you run with the environment variable LINE_PROFILER=1 or if '--profile' in sys.argv, then it enables profiling and at the end of your script it will output the profile text.

Here is a minimal example that will write a script to disk and then run it with profiling enabled or disabled by various methods:

# Write demo python script to disk
python -c "if 1:
    import textwrap
    text = textwrap.dedent(
        '''
        from line_profiler import profile

        @profile
        def plus(a, b):
            return a + b

        @profile
        def fib(n):
            a, b = 0, 1
            while a < n:
                a, b = b, plus(a, b)

        @profile
        def main():
            import math
            import time
            start = time.time()

            print('start calculating')
            while time.time() - start < 1:
                fib(10)
                math.factorial(1000)
            print('done calculating')

        main()
        '''
    ).strip()
    with open('demo.py', 'w') as file:
        file.write(text)
"

echo "---"
echo "## Base Case: Run without any profiling"
python demo.py

echo "---"
echo "## Option 0: Original Usage"
python -m kernprof -l demo.py
python -m line_profiler -rmt demo.py.lprof

echo "---"
echo "## Option 1: Enable profiler with the command line"
python demo.py --line-profile

echo "---"
echo "## Option 1: Enable profiler with an environment variable"
LINE_PROFILE=1 python demo.py

The explicit line_profiler.profile decorator can also be enabled and configured in the Python code itself by calling line_profiler.profile.enable(). The following example demonstrates this:

# In-code enabling
python -c "if 1:
    import textwrap
    text = textwrap.dedent(
        '''
        from line_profiler import profile
        profile.enable(output_prefix='customized')

        @profile
        def fib(n):
            a, b = 0, 1
            while a < n:
                a, b = b, a + b

        fib(100)
        '''
    ).strip()
    with open('demo.py', 'w') as file:
        file.write(text)
"
echo "## Configuration handled inside the script"
python demo.py

Likewise there is a line_profiler.profile.disable() function that will prevent any subsequent functions decorated with @profile from being profiled. In the following example, profiling information will only be recorded for func2 and func4.

# In-code enabling / disable
python -c "if 1:
    import textwrap
    text = textwrap.dedent(
        '''
        from line_profiler import profile

        @profile
        def func1():
            return list(range(100))

        profile.enable(output_prefix='custom')

        @profile
        def func2():
            return tuple(range(100))

        profile.disable()

        @profile
        def func3():
            return set(range(100))

        profile.enable()

        @profile
        def func4():
            return dict(zip(range(100), range(100)))

        print(type(func1()))
        print(type(func2()))
        print(type(func3()))
        print(type(func4()))
        '''
    ).strip()
    with open('demo.py', 'w') as file:
        file.write(text)
"

echo "---"
echo "## Configuration handled inside the script"
python demo.py

# Running with --line-profile will also profile ``func1``
python demo.py --line-profile

The core functionality in this module was ported from xdev.

class line_profiler.explicit_profiler.GlobalProfiler(config: ConfigArg = None)

Bases: object

Manages a profiler that will output on interpreter exit.

The line_profile.profile decorator is an instance of this object.

Parameters:

config (str | PurePath | bool | None) – Optional TOML config file from which to load the configurations (see Attributes); if not explicitly given (= True or None), it is either resolved from the LINE_PROFILER_RC environment variable or looked up among the current directory or its ancestors. Should all that fail, the default config file at importlib.resources.path('line_profiler.rc', 'line_profiler.toml') is used; passing False disables all lookup and falls back to the default configuration

Variables:

• setup_config (Dict[str, List[str]]) – Determines how the implicit setup behaves by defining which environment variables / command line flags to look for. Defaults to the [tool.line_profiler.setup] table of the loaded config file.

• output_prefix (str) – The prefix of any output files written. Should include a part of a filename. Defaults to the output_prefix value in the [tool.line_profiler.write] table of the loaded config file.

• write_config (Dict[str, bool]) – Which outputs are enabled; options are lprof, text, timestamped_text, and stdout. Defaults to the rest of the [tool.line_profiler.write] table of the loaded config file.

• show_config (Dict[str, bool]) – Display configuration options; some outputs force certain options (e.g. text always has details and is never rich). Defaults to the rest of the [tool.line_profiler.show] table of the loaded config file, excluding the [tool.line_profiler.show.column_widths] subtable.

• enabled (bool | None) – True if the profiler is enabled (i.e. if it will wrap a function that it decorates with a real profiler). If None, then the value defaults based on the setup_config, os.environ, and sys.argv.

Example

>>> from line_profiler.explicit_profiler import *  # NOQA
>>> self = GlobalProfiler()
>>> # Setting the _profile attribute prevents atexit from running.
>>> self._profile = LineProfiler()
>>> # User can personalize the configuration
>>> self.show_config['details'] = True
>>> self.write_config['lprof'] = False
>>> self.write_config['text'] = False
>>> self.write_config['timestamped_text'] = False
>>> # Demo data: a function to profile
>>> def collatz(n):
...     while n != 1:
...         if n % 2 == 0:
...             n = n // 2
...         else:
...             n = 3 * n + 1
...     return n
>>> # Disabled by default, implicitly checks to auto-enable on first wrap
>>> assert self.enabled is None
>>> wrapped = self(collatz)
>>> assert self.enabled is False
>>> assert wrapped is collatz
>>> # Can explicitly enable
>>> self.enable()
>>> wrapped = self(collatz)
>>> assert self.enabled is True
>>> assert wrapped is not collatz
>>> wrapped(100)
>>> # Can explicitly request output
>>> self.show()

enabled: bool | None

setup_config: dict[str, list[str]]

write_config: dict[str, Any]

output_prefix: str

show_config: dict[str, Any]

enable(output_prefix: str | None = None) → None

Explicitly enables global profiler and controls its settings.

Notes

Multiprocessing start methods like ‘spawn’/’forkserver’ can create helper/bootstrap interpreters that import this module. Those helpers must not claim ownership or register an atexit hook, otherwise they can clobber output from the real script process.

disable() → None

Explicitly initialize and disable this global profiler.

__call__(func: Callable) → Callable

If the global profiler is enabled, decorate a function to start the profiler on function entry and stop it on function exit. Otherwise return the input.

Parameters:

func (Callable) – the function to profile

Returns:

a potentially wrapped function

Return type:

Callable

show() → None

Write the managed profiler stats to enabled outputs.

If the implicit setup triggered, then this will be called by atexit.

line_profiler.explicit_profiler.is_mp_bootstrap() → bool

True when this interpreter invocation looks like multiprocessing bootstrapping/plumbing, where we must not claim ownership / write outputs.

Example

>>> # xdoctest: +SKIP('can be flaky at test time')
>>> import pytest
>>> if is_mp_bootstrap():
...     pytest.skip('Cannot test mp bootstrap detection from within an mp bootstrap process')
>>> import sys, subprocess, textwrap
>>> code = textwrap.dedent(r'''
... import multiprocessing as mp
... from line_profiler.explicit_profiler import is_mp_bootstrap
...
... def child(q):
...     q.put(is_mp_bootstrap())
...
... if __name__ == "__main__":
...     ctx = mp.get_context("spawn")
...     q = ctx.Queue()
...     p = ctx.Process(target=child, args=(q,))
...     p.start()
...     val = q.get()
...     p.join()
...     print(val)
... ''')
>>> out = subprocess.check_output([sys.executable, "-c", code], text=True).strip()
>>> out in {"True", "False"}
True