kernprof module

Script to conveniently run profilers on code in a variety of circumstances.

To profile a script, decorate the functions of interest with @profile:

echo "if 1:
    @profile
    def main():
        1 + 1
    main()
" > script_to_profile.py

Note

New in 4.1.0: Instead of relying on injecting @profile into the builtins you can now import line_profiler and use @line_profiler.profile to decorate your functions. This allows the script to remain functional even if it is not actively profiled. See line_profiler (link) for details.

Then run the script using kernprof:

kernprof -b script_to_profile.py

By default this runs with the default cProfile profiler and does not require compiled modules. Instructions to view the results will be given in the output. Alternatively, adding -v to the command line will write results to stdout.

To enable line-by-line profiling, line_profiler must be available and compiled, and the -l argument should be added to the kernprof invocation:

kernprof -lb script_to_profile.py

Note

New in 4.3.0: More code execution options are added:

• kernprof <options> -m some.module <args to module> parallels python -m and runs the provided module as __main__.

• kernprof <options> -c "some code" <args to code> parallels python -c and executes the provided literal code.

• kernprof <options> - <args to code> parallels python - and executes literal code passed via the stdin.

See also kernprof invocations.

For more details and options, refer to the CLI help. To view the kernprof help text run:

kernprof --help

which displays:

usage: kernprof [-h] [-V] [--config CONFIG] [--no-config]
                [--line-by-line [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [--builtin [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [-s SETUP] [-p {path/to/script | object.dotted.path}[,...]]
                [--preimports [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [--prof-imports [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [-o OUTFILE] [-v] [-q]
                [--rich [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [-u UNIT]
                [--skip-zero [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [--summarize [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]]
                [-i [OUTPUT_INTERVAL]]
                {path/to/script | -m path.to.module | -c "literal code"} ...

Run and profile a python script or module.

positional arguments:
  {path/to/script | -m path.to.module | -c "literal code"}
                        The python script file, module, or literal code to run
  args                  Optional script arguments

options:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit
  --config CONFIG       Path to the TOML file, from the `tool.line_profiler.kernprof`
                        table of which to load defaults for the options. (Default:
                        'pyproject.toml')
  --no-config           Disable the loading of configuration files other than the
                        default one

profiling options:
  --line-by-line [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        Use the line-by-line profiler instead of cProfile. Implies
                        `--builtin`. (Default: False; short form: -l)
  --builtin [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        Put `profile` in the builtins. Use
                        `profile.enable()`/`.disable()` to toggle profiling,
                        `@profile` to decorate functions, or `with profile:` to
                        profile a section of code. (Default: False; short form: -b)
  -s, --setup SETUP     Path to the Python source file containing setup code to
                        execute before the code to profile. (Default: N/A)
  -p, --prof-mod PROF_MOD
                        List of modules, functions and/or classes to profile specified
                        by their name or path. These profiling targets can be supplied
                        both as comma-separated items, or separately with multiple
                        copies of this flag. Packages are automatically recursed into
                        unless they are specified with `<pkg>.__init__`. Adding the
                        current script/module profiles the entirety of it. Only works
                        with line profiling (`-l`/`--line-by-line`). (Default: N/A;
                        pass an empty string to clear the defaults (or any `-p` target
                        specified earlier)
  ---preimports [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        Instead of eagerly importing all profiling targets specified
                        via `-p` and profiling them, only profile those that are
                        directly imported in the profiled code. Only works with
                        line profiling (`-l`/`--line-by-line`). (Default: False)
                        Eagerly import all profiling targets specified via `-p` and
                        profile them, instead of only profiling those that are
                        directly imported in the profiled code. Only works with line
                        profiling (`-l`/`--line-by-line`). (Default: True)
  --prof-imports [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        If the script/module profiled is in `--prof-mod`, autoprofile
                        all its imports. Only works with line profiling (`-l`/`--line-
                        by-line`). (Default: False)

output options:
  -o, --outfile OUTFILE
                        Save stats to OUTFILE. (Default:
                        '<script_or_module_name>.lprof' in line-profiling mode
                        (`-l`/`--line-by-line`); '<script_or_module_name>.prof'
                        otherwise)
  -v, --verbose, --view
                        Increase verbosity level (default: 0). At level 1, view the
                        profiling results in addition to saving them; at level 2,
                        show other diagnostic info.
  -q, --quiet           Decrease verbosity level (default: 0). At level -1, disable
                        helpful messages (e.g. "Wrote profile results to <...>"); at
                        level -2, silence the stdout; at level -3, silence the stderr.
  --rich [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        Use rich formatting if viewing output. (Default: False; short
                        form: -r)
  -u, --unit UNIT       Output unit (in seconds) in which the timing info is
                        displayed. (Default: 1e-06 s)
  --skip-zero [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        Hide functions which have not been called. (Default: False;
                        short form: -z)
  --summarize [Y[es] | N[o] | T[rue] | F[alse] | on | off | 1 | 0]
                        Print a summary of total function time. (Default: False)
  -i, --output-interval [OUTPUT_INTERVAL]
                        Enables outputting of cumulative profiling results to OUTFILE
                        every OUTPUT_INTERVAL seconds. Uses the threading module.
                        Minimum value (and the value implied if the bare option is
                        given) is 1 s. (Default: 0 s (disabled))

Note

New in 5.0.0: For more intuitive profiling behavior, profiling targets in --prof-mod (except the profiled script/code) are now:

• Eagerly pre-imported to be profiled (see line_profiler.autoprofile.eager_preimports), regardless of whether those imports directly occur in the profiled script/module/code.

• Descended/Recursed into if they are packages; pass <pkg_name>.__init__ instead of <pkg_name> to curtail descent and limit profiling to classes and functions in the local namespace of the __init__.py.

To restore the old behavior, pass the --no-preimports flag.

kernprof.execfile(filename, globals=None, locals=None)

Python 3.x doesn’t have execfile() builtin

class kernprof.ContextualProfile(*args, **kwds)

Bases: ByCountProfilerMixin, Profile

A subclass of Profile that adds a context manager for Python 2.5 with: statements and a decorator.

__call__(func)

enable_by_count(subcalls=True, builtins=True)

Enable the profiler if it hasn’t been enabled before.

disable_by_count()

Disable the profiler if the number of disable requests matches the number of enable requests.

class kernprof.RepeatedTimer(interval, dump_func, outfile)

Bases: object

Background timer for outputting file every n seconds.

Adapted from [SO474528].

References

[SO474528]

https://stackoverflow.com/questions/474528/execute-function-every-x-seconds/40965385#40965385

start()

stop()

kernprof.find_module_script(module_name, *, static=True, exit_on_error=True)

Find the path to the executable script for a module or package.

kernprof.find_script(script_name, *, exit_on_error=True)

Find the script.

If the input is not a file, then PATH will be searched.

kernprof.pre_parse_single_arg_directive(args, flag, sep='--')

Pre-parse high-priority single-argument directives like -m module to emulate the behavior of python [...].

Examples

>>> import functools
>>> pre_parse = functools.partial(pre_parse_single_arg_directive,
...                               flag='-m')

Normal parsing:

>>> pre_parse(['foo', 'bar', 'baz'])
(['foo', 'bar', 'baz'], None, [])
>>> pre_parse(['foo', 'bar', '-m', 'baz'])
(['foo', 'bar'], 'baz', [])
>>> pre_parse(['foo', 'bar', '-m', 'baz', 'foobar'])
(['foo', 'bar'], 'baz', ['foobar'])

Erroneous case:

>>> pre_parse(['foo', 'bar', '-m'])
Traceback (most recent call last):
  ...
ValueError: argument expected for the -m option

Prevent erroneous consumption of the flag by passing it ‘–’:

>>> pre_parse(['foo', '--', 'bar', '-m', 'baz'])
(['foo', '--'], None, ['bar', '-m', 'baz'])
>>> pre_parse(['foo', '-m', 'spam',
...            'eggs', '--', 'bar', '-m', 'baz'])
(['foo'], 'spam', ['eggs', '--', 'bar', '-m', 'baz'])

kernprof.no_op(*_, **__) → None

kernprof.main(args=None, *, exit_on_error=True)

Runs the command line interface

Note

To help with traceback formatting, the deletion of temporary files created during execution may be deferred to when the interpreter exits.