Python ships with two modules profile/cProfile and pstats to profile and read profiled data respectively. This article will guide about profiling the python application, storing the profiled data and illustrates how to use pstats module to read the saved profiled data.

Profiling is helpful to optimize the code and help you identify the long running function calls.

How to profile python app

Twisted the is the event-driven asynchronous network-programming framework for python. By convension, it is encourage to write the short function and refraining from creating long running function calls. Since a long running function call can put the global reactor loop in blocking mode. To write the optmized code, we nee

Simple way to save profiling stats into a file using cProfile is as follows

python -m cProfile -o profile_stats.prof my_app.py

Profiling Twisted Application

twistd -y myapp.tac --profile=profile_stats.prof --profiler=cProfile --savestats -n

Where,

-p, --profile=       Run in profile mode, dumping results to specified file.
      --pidfile=       Name of the pidfile [default: twistd.pid]
      --prefix=        use the given prefix when syslogging [default: twisted]
      --profiler=      Name of the profiler to use (profile, cprofile).
                       [default: cprofile]

      --savestats      save the Stats object rather than the text output of the

According to above command, output is saved to file named “statsfile”. The output written to the given file would be in binary, such that to read use have to use one of following method

Reading Profile stats

To read and analyze the saved profiled data, we can use the pstats module as follows

import pstats

profile_stats_file = "profile_stats.prof"

# Create stats object by passing stats file name 
stats = pstats.Stats(profile_stats_file)

# Print the statistics by function name
stats.strip_dirs().sort_stats("cumulative").print_stats()

# Printing stats in human readable manner 
# stats.strip_dirs().sort_stats("cumtime").print_stats(10)  # Show top 10 functions

Another way is to open the “Profile statistics browser”

python -m pstats <statsfile> 

Above command will open the “profile statistics browser”, which is basically the interpreter help you to interactively read, sort and analyze the profiled data

Commands:

EOF
add
callees
callers
help
quit
read
reverse
sort
stats
strip

We can sort the statistics according to different parameter such as

Valid sort keys (unique prefixes are accepted):
calls -- call count
ncalls -- call count
cumtime -- cumulative time
cumulative -- cumulative time
filename -- file name
line -- line number
module -- file name
name -- function name
nfl -- name/file/line
pcalls -- primitive call count
stdname -- standard name
time -- internal time
tottime -- internal time

To soft function calls based on cumulative time, issue the following command

stats cumtime