Python: How to profile the Twisted Application

neotam Avatar

Python: How to profile the Twisted Application

Python ships with two modules profile/cProfile and pstats to profile and read profiled data respectively. This article will guide about profiling the python application, storing the profiled data and illustrates how to use pstats module to read the saved profiled data.

Profiling is helpful to optimize the code and help you identify the long running function calls.

How to profile python app

Twisted the is the event-driven asynchronous network-programming framework for python. By convension, it is encourage to write the short function and refraining from creating long running function calls. Since a long running function call can put the global reactor loop in blocking mode. To write the optmized code, we nee

Simple way to save profiling stats into a file using cProfile is as follows

python -m cProfile -o

Profiling Twisted Application

twistd -y myapp.tac --profiler=cProfile --savestats -n


-p, --profile=       Run in profile mode, dumping results to specified file.
      --pidfile=       Name of the pidfile [default:]
      --prefix=        use the given prefix when syslogging [default: twisted]
      --profiler=      Name of the profiler to use (profile, cprofile).
                       [default: cprofile]

      --savestats      save the Stats object rather than the text output of the

According to above command, output is saved to file named “statsfile”. The output written to the given file would be in binary, such that to read use have to use one of following method

If you are manually starting the twisted application with out having to use command line app twistd, You can call the profile function from within the code

import profile, filename='profile/')

Reading Profile stats

To read and analyze the saved profiled data, we can use the pstats module as follows

import pstats

profile_stats_file = ""

# Create stats object by passing stats file name 
stats = pstats.Stats(profile_stats_file)

# Print the statistics by function name

# Printing stats in human readable manner 
# stats.strip_dirs().sort_stats("cumtime").print_stats(10)  # Show top 10 functions

Another way is to open the “Profile statistics browser”

python -m pstats <statsfile> 

Above command will open the “profile statistics browser”, which is basically the interpreter help you to interactively read, sort and analyze the profiled data



We can sort the statistics according to different parameter such as

Valid sort keys (unique prefixes are accepted):
calls -- call count
ncalls -- call count
cumtime -- cumulative time
cumulative -- cumulative time
filename -- file name
line -- line number
module -- file name
name -- function name
nfl -- name/file/line
pcalls -- primitive call count
stdname -- standard name
time -- internal time
tottime -- internal time

To soft function calls based on cumulative time, issue the following command

stats cumtime

Leave a Reply

Your email address will not be published. Required fields are marked *