How do I profile a Python script?

python

profiling

performance-optimizations

tools

byNikita Barsukov·Sep 6, 2024

Maximize your Python code efficiency with cProfile. It helps identify performance bottlenecks:

import cProfile
import pstats

def main():
    # Your lucrative business logic here

# Profile 'main' function like Tom Cruise in Mission Impossible, save stats
cProfile.run('main()', 'result.prof')

# Analyze top time-consuming functions, like Sherlock Holmes
p = pstats.Stats('result.prof').strip_dirs().sort_stats(-1).print_stats()

The code will time and rank functions within main. A quick-fire way to spot performance culprits. It's that simple!

Profiling Techniques: Unlock Efficiency!

Command-Line Profiling: Quick & Dirty!

Make the command-line your friend:

python -m cProfile -o output_stats.prof myscript.py

With -o, you can save the output for later like squirrels saving nuts for winter. For module-level profiling, unleash:

python -m cProfile -o output_stats.prof -m mymodule

Use profile.bat files for quick repeated profile missions.

Profiling the Spiders: Multi-Threaded Profiling

Embrace thread-specific profiling with cProfile.Profile(), or use threading.setprofile() for an overview of your multi-threaded applications. Create a ProfiledThread subclass:

import cProfile
import threading

class ProfiledThread(threading.Thread):
    def run(self):
        profiler = cProfile.Profile()
        profiler.enable()
        super().run()
        profiler.disable()
        profiler.dump_stats(f"profile-{self.name}.prof")

Swap threading.Thread with this class to profile threads like James Bond surveilling villains.

Profiling Picture Show: `pycallgraph`

Imagine visualizing your code's journey with pycallgraph:

pip install pycallgraph
pycallgraph graphviz -- ./myscript.py

The resulting heatmap pycallgraph.png will depict dependencies and hotspots.

Some Sssssssinful Analysis with `snakeviz`

Turn cProfile data into pie-charts of joy:

pip install snakeviz
python -m cProfile -o stats.prof your_script.py
snakeviz stats.prof

The analysis is launched post-execution in your browser. It's like eating cake slices of performance data!

Got Graphviz?

Install Graphviz for visual profiling tools:

sudo apt-get install graphviz

It boosts your graph-rendering artillery for visual geniuses.

Profiling Mastery: Tools & Tips

Store & Explore: Saving Statistics

Preserve your cProfile session for further investigation like a time capsule:

python -m cProfile -o profile.stats your_script.py

Crack open this capsule with pstats or gprof2dot for a deep dive into the data.

Get a Profiler, Be a Profiler

To broaden your toolkit, add python-profiler:

sudo apt-get install python-profiler

Don't forget to invite pycallgraph and gprof2dot using pip.

Python Docs: The Profiler's Bible

The Python Wiki and Docs offer a wealth of insight into performance optimizations. Unleash these resources, let them guide you like a compass in this quest.

Confused by Competitive coding? Let Profiling Help

In scenarios like Project Euler, profiling is a knight in shining armor saving you from the dragon of convoluted codes.

Keep it Crisp with SVG

For clarity at every zoom level, opt for cairo-renderer SVG when using pycallgraph:

pycallgraph graphviz --output-format=svg ./myscript.py

Retrofit your in-depth graphics with the scalability of SVG.

explain-codes / Python / How do I profile a Python script?

Linked

How to get current CPU and RAM usage in Python?



How to obtain a Thread id in Python?



Measuring elapsed time with the Time module



Which Python memory profiler is recommended?



How do I parallelize a simple Python loop?



How do I use threading in Python?



When and how should I use a ThreadLocal variable?



Profiling Techniques: Unlock Efficiency!Profiling Mastery: Tools & Tips