How to terminate a python subprocess launched with shell=True

Group Therapy: Managing Process Groups

Running a subprocess that breeds like rabbits? A.k.a. spawns loads of child processes? Killing Papa process may not make the kids stop. To clean up the entire family group, we use os.setsid() in a preexec_fn during Popen. It's like starting a fitness boot camp that ensures the whole family works out:

import os
import subprocess
import signal

# Launch subprocess in a new family boot camp
proc = subprocess.Popen('your_command', shell=True, preexec_fn=os.setsid)

# With one swift move, end camp. No more summer fun.
os.killpg(os.getpgid(proc.pid), signal.SIGTERM)


Dodge and Weave: Avoiding shell=True

To assert your dominance over subprocess termination and dodge shell=True, execute commands straight up! Pass your command as a sequence (not the dance one, sorry) to Popen:

import subprocess

# Command as a sequence of arguments. String cheese to cheese cube conversion!
cmd = ['your_command', 'arg1', 'arg2']

# Launch subprocess without the shell game.
proc = subprocess.Popen(cmd, shell=False)

# Square dance finish: Swing your partner round and round, then STOP.
proc.terminate()


Remember, cmd.split() is your BFF for going from shell=True to shell=False.

The Hulk: Handling Stubborn Processes

Dealing with a moody process that gives you the silent treatment after a SIGTERM? Go psychology on them and use psutil:

import psutil

# Wrap process and its kids in a warm, psychological hug
process = psutil.Process(proc.pid)
for child in process.children(recursive=True):
    child.kill()  # assert dominance, child processes terminated.
process.kill()  # 'I am inevitable' - Thanos Proc


Crisis Management: Handling Timeout Scenarios

Got a deadline to meet? Use TimeoutExpired to forcibly terminate a subprocess if proc.wait(timeout) doesn't return in time:

try:
    proc.wait(timeout=3)  # 3-second timeout for the process to end... like an intense buzzer beater
except subprocess.TimeoutExpired:
    proc.kill()  # This process needs to have several seats, immediately.


But I'm on Windows...

For Windows users, terminating subprocesses and their child processes requires adjustments. Never change, Windows, never change. Bring in the taskkill command to settle the score:

import subprocess

pid = proc.pid  # Your PID or mine?
subprocess.run(f"TASKKILL /F /PID {pid} /T", shell=True)


For Real Though: Subprocess Kill Techniques

The Secret Technique: exec Prefix

In the cases where shell=True is just unavoidable, prefixing the command with exec helps ensure that p.kill() goes after the actual Bash process:

proc = subprocess.Popen('exec your_command', shell=True)  # Who knew exec had it in them?


Fine Tuning with psutil

For process management precision, the psutil library is like the Rolex of Python tools. Here's how you can use it to lay down the law on stubborn subprocesses:

psutil.Process(proc.pid).kill()  # No kill like overkill, eh?


Mind the Pipes

When putting exec to work, remember to consider the impact on pipes and redirections. This could make your command read like gibberish. The fix? Careful structuring. Comment monologues not included.