How to call a shell script from python code?

Feeding script with parameters

Passing arguments to the script

You want to give arguments? Just append them to the list:

subprocess.run(['./script.sh', 'arg1', 'arg2'])  # script.sh says "Thanks for the parameters!"


Smart folks apply shlex.split() for the argument's safe insertion when the command is already a string:

import shlex, subprocess

command = './script.sh arg1 arg2'  # The string before its majestic transformation
subprocess.run(shlex.split(command))  # Now it's a well-behaved list ready for smooth execution


Catching the echo of script

Want to listen stdout and stderr? Then specify subprocess.PIPE:

result = subprocess.run(['./script.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)  # We're all ears now
output, errors = result.stdout, result.stderr


You're interested in output only? Just use subprocess.check_output():

output = subprocess.check_output(['./script.sh'])  # Output, you can't hide from me!


Running the script with grace

Managing errors and exceptions

Be prepared for unexpected tantrums like subprocess.CalledProcessError, which denotes non-zero exit statuses; they can't escape from the mighty try-except:

try:
    subprocess.check_call(['./script.sh'])
except subprocess.CalledProcessError as e:
    print(f'Error occurred: {e}')  # Catch me if you can; Python: "Challenge accepted!"


Check the exit status explicitly; extend your control lever to successful execution:

process = subprocess.run(['./script.sh'])
if process.returncode != 0:  # Any confessions, dear script?
    # Handle error case here


Advanced usage of subprocess

Need to craft complex interactions with the script, or manage advanced I/O? Embrace subprocess.Popen():

with subprocess.Popen(['./script.sh'], stdout=subprocess.PIPE) as proc:
    log.info(proc.stdout.read())  # Listen carefully, Popen is speaking!


Safety first!

Prevent security vulnerabilities

Avoid shell=True unless compulsory, as it may introduce security hazards, especially with unsanitized input. If you really need it, clean your inputs like a surgeon or stick to trusted values.

Maintain script's integrity

Your script should be as trustworthy as a guide dog. Let trusted sources provide your scripts, maintain them securely, and always review third-party scripts before running, because nobody wants a Trojan horse!

Workflows that flow

Bridging script and Python

Redirect script's chatter to Python to leverage Python's data processing strengths. Make your shell scripts and Python code work in harmony:

import subprocess

# Let's roll the film!
completed_process = subprocess.run(['./data_processing.sh'], stdout=subprocess.PIPE)

# Now Python takes the wheel
processed_data = completed_process.stdout.decode('utf-8')


Automate like a pro with Python

Merge your Python script prowess with cron jobs or scheduling libraries like schedule to automate tasks like a boss:

import schedule
import subprocess

def run_script():
    subprocess.run(['./scheduled_backup.sh'])  # "Get up, it's time for a backup!"

# Schedule the alarm for 7 p.m. (Backup's bedtime story)
schedule.every().day.at("19:00").do(run_script)

while True:  # Loops can be insomniacs too!
    schedule.run_pending()
    time.sleep(1)  # Nap between tasks 


Not all environments are created equal

When scripts travel from dev to prod, they might face climate change. Avoid nasty surprises by using virtual environments, docker containers, or smart path strategies.