Python subprocess/Popen with a modified environment

python

subprocess

environment-variables

process-management

byNikita Barsukov·Mar 3, 2025

To run a subprocess altering the environment variables, pass a customized env mapping to subprocess.Popen(). For instance:

import subprocess
import os

# Grab a copy of the current environment and add the new location to PATH
modified_env = {**os.environ, "PATH": "/new/path" + os.pathsep + os.environ["PATH"]}

# Run the subprocess within this updated environment
process = subprocess.Popen(["command"], env=modified_env)
process.wait()

Key takeaways:

Use {**os.environ, "NEW_VAR": "value"} to merge os.environ with your new variables.
Add os.pathsep to ensure cross-platform path delineation.
Utilize env=modified_env as an argument in subprocess.Popen(), effectively setting up a customized subprocess environment.

Deep dive into the environment

Safe handling of non-string environment keys

Cloning os.environ via .copy() opens up another can of worms: non-string keys. In Python 3, os.environ can contain byte keys. Handle with care!

Quick'n'dirty one-off variable modification

If you just need to set one variable for a subprocess, you might not want to go through the hassle of cloning the whole environment. Instead, you can specify it directly:

# Here's your subprocess. One custom environment variable, coming right up!
process = subprocess.Popen(["command"], env={**os.environ, "MY_VARIABLE": "value"})

Safety first!

Before modifying the environment, double-check that you're doing so safely. Why? Adjusting the PATH variable might help sneak in undesired binaries. As grandma used to say: measure twice, cut once!

Handling exceptional situations

Dealing with non-identifier keys

Some environment variables might not be valid Python identifiers, for example, 'SOME-VAR'. To include them, pack and pass them using the ** operator:

hard_to_swallow_pill = {'SOME-VAR': 'value'}
process = subprocess.Popen(["command"], env={**os.environ, **hard_to_swallow_pill})

Safely extending PATH

Modifying the PATH needs some extra TLC:

env = os.environ.copy()
env["PATH"] = "/new/path" + os.pathsep + env.get("PATH", '')

Your .get() method is like your mother – it ensures you're never left out! If PATH isn't in the current environment, it will default to ''.

Other ways to override environment

If you're feeling adventurous, you may try alternative methods to set the environment:

with open('envfile') as file:
    env = dict(line.strip().split('=', 1) for line in file if '=' in line)
process = subprocess.Popen(["command"], env={**os.environ, **env})

This allows you to pull environment variables from a file, which many consider more celebrity-like than copying os.environ 😎.

Advanced tricks

Python PEP 448

For those who enjoy diving deep, check out Python's PEP 448 for some nifty dictionary operations.

Direct shots work too

If you're a fan of the K.I.S.S. (Keep It Simple Stupid) principle, directly setting the env parameter in subprocess.Popen() might be your cup of tea, especially for temporary or immutable environment changes.

explain-codes / Python / Python subprocess/Popen with a modified environment

Linked

Assign output of os.system to a variable and prevent it from being displayed on the screen



Run a Python script from another Python script, passing in arguments



How do I execute a program or call a system command?



How can I access environment variables in Python?



Retrieving the output of subprocess.call()

