Reference requirements.txt for the install_requires kwarg in setuptools setup.py file

Synergizing setup.py with requirements.txt

This quick and dirty solution took care of the initial requirements, but to truly navigate packaging nirvana, it is essential to understand the synergy between setup.py and requirements.txt. Think of it just like your kitchen: `setup.py holds your fridge (flexible storage of dependencies), while requirements.txt maintains your precise shopping list.

Using flexible dependencies in setup.py

Just like you can't always get fair-trade organic bananas from the farmer's market, your setup.py should allow some wiggle room in versioning:

install_requires=[
    'guacamole>=1.0.0,<2.0.0',
    'tacos~=1.4.2',
]


(Please don't try to install guacamole and taco packages, it's a joke 😉)

Shopping precisely with requirements.txt

Your requirements.txt, on the other hand, should be strict as a drill sergeant by pinning exact versions, ensuring reproducibility:

Requests==2.23.0
Django==3.0.5
Numpy==1.18.2

Kinda like a bouncer at a club, the requirements.txt makes sure only the right versions get in.

Masterclass in dependencies

Project dependencies can be as tricky as untangling Christmas lights. When enlisting your software troops, remember:

The role of setup.py

setup.py serves as your project's core. It's tasked with the delicate job of stating what packages are essential for your application run smoothly. It's more about defining compatible packages rather than tying down to exact versions.

Manifest magic with requirements.txt

Manifest is just a fancy word for a detailed list. In this case, requirements.txt serves as a chronicler, jotting down every exact requirement. It's your reliable aide for controlled deployments and generating reproducible builds.

Houdini tricks for importing requirements

Sometimes you need to perform a Houdini-style escape act or parse something more complex in your requirements file. For such special occasions, slip out of the box using pip's internal function to parse your requirements.txt:

from pip._internal.req import parse_requirements

# Time to parse 'requirements.txt' (Don't let the cat distract you!)
install_reqs = parse_requirements('requirements.txt', session='hack')

# Pulling out package names (Watch your fingers!)
reqs = [str(ir.requirement) for ir in install_reqs]

setup(
    ...
    install_requires=reqs,
    ...
)


Beware that you're dabbling with pip's internal forces; usage may change across pip versions!

Dependency links: not just URLS

Dependencies are often chained or linked with VCS URLs in requirements.txt. To handle these, consider dependency_links:

dependency_links=[
    'git+https://repo.com/package.git#egg=package-1.0'  # Who knew GitHub could lay eggs? 🐔🥚
]


Ensure you have a mechanism to process these within your setup script.

Elegant deployment and development

While orchestrating a symphony of Python packages, it's crucial to differentiate between two types of dependencies: abstract (like your dreams) and concrete (like your deadline for this project).

Tips to sustain your sanity

• Identify and declare abstract couplings in setup.py, giving room for minor updates.
• Nail down exact versions in requirements.txt for reproducibility.
• Make requirements.txt your sidekick when dealing with complex dependencies.
• Consider pip-tools your friendly neighbourhood Spider-man in managing upgrades.
• Don't shy away from layering requirements to handle different environments (dev, testing or prod, you got this!).

Staying prepared

• Be battle-ready for potential package breaking changes, specify maximum versions when needed.
• Package dependencies can sometimes clash; remain vigilant as your project grows.
• Always TEST your install_requires against various environments and conditions.
• Regularly audit your requirements.txt. Is it still in sync with setup.py?

References