Explain Codes LogoExplain Codes Logo

Automatically create requirements.txt

python
pipreqs
requirements-txt
dependency-management
Alex KataevbyAlex Kataev·Oct 6, 2024
TLDR

pipreqs is your best friend for quick and easy creation of requirements.txt. It scans your Python project for imports and constructs a requirements.txt with only the relevant dependencies. Give it a whirl!

pip install pipreqs pipreqs /path/to/your_project

And voila, you have a project-specific requirements.txt in a snap.

To pip freeze or not to pip freeze

The key difference: While pip freeze goes all out and lists every single installed package, pipreqs is more refined and includes strictly the packages you used. This nails down requirements.txt to only truly relevant dependencies and keeps you off versioning landmines and bloated deployments.

How to manage your Python environment

Looking for other ways to manage your Python project’s dependencies? Here are a couple that could suit your fancy:

  • Pipenv: The combination of a virtual environment and dependency management brings consistency and load-off from your shoulders.
  • Conda: This does more than manage Python packages - it handles cross-platform environment management and can handle binary dependencies like a pro.

What about some edge cases?

Life ain’t all roses, here are some of those pesky non-ideal scenarios you may come across:

  • Private Repos: If you’ve been working with private package indices or custom packages that pipreqs can't find, you would need to put those on your requirements.txt manually.
  • Non-Python Dependencies: System-level dependencies? We have Docker or installation scripts to pair with requirements.txt for those.
  • Different Dependencies for Prod/Dev: Worried about dev-only dependencies messing with production? Then utilize pipreqs --savepath requirements-dev.in to have dev-specific packages, and a different requirements.in for production-only dependencies.

Next-level approach

Let's take it up a notch:

  • Version Lock: Use pip-compile from pip-tools to lock down versions of your packages. This puts the brakes on any undesired upgrades that might rain on your compatibility parade.
  • Multiple Environments: Fall back on pigar when dealing with different OS or when virtual environments give you the blues.
  • Continuous Integration (CI): Brew some automated magic by baking pipreqs into your CI pipeline. This keeps requirements.txt always fresh and updated.

Protect your castle

It's not just about making a fortress. A fortress is only as good as its defense. So, here are a few ways to safeguard your project:

  • Stay Vigilant: Regularly put your requirements.txt under the microscope with safety or pip-audit to ferret out vulnerability threats.
  • Keep Up To Date: Keep your dependencies current, but don't let the updates rule you. Remember to do controlled batch upgrades with rigorous testing for smooth sailing.
  • Check Compatibility: Remember to cross-check backward compatibility of packages, especially when you have pip-compile in your toolkit.