setup.py & pip: override one of the dependency's sub-dependency from requirements.txt

Question

I'm currently working on a package and in my `requirements.txt`, I have a dependency: `wikipedia`. Now, `wikipedia` 1.3 uses `requests-2.2.1` while my package uses version 2.3.0. Also, as one would expect, `wikipedia-1.3`'s installation depends on presence of it's dependency. But, If I start a new virtualenv and directly include `wikipedia` in my `requirements.txt`, it gives an `ImportError` on `requests` since at the time `setup.py` runs, `requests-2.3.0`'s `setup.py` doesn't execute unless all others execute. In the Figure attached below, there's no running `setup.py` for `requests` after it gets unpacked. For some weird reason, `wikipedia`'s `setup.py` contains `import wikipedia`, which in turn imports it's dependencies before they're even installed; however it passes the CI test because it's installing requirements separately through pip and then running `setup.py`. To over come this situation, I've made a setup script consisting of: ``` pip install -r requirements.txt pip install…

Accepted Answer

As noted by Martijn the correct way would be to specify a minimum version in the project assuming full compatibility is preserved in future releases of the sub-dependency.

If you do not have any way to change the requirements file you can download the project and edit the requirements file locally to specify whatever version you want. This can be done via the `pip download` command:

```
pip download wikipedia==1.3
```

Besides that if you want to use `pip` for the whole process and preserve `requests==2.3.0` without deleting and reinstalling again you can specify a `constraints` file. This can be done with:

```
pip install -c constraints.txt wikipedia==1.3
```

Where constraints.txt contains something like:

```
requests>=2.3.0
beautifulsoup4
```

This will produce a warning, but the `wikipedia` package will be installed:

```
wikipedia 1.3.0 has requirement requests==2.2.1, but you'll have requests 2.3.0 which is incompatible.
Installing collected packages: wikipedia
Successfully installed wikipedia-1.3.0
```

Now, if you really know what you are doing(or just want to try if it works) you can use the `--no-deps` flag which will ignore package dependencies entirely and will not produce the warning above:

```
pip install --no-deps -c constraints.txt wikipedia==1.3
```

In both cases `pip freeze` shows:

```
beautifulsoup4==4.6.0
bs4==0.0.1
requests==2.3.0
wikipedia==1.3.0
```

Note: This was tested with pip 10.0.1, but it should work with any recent pip version.

setup.py & pip: override one of the dependency's sub-dependency from requirements.txt

Solution

Problem