share
Stack Overflowinstalling python packages without internet and using source code as .tar.gz and .whl
[+97] [9] srinath
[2016-04-19 17:49:35]
[ python pip openstack ]
[ https://stackoverflow.com/questions/36725843/installing-python-packages-without-internet-and-using-source-code-as-tar-gz-and ]

we are trying to install couple of python packages without internet.

For ex : python-keystoneclient

For that we have the packages downloaded from https://pypi.python.org/pypi/python-keystoneclient/1.7.1 and kept it in server.

However, while installing tar.gz and .whl packages , the installation is looking for dependent packages to be installed first. Since there is no internet connection in the server, it is getting failed.

For ex : For python-keystoneclient we have the following dependent packages

stevedore (>=1.5.0)
six (>=1.9.0)
requests (>=2.5.2)
PrettyTable (<0.8,>=0.7)
oslo.utils (>=2.0.0)
oslo.serialization (>=1.4.0)
oslo.i18n (>=1.5.0)
oslo.config (>=2.3.0)
netaddr (!=0.7.16,>=0.7.12)
debtcollector (>=0.3.0)
iso8601 (>=0.1.9)
Babel (>=1.3)
argparse
pbr (<2.0,>=1.6)

When i try to install packages one by one from the above list, once again its looking for nested dependency .

Is there any way we could list ALL the dependent packages for installing a python module like python-keystoneclient.

(2) You may have luck using containers for this. - Matt Joyce
[+158] [2016-04-19 21:42:47] Praveen Yalagandula [ACCEPTED]

This is how I handle this case:

On the machine where I have access to Internet:

mkdir keystone-deps
pip download python-keystoneclient -d "/home/aviuser/keystone-deps"
tar cvfz keystone-deps.tgz keystone-deps

Then move the tar file to the destination machine that does not have Internet access and perform the following:

tar xvfz keystone-deps.tgz
cd keystone-deps
pip install python_keystoneclient-2.3.1-py2.py3-none-any.whl -f ./ --no-index

You may need to add --no-deps to the command as follows:

pip install python_keystoneclient-2.3.1-py2.py3-none-any.whl -f ./ --no-index --no-deps

(4) This could be updated with the "download" option of pip. Additionally, it would be nice to have an explanation of the pip option "no-index". - fffred
You can always get a light weight pip from here: bootstrap.pypa.io/get-pip.py - Praveen Yalagandula
(1) @GianlucaD'Ardia -- I downloaded a pip wheel as well and you can install it using itself with the following (trippy) command: python ~/eggs/pip-9.0.1-py2.py3-none-any.whl/pip install --no-index ~/eggs/pip-9.0.1-py2.py3-none-any.whl (replace the version particulars with your wheel's specifics) - Ben Mosher
Can you share the same for windows machine - Chetan.B
(6) To add some details: pip install --help indicates that: (1)-f or --find-links means ...If a local path or file:// url that's a directory, then look for archives in the directory listing., so here it locates the files in this dir; (2)--no-index means Ignore package index (only looking at --find-links URLs instead). So pip install xxx.whl -f ./ --no-index means: find install files in the current dir instead of searching in remote repo. - WesternGun
(4) What if my local machine uses a different platform? - deed02392
(2) To avoid platform issues, you can try to use universal wheels if possible (none-any wheels). But if you must target a specific platform, see the --platform option of pip. @deed02392 - Amit Naidu
if you need to download a particular version that also can be done Example 'pip3 download Babel==2.10.1 -d "/home/user1/lab/pip/deps/" ' - Arun
1
[+56] [2018-12-05 05:36:08] rouble

If you want to install a bunch of dependencies from, say a requirements.txt, you would do:

mkdir dependencies
pip download -r requirements.txt -d "./dependencies"
tar cvfz dependencies.tar.gz dependencies

And, once you transfer the dependencies.tar.gz to the machine which does not have internet you would do:

tar zxvf dependencies.tar.gz
cd dependencies
pip install * -f ./ --no-index

(1) this method worked for me, I just think you don't need really the quotation marks on the second line. - Georgi Stoyanov
(9) Is there an equivalent for * on Windows? I get an error saying: ERROR: Invalid requirement: '*' when on Windows station. - Dominik Novotný
Windows does not have tar, but you can make a zip folder. If not from the command line, you can use the typical right click menu to make a zip. - MadHatter
(2) @DominikNovotný there is another answer for this: for %x in (dir *.whl) do python -m pip install %x from stackoverflow.com/a/43314623/8537739. Aso notice, that installation goes alphabetically, and some libraries that depend from not yet installed will fail. So this solution could be improved. Or you need call this command untill all libraries will be installed. - Dmitry
2
[+12] [2016-04-19 21:56:17] Burhan Khalid

We have a similar situation at work, where the production machines have no access to the Internet; therefore everything has to be managed offline and off-host.

Here is what I tried with varied amounts of success:

  1. basket [1] which is a small utility that you run on your internet-connected host. Instead of trying to install a package, it will instead download it, and everything else it requires to be installed into a directory. You then move this directory onto your target machine. Pros: very easy and simple to use, no server headaches; no ports to configure. Cons: there aren't any real showstoppers, but the biggest one is that it doesn't respect any version pinning you may have; it will always download the latest version of a package.

  2. Run a local pypi server. Used pypiserver [2] and devpi [3]. pypiserver is super simple to install and setup; devpi takes a bit more finagling. They both do the same thing - act as a proxy/cache for the real pypi and as a local pypi server for any home-grown packages. localshop [4] is a new one that wasn't around when I was looking, it also has the same idea. So how it works is your internet-restricted machine will connect to these servers, they are then connected to the Internet so that they can cache and proxy the actual repository.

The problem with the second approach is that although you get maximum compatibility and access to the entire repository of Python packages, you still need to make sure any/all dependencies are installed on your target machines (for example, any headers for database drivers and a build toolchain). Further, these solutions do not cater for non-pypi repositories (for example, packages that are hosted on github).

We got very far with the second option though, so I would definitely recommend it.

Eventually, getting tired of having to deal with compatibility issues and libraries, we migrated the entire circus of servers to commercially supported docker containers.

This means that we ship everything pre-configured, nothing actually needs to be installed on the production machines and it has been the most headache-free solution for us.

We replaced the pypi repositories with a local docker image server.

[1] https://pypi.python.org/pypi/Basket/1.0
[2] https://pypi.python.org/pypi/pypiserver
[3] http://doc.devpi.net/latest/
[4] https://github.com/mvantellingen/localshop

3
[+3] [2022-11-28 20:26:21] Ajay Tom George

This is also an add up to the answer by Praveen Yalagandula. As downloading for specific version of python --python-version and specific platform like linux --platform manylinux1_x86_64 was not elaborated in the answer.

pip3 download somePackage --platform manylinux1_x86_64 --only-binary=:all: -d "/Users/ajaytomgeorge/Dev/wheels/"

There are advanced arguments also which you can pass through FULL LIST of ARGUMENTS [1]

Examples

--progress-bar 
--no-build-isolation
--use-pep517
--check-build-dependencies
--ignore-requires-python
-d
--platform
--python-version.
--implementations

Another e.g for python 2.7 and mac

pip download --only-binary=:all: --platform macosx-10_10_x86_64 --python-version 27 --implementation cp SomePackage
[1] https://pip.pypa.io/en/stable/cli/pip_download/

4
[+2] [2023-06-03 12:47:07] Taha Almasri

For windows users if you want to install pkgs in an env that is not connected to internet on a host that is connected to internet

mkdir dependencies
pip download -r requirements.txt -d "./dependencies"
tar cvfz dependencies.tar.gz dependencies

on a host that is not

tar zxvf dependencies.tar.gz
cd dependencies
for %f in (*.whl) do pip install --no-index --find-links=./ %f

5
[0] [2016-04-19 18:01:21] moveax3

pipdeptree is a command line utility for displaying the python packages installed in an virtualenv in form of a dependency tree. Just use it: https://github.com/naiquevin/pipdeptree


6
[0] [2020-05-05 21:16:56] Sudhik

This isn't an answer. I was struggling but then realized that my install was trying to connect to internet to download dependencies.

So, I downloaded and installed dependencies first and then installed with below command. It worked

python -m pip install filename.tar.gz

this doesn't work. - Anirban
7
[0] [2024-04-22 13:50:19] Maaz Irfan

First, you need to download the package and its dependencies on a machine that has internet access. You can use a tool like pip to download packages

pip download pandas

Once the files are on the server, you can install them using pip. Navigate to the directory containing the downloaded .whl files and run

pip install Pandas.whl

If your server uses a different version of Python or a different system architecture, make sure to download the packages that match the server’s Python version and architecture. If you encounter dependency conflicts during installation, you may need to adjust the versions of the packages you're installing. Adjust the versions in your pip download command accordingly.


8
[-1] [2022-02-14 13:56:59] Israel

You can manually download the 'whl' file from PyPI: https://pypi.org/project/google-cloud-debugger-client/#files

Then locate it in the root folder and you can just install it via pip:

pip install google_cloud_debugger_client-1.2.1-py2.py3-none-any.whl

(1) it will only work if the dependent library is already installed, if the dependent library is not installed then it will try to install it from the python public repo. But as per question you don't have internet, so it fail to install the dependent library. - Anirban
9