I'm trying to see all the dependencies that are required for an specific package(in this case I'm using pipdeptree) but it turns out that it only shows me the dependency tree for an installed package. Let's say that I'm using graphene:2.1.0, so e.g:
$ pipdeptree -p graphene
Warning!!! Possibly conflicting dependencies found:
* graphql-relay==0.5.0
- graphql-core [required: >=0.5.0,<2, installed: 2.3.1]
* social-auth-core==3.2.0
- requests [required: >=2.9.1, installed: 2.8.1]
------------------------------------------------------------------------
Now when I try $ pipdeptree -p graphene==40.0.2 (which does not exist).
Warning!!! Possibly conflicting dependencies found:
* graphql-relay==0.5.0
- graphql-core [required: >=0.5.0,<2, installed: 2.3.1]
* social-auth-core==3.2.0
- requests [required: >=2.9.1, installed: 2.8.1]
------------------------------------------------------------------------
It seems that it only takes into account a stable version, what I want is a dependency tree related to the specific version that I give by console, I hope I'm being clear with this.
I believe johnnydep can help with that:
$ johnnydep --verbose 0 'graphene==2.1.0'
name summary
------------------------------ ---------------------------------------
graphene==2.1.0 GraphQL Framework for Python
├── aniso8601<4,>=3 A library for parsing ISO 8601 strings.
├── graphql-core<3,>=2.0 GraphQL implementation for Python
│ ├── promise<3,>=2.3 Promises/A+ implementation for Python
│ │ └── six Python 2 and 3 compatibility utilities
│ ├── rx<2,>=1.6 Reactive Extensions (Rx) for Python
│ └── six>=1.10.0 Python 2 and 3 compatibility utilities
├── graphql-relay<1,>=0.4.5 Relay implementation for Python
│ ├── graphql-core<2,>=0.5.0 GraphQL implementation for Python
│ │ ├── promise>=2.0 Promises/A+ implementation for Python
│ │ │ └── six Python 2 and 3 compatibility utilities
│ │ └── six>=1.10.0 Python 2 and 3 compatibility utilities
│ ├── promise>=0.4.0 Promises/A+ implementation for Python
│ │ └── six Python 2 and 3 compatibility utilities
│ └── six>=1.10.0 Python 2 and 3 compatibility utilities
├── promise<3,>=2.1 Promises/A+ implementation for Python
│ └── six Python 2 and 3 compatibility utilities
└── six<2,>=1.10.0 Python 2 and 3 compatibility utilities
Related
I have a private repo on GitHub containing a Poetry package which I install with
poetry add git+https://github.com/<USERNAME>/<REPO>.git#<BRANCH>
This project's structure is (assume it's called package_1)
├── README.md
├── package_1
│ ├── __init__.py
│ ├── subpackage_1
│ │ ├── __init__.py
│ │ └── subpackage_1.py
│ ├── subpackage_2
│ │ ├── __init__.py
│ │ └── subpackage_2.py
│ └── subpackage_3
│ ├── __init__.py
│ └── subpackage_3.py
├── poetry.lock
├── pyproject.toml
└── tests
├── __init__.py
└── test_package_1.py
package_1/init.py is just __version__ = "0.3.3"
When I run poetry add git+https://github.com/<USERNAME>/<REPO>.git#<BRANCH> on a new package, the package_1 package is correctly installed but two things happen which I don't understand:
1: The new package's pyproject.toml file shows package_1's version as 0.2.0 when clearly the version is 0.3.3 as mentioned above;
2: The poetry.lock file looks like this after I install package_name
[[package]]
name = "package-name"
version = "0.2.0"
...
Why is the name package-name instead of package_name (notice the _ and -)?
I have a local python project (A) that depends on another local python project (B).
I have added the project B as a git submodule to a dependencies folder.
When I pull the repo with all the submodules, I would like to have a setup.py in A such that wenn I run pip install -e . in A, it also installs the setup.py in B.
The repo looks like this:
A
├── A
│ └── foo.py
│
├── dependencies
│ └── B
│ ├── B
│ │ └── foo.py
│ │
│ └── setup.py
│
└── setup.py
I tried from setuptools import setup, find_packages
setup(
name='A',
packages=find_packages(),
)
But that only finds package A, which makes sense.
I also tried adding a preinstall command
class PreInstallCommand(install):
"""Pre-installation for installation mode."""
def run(self):
check_call("pip install -e dependencies/B".split())
install.run(self)
setup(
...
cmdclass={
'install': PreInstallCommand,
},
...
)
But that one is not triggered when I install it in editable mode (pip install -e).
Thanks in advance for your advice! :)
Cheers!
Hi guys i have a problem with creating python package out of my project. I read some tutorials but every one leads to uploading files to pypi which i dont want to do. I just want to pip install it to my machine locally using tar.gz file.
Here's a structure of my project folder:
root
├── src
│ ├── __init__.py
│ ├── config.py
│ └── sth_a
│ │ ├── __init__.py
│ │ └── a.py
│ └── sth_b
│ ├── __init__.py
│ └── b.py
└── setup.py
Here is how my setup.py file looks like:
from setuptools import setup, find_packages
setup(name="mypkg",
version='0.0.1',
author="whatever",
packages=find_packages()
)
First i run command:
python setup.py sdist bdist_wheel
then it creatates dist dictionary with tar.gz file and wheel file, so i just run
pip3 install dist/mypkg-0.0.1.tar.gz
After this first problem emerges. To import these files somewhere else i need to write
from src.sth_a.a import *
but i want to do this like this
from mypgk.src.sth_a.a import *
or even if i just want to 'publish' for example functions from file a.py
from mypck.a import *
Also i was having another issues bit this answer helped me a bit but it is not still what i want pip install . creates only the dist-info not the package
On a fresh python3.7 virtual environment I try to install a downloaded wheel but the installation step on Windows takes ~20 seconds (on linux is a bit faster but still ~20seconds)
Executed steps
Here I create venv, update pip, setuptools & wheel, download scons, nothing really interesting here.
PS C:\s\tmp> C:\Python37\python.exe -m venv venv37
PS C:\s\tmp> .\venv37\Scripts\activate
(venv37) PS C:\s\tmp> pip install -U pip setuptools wheel
Requirement already up-to-date: pip in c:\s\tmp\venv37\lib\site-packages (20.1.1)
Collecting setuptools
Using cached setuptools-49.1.0-py3-none-any.whl (789 kB)
Collecting wheel
Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Installing collected packages: setuptools, wheel
Attempting uninstall: setuptools
Found existing installation: setuptools 47.1.0
Uninstalling setuptools-47.1.0:
Successfully uninstalled setuptools-47.1.0
Successfully installed setuptools-49.1.0 wheel-0.34.2
(venv37) PS C:\s\tmp> Measure-Command { pip download scons | Out-Default }
Collecting scons
Using cached SCons-4.0.0-py3-none-any.whl (4.0 MB)
Saved c:\s\tmp\scons-4.0.0-py3-none-any.whl
Collecting pywin32>=1.0; platform_system == "Windows"
Using cached pywin32-228-cp37-cp37m-win_amd64.whl (9.1 MB)
Saved c:\s\tmp\pywin32-228-cp37-cp37m-win_amd64.whl
Collecting setuptools
Using cached setuptools-49.1.0-py3-none-any.whl (789 kB)
Saved c:\s\tmp\setuptools-49.1.0-py3-none-any.whl
Successfully downloaded scons pywin32 setuptools
Seconds : 2
(venv37) PS C:\s\tmp> ls
Directory: C:\s\tmp
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 7/10/2020 2:38 PM venv37
-a---- 7/10/2020 2:40 PM 9100091 pywin32-228-cp37-cp37m-win_amd64.whl
-a---- 7/10/2020 2:40 PM 3970271 SCons-4.0.0-py3-none-any.whl
-a---- 7/10/2020 2:40 PM 789761 setuptools-49.1.0-py3-none-any.whl
Here I try to install the downloaded wheel, which takes up to 17 seconds.
(venv37) PS C:\s\tmp> Measure-Command { python -m pip install -vvv -U --no-index --find-links . scons | Out-Default }
Non-user install because user site-packages disabled
Ignoring indexes: https://pypi.org/simple
Created temporary directory: C:\Users\FRANCE~1\AppData\Local\Temp\pip-ephem-wheel-cache-56rg4dkw
Created temporary directory: C:\Users\FRANCE~1\AppData\Local\Temp\pip-req-tracker-awspyjg9
Initialized build tracking at C:\Users\FRANCE~1\AppData\Local\Temp\pip-req-tracker-awspyjg9
Created build tracker: C:\Users\FRANCE~1\AppData\Local\Temp\pip-req-tracker-awspyjg9
Entered build tracker: C:\Users\FRANCE~1\AppData\Local\Temp\pip-req-tracker-awspyjg9
Created temporary directory: C:\Users\FRANCE~1\AppData\Local\Temp\pip-install-1_k389zs
Looking in links: .
0 location(s) to search for versions of scons:
Skipping link: not a file: . (from -f)
Skipping link: wrong project name (not scons): file:///C:/s/tmp/pywin32-228-cp37-cp37m-win_amd64.whl
Found link file:///C:/s/tmp/SCons-4.0.0-py3-none-any.whl, version: 4.0.0
Skipping link: wrong project name (not scons): file:///C:/s/tmp/setuptools-49.1.0-py3-none-any.whl
Skipping link: not a file: file:///C:/s/tmp/venv37
Local files found: C:\s\tmp\SCons-4.0.0-py3-none-any.whl
Given no hashes to check 1 links for project 'scons': discarding no candidates
Using version 4.0.0 (newest of versions: 4.0.0)
Processing c:\s\tmp\scons-4.0.0-py3-none-any.whl
Added scons from file:///C:/s/tmp/SCons-4.0.0-py3-none-any.whl to build tracker 'C:\\Users\\FRANCE~1\\AppData\\Local\\Temp\\pip-req-tracker-awspyjg9'
Removed scons from file:///C:/s/tmp/SCons-4.0.0-py3-none-any.whl from build tracker 'C:\\Users\\FRANCE~1\\AppData\\Local\\Temp\\pip-req-tracker-awspyjg9'
Requirement already satisfied, skipping upgrade: setuptools in c:\s\tmp\venv37\lib\site-packages (from scons) (49.1.0)
0 location(s) to search for versions of pywin32:
Found link file:///C:/s/tmp/pywin32-228-cp37-cp37m-win_amd64.whl, version: 228
Skipping link: wrong project name (not pywin32): file:///C:/s/tmp/SCons-4.0.0-py3-none-any.whl
Local files found: C:\s\tmp\pywin32-228-cp37-cp37m-win_amd64.whl
Given no hashes to check 1 links for project 'pywin32': discarding no candidates
Using version 228 (newest of versions: 228)
Processing c:\s\tmp\pywin32-228-cp37-cp37m-win_amd64.whl
Added pywin32>=1.0; platform_system == "Windows" from file:///C:/s/tmp/pywin32-228-cp37-cp37m-win_amd64.whl (from scons) to build tracker 'C:\\Users\\FRANCE~1\\AppData\\Local\\Temp\\pip-req-tracker-awspyjg9'
Removed pywin32>=1.0; platform_system == "Windows" from file:///C:/s/tmp/pywin32-228-cp37-cp37m-win_amd64.whl (from scons) from build tracker 'C:\\Users\\FRANCE~1\\AppData\\Local\\Temp\\pip-req-tracker-awspyjg9'
Installing collected packages: pywin32, scons
Created temporary directory: C:\Users\FRANCE~1\AppData\Local\Temp\pip-unpacked-wheel-vwh0ebm3
Created temporary directory: C:\Users\FRANCE~1\AppData\Local\Temp\pip-unpacked-wheel-dxib5q_u
Successfully installed pywin32-228 scons-4.0.0
Removed build tracker: 'C:\\Users\\FRANCE~1\\AppData\\Local\\Temp\\pip-req-tracker-awspyjg9'
Seconds : 17
Is there anything I can do to speed up the installation step?
Update (7.13.2020):
After bdbaddog's comment I shortly had a look at the repo, as he suggested dobook artifacts seem to be the ones to blame for this slow installation.
(venv37) PS C:\src\scons\SCons> dust
397K ┌── Node │ █ │ 2%
522K │ ┌── RELEASE-NOTES.xml │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█ │ 3%
612K │ ├── manpages │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█ │ 3%
666K │ ├── RELEASE-NOTES.html │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█ │ 3%
696K │ ├── webhelp │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█ │ 3%
417K │ │ ┌── template-pages.xml │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 2%
860K │ ├─┴ roundtrip │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 4%
876K │ ├── slides │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 4%
919K │ ├── params │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 4%
1.1M │ │ ┌── reference.xml.included│ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 6%
1.2M │ ├─┴ docsrc │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 6%
1.3M │ ├── xhtml-1_1 │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 7%
1.3M │ ├── xhtml │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │ 7%
1.6M │ ├── html │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓███ │ 8%
447K │ │ ┌── param.xml │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓███ │ 2%
1.8M │ ├─┴ fo │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓███ │ 9%
3.1M │ ├── common │ ░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█████ │ 15%
16M │ ┌─┴ docbook-xsl-1.76.1 │ ░░████████████████████████ │ 83%
16M │ ┌─┴ docbook │ ░░████████████████████████ │ 83%
17M ├─┴ Tool │ ██████████████████████████ │ 89%
20M ┌─┴ . │████████████████████████████ │ 100%
How can one command find out all dependencies of the package and links to dependent packages in the repository pip?
You can use johnnydep
pip install johnnydep
Example: johnnydep ipython
ipython IPython: Productive Interactive Computing
├── appnope Disable App Nap on OS X 10.9
├── backcall Specifications for callback functions passed in to an API
├── decorator Better living through Python with decorators
├── jedi>=0.10 An autocompletion tool for Python that can be used for text editors.
│ └── parso>=0.3.0 A Python Parser
├── pexpect Pexpect allows easy control of interactive console applications.
│ └── ptyprocess>=0.5 Run a subprocess in a pseudo terminal
├── pickleshare Tiny 'shelve'-like database with concurrency support
├── prompt-toolkit<2.1.0,>=2.0.0 Library for building powerful interactive command lines in Python
│ ├── six>=1.9.0 Python 2 and 3 compatibility utilities
│ └── wcwidth Measures number of Terminal column cells of wide-character codes
├── pygments Pygments is a syntax highlighting package written in Python.
├── setuptools>=18.5 Easily download, build, install, upgrade, and uninstall Python packages
├── simplegeneric>0.8 Simple generic functions (similar to Python's own len(), pickle.dump(), etc.)
└── traitlets>=4.2 Traitlets Python config system
├── decorator Better living through Python with decorators
├── ipython-genutils Vestigial utilities from IPython
└── six Python 2 and 3 compatibility utilities
You can also display download links to dependant packages by adding the --fields download_link option:
johnnydep ipython --fields download_link
.
name download_link
-------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------
ipython https://files.pythonhosted.org/packages/a0/27/29d66ed395a5c2c3a912332d446a54e2bc3277c36b0bbd22bc71623e0193/ipython-7.0.1-py3-none-any.whl
├── appnope https://files.pythonhosted.org/packages/87/a9/7985e6a53402f294c8f0e8eff3151a83f1fb901fa92909bb3ff29b4d22af/appnope-0.1.0-py2.py3-none-any.whl
├── backcall https://files.pythonhosted.org/packages/84/71/c8ca4f5bb1e08401b916c68003acf0a0655df935d74d93bf3f3364b310e0/backcall-0.1.0.tar.gz
├── decorator https://files.pythonhosted.org/packages/bc/bb/a24838832ba35baf52f32ab1a49b906b5f82fb7c76b2f6a7e35e140bac30/decorator-4.3.0-py2.py3-none-any.whl
├── jedi>=0.10 https://files.pythonhosted.org/packages/7a/1a/9bd24a185873b998611c2d8d4fb15cd5e8a879ead36355df7ee53e9111bf/jedi-0.13.1-py2.py3-none-any.whl
│ └── parso>=0.3.0 https://files.pythonhosted.org/packages/09/51/9c48a46334be50c13d25a3afe55fa05c445699304c5ad32619de953a2305/parso-0.3.1-py2.py3-none-any.whl
├── pexpect https://files.pythonhosted.org/packages/89/e6/b5a1de8b0cc4e07ca1b305a4fcc3f9806025c1b651ea302646341222f88b/pexpect-4.6.0-py2.py3-none-any.whl
│ └── ptyprocess>=0.5 https://files.pythonhosted.org/packages/d1/29/605c2cc68a9992d18dada28206eeada56ea4bd07a239669da41674648b6f/ptyprocess-0.6.0-py2.py3-none-any.whl
├── pickleshare https://files.pythonhosted.org/packages/9a/41/220f49aaea88bc6fa6cba8d05ecf24676326156c23b991e80b3f2fc24c77/pickleshare-0.7.5-py2.py3-none-any.whl
├── prompt-toolkit<2.1.0,>=2.0.0 https://files.pythonhosted.org/packages/e5/c5/f1ee6698bdcf615f171a77e81ca70293b16a6d82285f1760b388b4348263/prompt_toolkit-2.0.5-py3-none-any.whl
│ ├── six>=1.9.0 https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
│ └── wcwidth https://files.pythonhosted.org/packages/7e/9f/526a6947247599b084ee5232e4f9190a38f398d7300d866af3ab571a5bfe/wcwidth-0.1.7-py2.py3-none-any.whl
├── pygments https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl
├── setuptools>=18.5 https://files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl
├── simplegeneric>0.8 https://files.pythonhosted.org/packages/3d/57/4d9c9e3ae9a255cd4e1106bb57e24056d3d0709fc01b2e3e345898e49d5b/simplegeneric-0.8.1.zip
└── traitlets>=4.2 https://files.pythonhosted.org/packages/93/d6/abcb22de61d78e2fc3959c964628a5771e47e7cc60d53e9342e21ed6cc9a/traitlets-4.3.2-py2.py3-none-any.whl
├── decorator https://files.pythonhosted.org/packages/bc/bb/a24838832ba35baf52f32ab1a49b906b5f82fb7c76b2f6a7e35e140bac30/decorator-4.3.0-py2.py3-none-any.whl
├── ipython-genutils https://files.pythonhosted.org/packages/fa/bc/9bd3b5c2b4774d5f33b2d544f1460be9df7df2fe42f352135381c347c69a/ipython_genutils-0.2.0-py2.py3-none-any.whl
└── six https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl