I made a python (3) package and I have been trying to upload it on Github. I also know how to push and install a git using pip. To test if it works as anticipated, I made a virtual environment on my local computer (linux) and pip installed my already pushed private package in there without a problem.
The issue is that I don't know how to access it!!! (I know how to activate and use virtualenvs; I don't know how to call my package) My package has a main interface that one would need to call it in terminal as follows:
python3 myui.py some_args *.data
and it's supposed to create some files where it's called. In other words, it's not exactly a module like numpy to be imported. I have watched/read many tutorials and documentations on the web and tbh I'm lost here.
You are looking for the -m flag. If you installed everything correctly, then the following command should allow you to run your script (based on your example). Note that you shouldn't add the file extension '.py'.
python3 -m myui some args *.data
If you have an actual package (directory with __init__.py file and more) instead of a module (a single .py file), then you can add a __main__.py file to that package. Python will execute this script when you use the -m flag with the package's name, in the same way as shown above.
python3 -m mypackage some args *.data
If you want to run a different script that is nested somewhere inside of that package, you can still run it by specifying its module name:
python3 -m mypackage.subpackage.myscript some args *.data
Another common way to make your script available uses the setup script (setup.py) or setup configuration file (setup.cfg) that is used to install the module or package. In that case, you can add an entry point to map a command to a specific module/function/etc. (as described in this Python packaging tutorial) so that you can run that command instead of having to use the -m flag with Python.
$ mycommand some args *.data
Related
In our project, we are using Click to create command-line interfaces. We have a folder called foo in the project, and __main__.py file there contains Click.group definition.
For now, we can access the cli using -
python foo [OPTIONS]
but, we wish to simplify it to just -
foo [OPTIONS]
I had a look at Setuptools integration using Click, and I was successful in creating command after following the instructions. But, what steps are to be done next is unclear to me.
After I introduce setup.py to the project, do I need to ask everyone using the project to run pip install --editable .?
What happens if someone adds new commands/options in the project? Will all users again need to run pip commands?
The project will always run in UNIX environment and we are wondering if there exists a shebang-way of running foo/__main__.py with just foo. If we had foo (not foo.py) as file containing Click.group definition, adding a shebang at the top of file would remove the need to use python while invoking commands.
pip install --editable . means that if you manipulate the files inside the package, it will have immediately an effect when you use the package in Python, because import yourpackagename will refer to the folder with the original package.
pip install . will copy the package folder into some other path and use that when you use import yourpackagename, so any modifications to the original package files will not have an effect until you reinstall it.
I want to build an application, say it's called helloworld, with a command line interface in python. My application as multiple nested modules and a top level module main.py.
Say the layout is :
helloworld/
helloworld/
__init__.py
module1/
module2/
module3/
setup.py
main.py
At this point, once my project is installed I can run it using
#> python path/to/helloworld/main.py arg1 arg2 arg3
I want to be able to interact with the program using commands such as :
#> helloworld arg1 arg2 arg3, in a similar manner as for example, the Flask framework comes with a command line application I can use with #> flask run.
I looked around and found questions suggesting using a shebang. As far as I can tell, this is not what I want.
I want a self contained application I can install using pip and then launch directly without any reference to the python interpreter.
How should I go about this ?
EDIT
It is an important requirement to me that the package can be installed using pip and the command line interface is immediately available. Shebangs/fiddling manually with the path/creating an additional shell script do not solve my problem.
You should add main.py to your PATH. What happens when you are running flask run is that your teminal looks up the command flask in PATH and runs the program that it is pointing to. You could see it as a kind of shortcut to the program Flask.
By adding your program to your PATH, you can tell the computer that if you type helloworld in your terminal, the terminal should run /my/path/to/helloworld.py.
I don't know what OS you are on, so here are links for most common OS on how to add a PATH variable.
Windows
Linux
Mac OSX
Edit: After you gave more explanation, setuptools is what you are looking for. Please look at this reference to see how to use it.
There's an explanation here of how to tell setuptools to add a script to the Python Scripts-folder during the pip installation. You can then make that script be for example a bat-file that calls your Python-script. In that way you achieve both that you do not need to write .py for your script to run, and that it happens automatically.
I know this has already been answered, but just in case someone comes across this in the future..
Here's what worked for a new package I'm developing after reading up on this from many different sources.
tldr; use setuptools and add your command as a console_scripts option under [options.entry_points] in setup.cfg.
You will also need some other fields in setup.cfg or setup.py for package discovery to work. Check out the setuptools docs for that.
Example entry point:
[options.entry_points]
console_scripts =
mycmd = mypackage.mymodule:myfunc
For local development, you can install the package in editable (development) mode with pip install -e . which will allow you to test the command as if you pip intalled it.
Alternatively, you could create a __main__.py that initializes your module when you run python -m mypackage from the root.
from mypackage import app
if __name__ == '__main__':
app.main()
I personally opted for both. Usually your main.py (or app.py in my example) belongs in your package not project level dir. Then you would create __main__.py there as well.
I want to add a specific library path only to python2. After adding export PYTHONPATH="/path/to/lib/" to my .bashrc, however, executing python3 gets the error: Your PYTHONPATH points to a site-packages dir for Python 2.x but you are running Python 3.x!
I think it is due to that python2 and python3 share the common PYTHONPATH variable.
So, can I set different PYTHONPATH variables respectively for python2 and python3. If not, how can I add a library path exclusively to a particular version of python?
PYTHONPATH is somewhat of a hack as far as package management is concerned. A "pretty" solution would be to package your library and install it.
This could sound more tricky than it is, so let me show you how it works.
Let us assume your "package" has a single file named wow.py and you keep it in /home/user/mylib/wow.py.
Create the file /home/user/mylib/setup.py with the following content:
from setuptools import setup
setup(name="WowPackage",
packages=["."],
)
That's it, now you can "properly install" your package into the Python distribution of your choice without the need to bother about PYTHONPATH. As far as "proper installation" is concerned, you have at least three options:
"Really proper". Will copy your code to your python site-packages directory:
$ python setup.py install
"Development". Will only add a link from the python site-packages to /home/user/mylib. This means that changes to code in your directory will have effect.
$ python setup.py develop
"User". If you do not want to write to the system directories, you can install the package (either "properly" or "in development mode") to /home/user/.local directory, where Python will also find them on its own. For that, just add --user to the command.
$ python setup.py install --user
$ python setup.py develop --user
To remove a package installed in development mode, do
$ python setup.py develop -u
or
$ python setup.py develop -u --user
To remove a package installed "properly", do
$ pip uninstall WowPackage
If your package is more interesting than a single file (e.g. you have subdirectories and such), just list those in the packages parameter of the setup function (you will need to list everything recursively, hence you'll use a helper function for larger libraries). Once you get a hang of it, make sure to read a more detailed manual as well.
In the end, go and contribute your package to PyPI -- it is as simple as calling python setup.py sdist register upload (you'll need a PyPI username, though).
You can create a configuration file mymodule.pth under lib/site-packages (on Windows) or lib/pythonX.Y/site-packages (on Unix and Macintosh), then add one line containing the directory to add to python path.
From docs.python2 and docs.python3:
A path configuration file is a file whose name has the form name.pth and exists in one of the four directories mentioned above; its contents are additional items (one per line) to be added to sys.path. Non-existing items are never added to sys.path, and no check is made that the item refers to a directory rather than a file. No item is added to sys.path more than once. Blank lines and lines beginning with # are skipped. Lines starting with import (followed by space or tab) are executed.
I found that there is no way to modify PYTHONPATH that is only for python2 or only for python3. I had to use a .pth file.
What I had to do was:
make sure directory is created in my home: $HOME/.local/lib/python${MAJOR_VERSION}.${MINOR_VERSION}/site-packages
create a .pth file in that directory
test that your .pth file is work
done
For more info on `.pth. file syntax and how they work please see: python2 docs and python3 docs.
(.pth files in a nutshell: when your python interpreter starts it will look in certain directories and see the .pth file, open those files, parse the files, and add those directories to your sys.path (i.e. the same behavior as PYTHONPATH) and make any python modules located on those directories available for normal importing.)
If you don't want to bother with moving/adding documents in lib/site-packages, try adding two lines of code in the python2.7 script you would like to run (below.)
import sys
sys.path = [p for p in sys.path if p.startswith(r'C:\Python27')]
This way, PYTHONPATH will be updated (ignore all python3.x packages) every time you run your code.
I'd like to start developing an existing Python module. It has a source folder and the setup.py script to build and install it. The build script just copies the source files since they're all python scripts.
Currently, I have put the source folder under version control and whenever I make a change I re-build and re-install. This seems a little slow, and it doesn't settle well with me to "commit" my changes to my python install each time I make a modification. How can I cause my import statement to redirect to my development directory?
Use a virtualenv and use python setup.py develop to link your module to the virtual Python environment. This will make your project's Python packages/modules show up on the sys.path without having to run install.
Example:
% virtualenv ~/virtenv
% . ~/virtenv/bin/activate
(virtenv)% cd ~/myproject
(virtenv)% python setup.py develop
Virtualenv was already mentioned.
And as your files are already under version control you could go one step further and use Pip to install your repo (or a specific branch or tag) into your working environment.
See the docs for Pip's editable option:
-e VCS+REPOS_URL[#REV]#egg=PACKAGE, --editable=VCS+REPOS_URL[#REV]#egg=PACKAGE
Install a package directly from a checkout. Source
will be checked out into src/PACKAGE (lower-case) and
installed in-place (using setup.py develop).
Now you can work on the files that pip automatically checked out for you and when you feel like it, you commit your stuff and push it back to the originating repository.
To get a good, general overview concerning Pip and Virtualenv see this post: http://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django
Install the distrubute package then use the developer mode. Just use python setup.py develop --user and that will place path pointers in your user dir location to your workspace.
Change the PYTHONPATH to your source directory. A good idea is to work with an IDE like ECLIPSE that overrides the default PYTHONPATH.
Is it possible to run Python egg files directly as you can run jar files with Java?
For example, with Java you might dos something like:
$ java -jar jar-file
A python egg is a "a single-file importable distribution format". Which is typically a python package.
You can import the package in the egg as long as you know it's name and it's in your path.
You can execute a package using the "-m" option and the package name.
However, python packages generally do not do anything when executed, and you may get an error. The -c option can be used to run code. (See http://docs.python.org/using/cmdline.html for details on command line options)
> python -m sphinx
sphinx is a package and cannot be directly executed
> python -c "import <package in an egg>; <function>();"
> python -c "import sphinx; print sphinx.package_dir"
C:\Python26\lib\site-packages\sphinx-0.6.1-py2.6.egg\sphinx
As of Python 2.6, you can use python some.egg and it will be executed if it includes a module named __main__.
For earlier versions of Python, you can use PYTHONPATH=some.egg python -m some module, and somemodule from the egg will be run as the main module. (Note: if you're on Windows, you'd need to do a separate SET PYTHONPATH=some.egg.)
For example, if you want to import the suds module which is available as .egg file:
egg_path='/home/shahid/suds_2.4.egg'
sys.path.append(egg_path)
import suds
#... rest of code
Python Egg file direct execution steps
Suppose if you have egg file and driver file to run through below command.
PYTHONPATH=eggfilename.egg python driverfile.py
above command for without install egg file with python code.