I would like to install some Python module, namely the MATLAB Engine. MATLAB is installed globally under /usr/local/sw/matlab, so I change to the corresponding directory and then run setup.py as explained in the linked documentation. However, I am not root, so I added the --user flag, as documented by the official Python Docs.
So this is what I get:
> python setup.py install --user
running install
running build
running build_py
creating build
error: could not create 'build': Permission denied
Apparently it tries to build the module in the current directory, where I have no write access. I can sidestep this by running
python setup.py build --build-base=/path/to/temp/build/dir
But then I don't know how to install that. Also I cannot combine those 2 approaches as install does not recognize the --build-base option.
I furthermore tried to copy the whole matlabroot/extern/engines/python directory somewhere else and run python setup.py install --user there, but this yields
error: The installation of MATLAB is corrupted. Please reinstall MATLAB or contact Technical Support for assistance.
I guess it does not find the matlabroot anymore.
So how do I build and install some source located somewhere under /usr... without being root?
You can use Virtualenv or venv to deal with this kind of issues.
Edit: As we went a bit deeper into issue itself, we managed to figure out the answer.
If you wish to use virtual environment: python setup.py build --build-base="/path/to/directory_you_can_write_to" install --prefix="/path/to/virtualenv/" --user
If you wish to accomplish what #Feodoran asked for: python setup.py build --build-base="/path/to/directory_you_can_write_to" install --user
Useful links:
https://de.mathworks.com/help/matlab/matlab_external/install-matlab-engine-api-for-python-in-nondefault-locations.html
https://python.org/3.2/install
Related
I deploy my app with virtualenv --no-site-packages and pip install -r requirements.txt. There is feedgen module in requirements, that has site-packages/feedgen/ext directory with extensions. Among other, I deploy foobar.py extension for feedgen, and the answer is:
What is the best practice to deploy module's lib? In case of, e.g., Heroku, AWS,etc
Is it ok to use setup.py? Or Makefile? Or shell-script to copy file? Or what else?
PS I,m not ready to contribute to feedgen yet, as my code is still 'smells'. (I understand that so the problem would be solved by itself)
The most popular practice is to put a module on pypi and be able to install it with pip. The second way is cloning from github and running python setup.py install. The second way is usually used for installation from source and the first often provides compiled wheels that do not need compilation.
If you do not want to share the code you could use installation from files in pip and use private git repositories instead of github.
An example from pyarrow:
Usual installation from pip is written on this page:
pip install pyarrow
Installation from source. Here it is a Cython module so the author uses build_ext instead of install
git clone https://github.com/apache/arrow.git
...
cd arrow/python
...
python setup.py build_ext --inplace
today I attempted to remove a file after my package (a python wheel) was installed via pip with the -t --target option.
Post-install script with Python setuptools
I am subclassing install in my setup.py like this:
class PostInstallCommand(install):
"""Post-installation for installation mode."""
def run(self):
install.run(self)
# here I am using
p = os.path.join(self.install_libbase,"myPackage/folder/removeThisPyc.pyc")
if os.path.isfile(p):
os.unlink(p)
#there is also self.install_platlib and
#self.install_purelib which seem to be used by pip distutil scheme
#Have not tested those yet
when running
python setup.py install
this works the file is removed upon install.
But through
pip install path-to-my-wheel.whl
this does not work and the file is still there.
pip install -t /target/dir path-to-my-wheel.whl
does not work either...
So question is, what is pip doing with distutils and or setuptools and how can make this work?
Another thing I noticed is that pip does not seem to be printing anything, I am printing in my setup.py in verbose mode?
Is there away to see the full output from python instead of the "pip" only stuff?
Reading educates:
http://pythonwheels.com/
2. Avoids arbitrary code execution for installation. (Avoids setup.py)
As I am using wheels and wheels wont execute the setup.py, my concept of doing this is rubbish.
https://github.com/pypa/packaging-problems/issues/64
I guess this is between deployment and installation, though I would obviously count my little change to installation...
Is there a way to avoid pyc file creation upon a pip install whl ?
Two options in setup.py develop and install are confusing me. According to this site, using develop creates a special link to site-packages directory.
People have suggested that I use python setup.py install for a fresh installation and python setup.py develop after any changes have been made to the setup file.
Can anyone shed some light on the usage of these commands?
python setup.py install is used to install (typically third party) packages that you're not going to develop/modify/debug yourself.
For your own stuff, you want to first install your package and then be able to frequently edit the code without having to re-install the package every time — and that is exactly what python setup.py develop does: it installs the package (typically just a source folder) in a way that allows you to conveniently edit your code after it’s installed to the (virtual) environment, and have the changes take effect immediately.
Note: It is highly recommended to use pip install . (regular install) and pip install -e . (developer install) to install packages, as invoking setup.py directly will do the wrong things for many dependencies, such as pull prereleases and incompatible package versions, or make the package hard to uninstall with pip.
Update:
The develop counterpart for the latest python -m build approach is as follows (as per):
From the documentation. The develop will not install the package but it will create a .egg-link in the deployment directory back to the project source code directory.
So it's like installing but instead of copying to the site-packages it adds a symbolic link (the .egg-link acts as a multiplatform symbolic link).
That way you can edit the source code and see the changes directly without having to reinstall every time that you make a little change. This is useful when you are the developer of that project hence the name develop. If you are just installing someone else's package you should use install
Another thing that people may find useful when using the develop method is the --user option to install without sudo. Ex:
python setup.py develop --user
instead of
sudo python setup.py develop
I have a setup.py file that builds an extension. If I change one of the source files, distutils recognizes this and rebuilds the extension, showing all the compile / link commands.
However, if the only thing I change is setup.py (I'm fiddling trying to make library dependencies work), then it doesn't seem to rebuild (e.g., none of the compile/link commands show up). I've tested this by removing one of the source files in the line
sources = ['foo.c', 'bar.c' ...]
and when I pip install -e . or python setup.py install, it still creates a new file for the extension, but it must be a version cached somewhere, since it shouldn't compile.
How do I clear this cache? I have tried
python setup.py clean --all
or using the --ignore-installed and --no-cache-dir flags when doing pip install -e .
The only way I have found to make it rebuild is if I add garbage in a source file, triggering a rebuild and error, remove the garbage, and pip install -e . again...
Just delete under site-packages path any file related to it, you may find sometimes more than one version or some files packaged as zip files or run the following command python setup.py clean --all.
Recompile and install again.
But I will recommend to use python setup.py develop so you don't need to reinstall it with every change, you will be able to frequently edit your code and not have to re-install it again. python setup.py install is used to install typically a ready to use third-party packages.
Check here to better understand python packaging.
Summary:
python setup.py clean --all
python setup.py develop
I needed to run
python setup.py clean --all
python setup.py develop
Thanks to DhiaTN for getting me there.
I am trying to install YAML module (PyYAML3.10) for Python 2.7 on my mac. I get the foll error message:
$ python setup.py install
running install
running build
running build_py
running build_ext
running install_lib
creating /Library/Python/2.7/site-packages/yaml
error: could not create '/Library/Python/2.7/site-packages/yaml': Permission denied
Can anyone help me out with this?
Thanks!
It's a matter of permissions... Try with: sudo python setup.py install
The error is very clear:
error: could not create '/Library/Python/2.7/site-packages/yaml': Permission denied
You don't have permission to write to this directory. If you run setup.py using sudo, you'll run it with root privileges, which will let you install it:
sudo python setup.py install
You may want to look into something like macports, which gives you a convenient way of installing (and uninstalling, ugrading, etc) a variety of open source software. It includes packaged versions of the YAML module.
I would suggest that you use pip to install the package and virtualenv to make pip install packages as your user. This way you won't get anymore Permission denied messages. See this guide for learning more about virtualenv. It might seem a bit hard to grip at first but I promise you that the effort is well worth it.