This is a linux problem that's been stumping me for a while. On my work linux box (running RedHat) I was trying to install some Python packages. However, some funny business is occurring, see below:
This is my objective and first command
pip install scrapylib
This creates a 'Permission Denied' error
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/site-packages/scrapylib'
My knee-jerk reaction is to put "sudo" in front of the exact same command...
sudo pip install scrapylib
That generates this exception,
sudo: pip: command not found
In order to get around this, I type
sudo su - root
Now this command runs 100% successfully
pip2.7 install scrapylib
Finally, I change back to me
sudo su - uspowpow
Can anyone explain this phenomenon to me? I'm a recent college grad with nothing other than basic Linux knowledge, if someone could explain why placing "sudo" in front of a valid command makes it invalid, I would be extremely grateful (both for the fix and for the knowledge).
This can happen for several reasons. To pick a few:
Your command may not be in the PATH enforced by sudo. For a command in /usr/local/bin (where your pip appears to be), this is downright likely.
Your command may require an alias or shell function to be valid (perhaps you have pip aliased to pip2.7?), or may be a shell builtin itself (though this wouldn't be the case for pip). By default sudo directly invokes children using a member of the execv* family of syscalls, with no shell, so (1) shell functions and aliases aren't invoked under it all; and (2) even if you were using a shell, that shell wouldn't run your current user's dotfiles (being a noninteractive shell, if given a single command to run, it won't run most dotfiles).
To know what the actual cause is, the place to start is to find out what kind of command pip is. To do this, run:
$ type pip
If the answer you get is:
pip is /usr/local/bin/pip
...then you should look at the secure_path value set in your /etc/sudoers to ensure that it contains /usr/local/bin, or just run sudo /usr/local/bin/pip to sidestep the issue. By contrast, if you get:
pip is aliased to `pip2.7'
...then you know the problem is that it's an alias, and you need to run sudo pip2.7 (if the PATH is not also a problem).
Related
I'm taking a class on Coursera, this is what it's asking me to do.
brew info python
The section you are interested in is the location of where it was
installed.
Unversioned symlinks python, python-config, pip etc. pointing to
python3, python3-config, pip3 etc., respectively, have been
installed into /opt/homebrew/opt/python#3.9/libexec/bin
The /opt/homebrew/opt/python#3.x/libexec/bin is the one you want to
use and set for your path. Copy it from the terminal. Depending on the
version of Mac OS you are running the next steps may vary slightly.
When I enter in: /opt/homebrew/opt/python#3.9/libexec/bin
I get this:
zsh: permission denied: /opt/homebrew/opt/python#3.9/libexec/bin
Can anyone help?
You're not supposed to execute /opt/homebrew/opt/python#3.9/libexec/bin as an actual command. It's a directory.
You're supposed to add it to your PATH environment variable, so that you can run the commands contained in that directory.
Add this line to your ~/.zshrc file:
typeset -x PATH=/opt/homebrew/opt/python#3.9/libexec/bin:$PATH
Our server was updated to use the NFS system, and the previous /scratch path is now mapped to /local/scratch. My anaconda3 was installed under /scratch/xxx/anaconda3, which basically means it is now under /local/scratch/xxx/anaconda3. Intuitively, I have updated the PATH variable in .bash_profile, and running which conda actually gives me
/local/scratch/xxx/anaconda3/bin/conda
However, when I actually tried to run $conda, the system reports the error below:
-bash: /scratch/xxx/anaconda3/bin/conda: No such file or directory
which is quite weird and indicates an inconsistency with the result returned by which conda.
In addition, I tried to directly cd into /local/scratch/xxx/anaconda3/bin/ and execute $./conda. It reports
-bash: ./conda: /scratch/xxx/anaconda3/bin/python: bad interpreter: No such file or directory
My wild guess is that in addition to the environment variable in .bash_profile, there are some internal settings of anaconda stored elsewhere, because the system actually knows the right path for python, while conda tries to execute python from the old path (i.e., /scratch/xxx/anaconda3/bin/python in the above error message). However, I couldn't seemingly find this file and change it to get things work.
Somehow, I feel this should be a common issue, however, I did not find any solutions from the internet. Any suggestions would be greatly appreciated.
I have fixed my issue using cpr (credits to #merv). Here I post the detailed instructions, so if some one has the same problem, he/she can almost just follow them to get things fixed.
1. Installation
According to anaconda.org, the recommended command to install cpr is
$ conda install -c anaconda conda-prefix-replacement
You may find this to be actually very awkward; the reason we need cpr is that our conda is broken due to the prefix issue, if we can run conda command to install cpr, then why should we need cpr?
Instead, we may install cpr directly from github:
$ git clone https://github.com/conda/conda-prefix-replacement.git
$ cd conda-prefix-replacement
$ python setup.py install
2. Rehome
After installation, running the rehome command to fix the prefix issue is quite straightforward. Say your anaconda is moving from path_A to path_B, simply do
$ cpr rehome path_B --old-prefix path_A
In my case, the command should be
$ cpr rehome /local/scratch/xxx/anaconda3 --old-prefix /scratch/xxx/anaconda3
After running the cpr command, exit the terminal and re-login. Everything should be good now!
When I use pip it usually doesn't work without sudo. I often see people use pip without sudo, so what am I doing wrong?
I read that it is not recommended to install pip packages with sudo. I know that with virtualenv I can use pip without sudo, but to install virtualenv I have to use sudo first.
When I try to install pip without sudo, I get:
PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.4/dist-packages/pip'
When trying to install flask with pip3 install flask:
PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.4/dist-packages/werkzeug'
The reason is that your regular user doesn't have the permissions needed to modify the system directories. Just like in this message:
PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.4/dist-packages/pip'
Here is a brief rundown of what you need to know:
Your system has python installed, in order to modify the system's python you have to use sudo or be the root user.
You can install python libraries in your home directory without using sudo, but only you (not other users of the system) will be able to use it. Do this with pip install --user package-name like gongzhitaao mentioned.
You can also create unique installations of python in a directory of your choosing, like The Laughing Man mentioned. This is called virtualenv, and I think this is the most-preferred way to work.
sudo is used in Unix/Linux systems to perform tasks as another user, using their permissions, such as the ability to write to certain directories. When you don't specify the user to emulate, such as when running
sudo pip install flask
you are attempting to run the command as the system administrator, known as root in many environments. You'll be asked for the administrator password (which can be your own, if you've given your user admin privileges), then the specified command runs as that user, meaning that it has read/write access to essentially every file and directory on the system (there are some exceptions, but they're mostly corner cases and not very important here). This means that you need to be extra careful when using sudo, as an error as small as a single space can really mess things up: compare
sudo rm -rf /usr/local/lib/python3.4/dist-packages/numpy*
with
sudo rm -rf /usr /local/lib/python3.4/dist-packages/numpy*
See that space between /usr and local/? You just started deleting the entire /usr folder, which contains a large portion of the vital files and programs on the system. Hope you have a backup! Now, this doesn't mean you need to be scared to death of sudo, but you do need to have a healthy respect for it.
Python installations tend to be system-level (yes, I know there are exceptions), which means you need to use sudo to modify them, such as when installing 3rd-party modules with pip. If you run
ls -l /usr/local/lib/python3.4
you'll see something along the lines of
drwxrwsr-x 125 root 4096 Nov 3 00:40 dist-packages
showing that the directory you're trying to install to with pip is owned by root, so the use of sudo is necessary.
Now, there are a couple of ways around this. If you're comfortable with it, and don't mind modifying the system's global packages, go ahead and use sudo with pip (in fact, you may need to use sudo -H ... if you get a little message in yellow at the beginning about permissions in your home directory). All of your modules will be installed to /usr/local/lib/python3.4/dist-packages and be available to all users on the system.
The second option is to use pip's --user option, which will create a lib/python3.4/site-packages hierarchy in your home directory (~) and store any installed modules there, along with scripts in ~/bin (which you should add to your $PATH. The advantage of this method is that you don't need to use sudo, so you won't accidentally overwrite system-dependent modules where specific versions are required for other programs to run. The disadvantage is that the installed modules are only available to you, so you may run into issues if, for example, your web server is trying to run Flask as itself, and can't read the source files. However, that's nothing that a little config file-editing can't fix. This is my recommended solution for most users.
The third option is to use virtual environments like virtualenv. This will create a custom Python installation in the location of your choosing, with a separate python executable and site-packages hierarchy (there are options as to whether or not you want to link to or use the system's dist-packages repository). You can pip install packages directly into the virtualenv, and make as many environments as your little heart desires, each with slightly different versions of various dependencies, for example, so you can more robustly test your programs. You can turn on and off the virtual environments, so for example you could have a couple running in different tabs of Terminal, for example, testing things in parallel. This is a close second-place recommendation of mine, because there's (slightly) more work involved in activating and using the environments, and you may get confused about which one you're working in if you're not very good about naming them. Downsides include the lack of system-wide availability, like the second option, and the fact that the virtual environment needs to be manually activated before use.
So, take a look at the options, and see which is best for your system and your particular situation. Good luck!
I'm on a 64-bit Centos 5 Linux. This comes with Python2.4, but I needed 2.6 or higher to get psycopg2. I installed a virtual environment to run Python2.7.3 (based on this http://toomuchdata.com/2012/06/25/how-to-install-python-2-7-3-on-centos-6-2/) Now things seem to be working well when I'm logged in as root.
But I need to be able to use the virtualenv as a non-root user, joshmoe. (And start/stop postgresql as this other user)
The command for getting into the virtualenv is:
source /root/Python-2.7.3/projectdir/bin/activate
Understandably, when I run this command as joshmoe, I get a permission denied message:
-bash: /root/Python-2.7.3/projectdir/bin/activate: Permission denied
In an attempt to make this command available to joshmoe, I edited /etc/sudoers to include the following line:
joshmoe ALL=NOPASSWD: /root/Python-2.7.3/projectdir/bin/activate
Now from the joshmoe login, when I run:
sudo source /root/Python-2.7.3/projectdir/bin/activate
I get the following:
sudo: source: command not found
It seems pretty clear to me that I'm not using the sudoers file correctly to specify sourcing this file -- I seem to be giving access to that 'activate' file successfully, but not to the built-in source function. Either of following lines give syntax errors in sudoers:
joshmoe ALL=NOPASSWD: source
joshmoe ALL=NOPASSWD: source /root/Python-2.7.3/projectdir/bin/activate
I've found a good many examples of shudders statements. But I can't figure out the syntax that will make this work. I'm hoping once I get this one, I'll be able to give access to statements like 'service postgresql status' and so forth.
Your problem is that you user doesn't have root access to your virutalenv directory. You need to grant the user access for it to work, for example using:
sudo chmod o+x /root # grant access to root directory
sudo chmod o+rX /root/Python-2.7.3 -R # recursively grant access to everything in Python-2.7.3
source isn't a program, it's a shell butiltin which causes a script to be sourced in the current shell. sudo can only execute programs, not shell builtins, functions or aliases, that's why you get that error.
You can't run source as as sudo source; it's a built-in command for bash, not a program. What you want to do here is probably start a new shell as superuser using sudo -s.
I am trying to get the python-connect-mysql module to run, but when I execute python setup.py install on my command shell, I receive the following error:
[Errno 13] Permission denied: '/Library/Python/2.7/site-packages/test-easy-install-2259.write-test'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/Library/Python/2.7/site-packages/
I have Python 3.2 installed and need to install to /Library/Python/3.2/site-packages/
I have looked through many solutions, but do not understand the command shell very well. I tried the following to edit my bash_profile and received an error saying the file didn't exist:
sudo /Applications/TextEdit.app/Contents/MacOS/TextEdit ~/.bash_profile
Then I tried: touch ~/.bash_profile but didn't know what to do once I got there--the blog I read had very sketchy instructions--so I hit exit and it logged me out.
This seems to be the biggest hurdle for me to download third party packages. I tried the sudo /Applications/TextEdit.app... method again to see if the terminal would let me open up the bash_profile since I had touched it and received an error message that I do not have permission to open the file. Can someone please help? I am going crazy!
While using sudo is a possibility, you should always use virtualenv to manage your python modules in case somebody decides to replace setup.py with some kind of malicious code (and seriously, who looks at the code of setup.py everytime when installing a new module) that giving root rights to execute wouldn't exactly benefit your system.
Try using the Python 3 executable with elevated rights:
sudo python3 setup.py install
The path /Library/Python/* is used to store python modules available for all your system globally, and these directories are created from root user, that's why you get the "permission denied" error. Just use sudo.
sudo python setup.py install
Also you may have a look at virtualenv