Pydoop Installation on CentOS 6 - python

I am trying to install Pydoop (https://crs4.github.io/pydoop/installation.html) on CentOS 6 (Hortonworks HDP physical cluster, 4 nodes). I have Python 3.4 and Python 2.6 installed on my master. When I install it using Python 3 it shows syntax error like below:
[root#abc]# python3 setup.py build
File "setup.py", line 45
print 'using setuptools version', setuptools.__version__
^
SyntaxError: Missing parentheses in call to 'print'
When I install it using Python 2 it shows following error
[root#abc]# python setup.py build
using setuptools version 35.0.2
Traceback (most recent call last):
File "setup.py", line 65, in <module>
import pydoop
File "/root/pydoop/pydoop/__init__.py", line 42, in <module>
_HADOOP_INFO = _PATH_FINDER.find() # fill the cache ASAP
File "/root/pydoop/pydoop/hadoop_utils.py", line 624, in find
info[a] = getattr(self, a)()
File "/root/pydoop/pydoop/hadoop_utils.py", line 433, in hadoop_home
_hadoop_home_from_version_cmd() or
File "/root/pydoop/pydoop/hadoop_utils.py", line 394, in
_hadoop_home_from_version_cmd
output = sp.check_output([hadoop_exec, 'version'])
AttributeError: 'module' object has no attribute 'check_output'
I don't want to upgrade default python from 2.6 to 2.7 as it may break yum which uses python2.6 and upgrading it may cause problems. Any suggestions please?

You can install Python 2.7 and pip on CentOS 6 without breaking yum:
# yum update
# yum install centos-release-SCL epel-release
# yum install python27 python27-devel
# echo ". /opt/rh/python27/enable" >/etc/profile.d/python.sh
# source /etc/profile
# curl -o get-pip.py https://bootstrap.pypa.io/get-pip.py
# python get-pip.py
# python --version
Python 2.7.13
# pip --version
pip 9.0.1 from /opt/rh/python27/root/usr/lib/python2.7/site-packages (python 2.7)

Related

Centos 7 and netsnmp for python 3

I installed python3 on centos 7 since I should start moving my scripts from python 2 to 3.
yum install python3
# rpm -qa |grep python3
python3-libs-3.6.8-13.el7.x86_64
python3-setuptools-39.2.0-10.el7.noarch
python3-pip-9.0.3-7.el7_7.noarch
python3-3.6.8-13.el7.x86_64
I have a script that imports netsnmp and uses it. Works fine under python 2 but does not under python 3.
# rpm -qa |grep python |grep -i snmp
net-snmp-python-5.7.2-48.el7_8.1.x86_64
Under python3 I get this error:
Traceback (most recent call last):
File "up.py", line 52, in <module>
oid = netsnmp.Varbind('.1.3.6.1.2.1.1.5.0')
AttributeError: module 'netsnmp' has no attribute 'Varbind'
Is there a way to get netsnmp working with python 3 under centos 7 with minimal fuss and without breaking things?
Just run.
yum install python2 -y
Then
yum install python36 -y
It should work due to it covered or updates required dependencies.

Ubuntu 17: Can't install mysqlclient

Trying to follow a Django tutorial but I cannot install mysqlclient.
The tutorial claims that I can do so with the following command:
pip install mysqlclient
but this generates this error:
Collecting mysqlclient Using cached mysqlclient-1.3.12.tar.gz
Complete output from command python setup.py egg_info:
/bin/sh: 1: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-rrolctwh/mysqlclient/setup.py", line 17, in <module>
metadata, options = get_config()
File "/tmp/pip-build-rrolctwh/mysqlclient/setup_posix.py", line 44, in get_config
libs = mysql_config("libs_r")
File "/tmp/pip-build-rrolctwh/mysqlclient/setup_posix.py", line 26, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
OSError: mysql_config not found
---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-rrolctwh/mysqlclient/
I have the most up-to-date pip and virtualenv installed.
I would like to be able to install mysqlclient so that I may continue with the tutorial.
You should also install the mysql and python development headers and libraries:
https://github.com/PyMySQL/mysqlclient-python#prerequisites
I was facing the same problems, but following the instructions in the Official mysqlclient documentation fixed it for me
but just to clarify I was running python 3.5 from a virtual environment
and after installing the prerequisites, it all worked fine
The following solved it for me :
You may need to install the Python 3 and MySQL development headers and libraries like so:
$ sudo apt-get install python3-dev default-libmysqlclient-dev build-essential # Debian / Ubuntu
% sudo yum install python3-devel mysql-devel # Red Hat / CentOS
Then you can install mysqlclient via pip now:
$ pip install mysqlclient
Source : https://github.com/PyMySQL/mysqlclient#prerequisites
I had also similar issue on Centos 6, where there was a problem with mysql migration to maria, I had some conflicts, but finally I installed:
yum list installed |grep MariaDB
MariaDB-client.x86_64 10.2.7-1.el6 #bull
MariaDB-common.x86_64 10.2.7-1.el6 #bull
MariaDB-compat.x86_64 10.2.7-1.el6 #bull
MariaDB-devel.x86_64 10.2.7-1.el6 #bull
MariaDB-server.x86_64 10.2.7-1.el6 #bull
And the issue was resolved.

pip installing packages failed

With PyCharm I tried to add a project interpreter for my Docker container
but then I get this incomprehensible error.
Invalid requirement: 'redis\>=2.10.5'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 82, in __init__
req = Requirement(req)
File "/usr/local/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py", line 96, in __init__
requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "'\\>=2.10.'
What can I do about it?
From the official documentation of pip:
https://pip.pypa.io/en/stable/reference/pip_install/#examples
$ pip install SomePackage # latest version
$ pip install SomePackage==1.0.4 # specific version
$ pip install 'SomePackage>=1.0.4' # minimum version
It seems you have to use apostrophes for a minimum version.
Backslash in redis\>=2.10.5 is the culprit of the problem. Replace redis\>=2.10.5 with 'redis>=2.10.5'.

How to build scikit-image in travis-ci?

I am trying to build a script that makes use of scikit-image on Travis-ci. It complains that six is not up to date. Just to be sure, I tried to replicate the .travis.yml of the scikit-image project on github, but it did nothing to change the situation. I also tried to update six, but to no avail...
Does someone know how to build scikit-image on travis?
My requirements.txt:
# Requirements list:
scikit-image
My trace:
$ source ~/virtualenv/python3.3/bin/activate
$ python --version
Python 3.3.5
$ pip --version
pip 6.0.7 from /home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages (python 3.3)
5.80s$ pip install -r requirements.txt
You are using pip version 6.0.7, however version 6.0.8 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting scikit-image (from -r requirements.txt (line 2))
Downloading scikit-image-0.10.1.tar.gz (16.7MB)
100% |################################| 16.7MB 20kB/s
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 110, in <module>
check_requirements()
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 105, in check_requirements
% ((package_name, ) + min_version))
ImportError: You need `six` version 1.3 or later.
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 110, in <module>
check_requirements()
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 105, in check_requirements
% ((package_name, ) + min_version))
ImportError: You need `six` version 1.3 or later.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-wsd_vd/scikit-image
The command "pip install -r requirements.txt" failed and exited with 1 during .
Your build has been stopped.
If you get an error that a specific dependency isn't being met, being more specific about that dependency usually helps. I like to put dependencies that I don't directly use in a before_install section.
before_install:
pip install six>=1.3
install:
pip install -r requirements.txt
pip allows you to specify constraints on the versions of packages you install. If you had to use 1.3 exactly you could use ==.

Install pip failed

i am using redhat 5.3
I using following command to update python to 2.7. (it was python 2.4 before update)
# xz -d Python-2.7.6.tar.xz
# tar xvf Python-2.7.6.tar
# cd Python-2.7.6/
# ./configure
# make && make install
And I Install the python-setuptools
# yum install python-setuptools
Then I install the pip print the error
^
SyntaxError: invalid syntax
File "/usr/lib/python2.4/site-packages/pip-1.5.6-py2.4.egg/pip/_vendor/colorama/__init__.py", line 2
from .initialise import init, deinit, reinit
^
SyntaxError: invalid syntax
File "/usr/lib/python2.4/site-packages/pip-1.5.6-py2.4.egg/pip/_vendor/colorama/winterm.py", line 2
from . import win32
^
SyntaxError: invalid syntax
Adding pip 1.5.6 to easy-install.pth file
Installing pip script to /usr/bin
Installing pip2 script to /usr/bin
Installing pip2.4 script to /usr/bin
Installed /usr/lib/python2.4/site-packages/pip-1.5.6-py2.4.egg
Processing dependencies for pip
And I run the command pip , it print following errors:
# pip
Traceback (most recent call last):
File "/usr/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2603, in <module>
File "/usr/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 666, in require
File "/usr/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 565, in resolve
pkg_resources.DistributionNotFound: pip==1.5.6
And I try to uninstall that, it tell me that:
[root#linkea-dev-srv1 /]# yum remove pip
Loaded plugins: fastestmirror
Repository base is listed more than once in the configuration
Setting up Remove Process
No Match for argument: pip
Loading mirror speeds from cached hostfile
No Packages marked for removal
You have new mail in /var/spool/mail/root
So I try to reinstall pip by python get-pip.py.
It also has error.
# python get-pip.py
Traceback (most recent call last):
File "get-pip.py", line 17474, in <module>
main()
File "get-pip.py", line 17466, in main
bootstrap(tmpdir=tmpdir)
File "get-pip.py", line 17406, in bootstrap
import pip
File "/tmp/tmpB9jhvw/pip.zip/pip/__init__.py", line 9, in <module>
File "/tmp/tmpB9jhvw/pip.zip/pip/log.py", line 9, in <module>
File "/tmp/tmpB9jhvw/pip.zip/pip/_vendor/colorama/__init__.py", line 2, in <module>
File "/tmp/tmpB9jhvw/pip.zip/pip/_vendor/colorama/initialise.py", line 5, in <module>
File "/tmp/tmpB9jhvw/pip.zip/pip/_vendor/colorama/ansitowin32.py", line 6, in <module>
File "/tmp/tmpB9jhvw/pip.zip/pip/_vendor/colorama/winterm.py", line 2, in <module>
File "/tmp/tmpB9jhvw/pip.zip/pip/_vendor/colorama/win32.py", line 7, in <module>
File "/usr/local/lib/python2.7/ctypes/__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: No module named _ctypes
You should be very careful when upgrading Python on RedHat (including Fedora/CentOS) because a large majority of the system applications rely on the bundled version of Python. This is especially true of yum, which relies on the version of Python that comes bundled with the operating system.
In short - if you upgrade the version of Python on RedHat/CentOS/Fedora, yum will not work correctly.
To install a different version of Python on these distributions, you have two main options. One is to find someone that has released a rpm, download an install that; but you may have to deal with dependencies on your own.
The other (simpler) option is to download the source of Python, compile it and then install it separately (using altinstall).
You'll need to install the following packages first:
yum groupinstall "Development tools"
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel
Next, download the source and extract it. As of this writing, the latest version is 2.7.8:
wget "https://www.python.org/ftp/python/2.7.8/Python-2.7.8.xz"
tar xvf Python-2.7.8.xz
cd Python-2.7.8
Now you have to build and install Python to an alternate location, with the following:
./configure --prefix=/usr/local
make && make altinstall
Once this is done, you'll have /usr/local/bin/python2.7 available to you. Next, you'll want to install pip for this version of Python. To do that, first install setuptools:
wget "https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py"
/usr/local/bin/python2.7 ez_setup.py
/usr/local/bin/easy_install-2.7 pip
The final step is to make sure that this version of Python is available to you. You can add /usr/local/bin/ to your $PATH by modifying your shell's environment (for example, adding it in $HOME/.bashrc); or you can as root create symbolic links in /usr/bin/ to these utilities. Make sure you give them version-specific tags:
ln -sf /usr/local/bin/python2.7 /usr/bin/python2.7
ln -sf /usr/local/bin/pip /usr/local/bin/pip2.7
This will keep your default Python to the system's version (recommended), and then give you the option to use 2.7 when needed.

Categories