We Keep Coding

Error installing pyspark - SyntaxError: invalid syntax

I am not able to install pyspark or upgrade pip. I keep running into syntax error. I do not have this issue if I use pip3 however the program that runs the code must use python27
#### Installing pyspark using root
$python -V
Python 2.7.18
$pip install pyspark
Collecting pyspark
Using cached
https://files.pythonhosted.org/packages/b8/01/b2393cee7f6180d9150274e92c8bdc1c81220e2ad7554ee5febca1866899/pyspark-3.3.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-l7QVd5/pyspark/setup.py", line 31
file=sys.stderr)
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-l7QVd5/pyspark/
You are using pip version 8.1.2, however version 22.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
##### Upgrading pip using root
$pip install --upgrade pip
Cache entry deserialization failed, entry ignored
Collecting pip
Downloading
https://files.pythonhosted.org/packages/4b/b6/0fa7aa968a9fa4ef63a51b3ff0644e59f49dcd7235b3fd6cceb23f202e08/pip-22.1.2.tar.gz (2.1MB)
100% |████████████████████████████████| 2.1MB 388kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-9iqM1O/pip/setup.py", line 7
def read(rel_path: str) -> str:
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-9iqM1O/pip/
You are using pip version 8.1.2, however version 22.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

My Kali linux showing 'No module named importlib' 'from importlib import import_module'

Trying to install build dependencies with pip which is installing in /usr/local/lib/python2.7/dist-packages
As I try to install proceed gives an error
Collecting git+https://github.com/kti/python-netfilterqueue (from -r requirements.txt (line 1))
Cloning https://github.com/kti/python-netfilterqueue to /tmp/pip-req-build-VQbxfT
Running command git clone -q https://github.com/kti/python-netfilterqueue /tmp/pip-req-build-VQbxfT
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python2 /usr/lib/python2.7/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpNlNAvr
cwd: /tmp/pip-req-build-VQbxfT
Complete output (4 lines):
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/_vendor/pep517/_in_process.py", line 16, in <module>
from importlib import import_module
ImportError: No module named importlib
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python2 /usr/lib/python2.7/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpNlNAvr Check the logs for full command output.
so I try to install importlib with pip and I still get the same error. Then I install it directly to /usr/lib/python2.7/dist-packages/pip/_vendor/pep517/ using
pip install importlib --target=/usr/lib/python2.7/dist-packages/pip/_vendor/pep517/
because pip has been installing all packages to /usr/local/lib/python2.7/dist-packages,
but I still get the error either way

Assuming you are trying to install python-netfilterqueue.
You can visit github page and see this
The current version of NetfilterQueue requires Python 3.6 or later.
So you can't install this lib for python 2.7
But you can install 0.9.0 as it also stated that
The last version with support for Python 2.7 was 0.9.0.
pip install NetfilterQueue=0.9.0

Try switching to python3 because this will definitely work.
First you have to install cython.
pip install cython
Then after that you need to make sure you install NetfilterQueue from source.
git clone https://github.com/oremanj/python-netfilterqueue
Navigate to the netfilterqueue directory.
cd python-netfilterqueue
Then run this command.
python3 setup.py install
That's it. It should work properly

FileNotFoundError: [Errno 2] No such file or directory: setup.py when importing a custom package from GitHub [duplicate]

When running pip install, I'm getting errors like this:
$ pip install pycairo
Downloading/unpacking pycairo
You are installing a potentially insecure and unverifiable file. Future versions of pip will default to disallowing insecure files.
Downloading pycairo-1.10.0.tar.bz2 (246kB): 246kB downloaded
Running setup.py egg_info for package pycairo
Traceback (most recent call last):
File "<string>", line 16, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '/home/naught101/.virtualenvs/virtualenv-3/build/pycairo/setup.py'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '/home/naught101/.virtualenvs/virtualenv-3/build/pycairo/setup.py'
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/naught101/.virtualenvs/virtualenv-3/build/pycairo
Storing complete log in /home/naught101/.pip/pip.log
This is happening for pycairo, sip.
$ pip --version
pip 1.4.1 from /home/naught101/.virtualenvs/virtualenv-3/lib/python3.3/site-packages (python 3.3)

pycairo is not built by setuptools, and therefore can't be installed by pip.
As the INSTALL documentation says:
Install Procedure
-----------------
$ ./waf --help # shows available waf options
$ ./waf configure # use --prefix and --libdir if necessary
# --prefix=/usr --libdir=/usr/lib64 for Fedora 64-bit
$ ./waf build
$ ./waf install
Use
$ python3 ./waf ...
if you have python2 and python3 installed, and the default is python 2.
You can still use pip to fetch it:
$ pip install --download=. pycairo
Downloading/unpacking pycairo
Downloading pycairo-1.10.0.tar.bz2 (246kB): 246kB downloaded
$ tar xf pycairo-1.10.0.tar.bz2
$ cd pycairo-1.10.0
$ python ./waf etc.
Just make sure you use the Python from your virtualenv, and pass the right options to the configure command (--prefix=$VIRTUAL_ENV should be sufficient, but don't quote me on that; you might also have to set the PYTHON environment variable to the path to your virtualenv Python, or other things).
Another option is to use an alternate set of Cairo bindings, like cairocffi. But that's obviously a big change, which may not be appropriate.

Tox fails because setup.py can't find the requirements.txt

I have added tox to my project and my tox.ini is very simple:
[tox]
envlist = py37
[testenv]
deps =
-r{toxinidir}/requirements_test.txt
commands =
pytest -v
But when I run tox, I get the following error:
ERROR: invocation failed (exit code 1), logfile: /path/to/my_project/.tox/py37/log/py37-2.log
========================================================================================= log start ==========================================================================================
Processing ./.tox/.tmp/package/1/my_project-0+untagged.30.g6909bfa.dirty.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-ywna_4ks/setup.py", line 15, in <module>
with open(requirements_path) as requirements_file:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-ywna_4ks/requirements.txt'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-ywna_4ks/
You are using pip version 10.0.1, however version 19.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
========================================================================================== log end ===========================================================================================
__________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR: py37: InvocationError for command /path/to/my_project/.tox/py37/bin/python -m pip install --exists-action w .tox/.tmp/package/1/my_project-0+untagged.30.g6909bfa.dirty.zip (exited with code 1)
Here is my setup.py:
-*- coding: utf-8 -*-
import os
import sys
from setuptools import setup, find_packages
import versioneer
here = os.path.abspath(os.path.dirname(__file__))
sys.path.insert(0, here)
requirements_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'requirements.txt')
with open(requirements_path) as requirements_file:
requires = requirements_file.readlines()
setup(
name='my_project',
version=versioneer.get_version(),
cmdclass=versioneer.get_cmdclass(),
maintainer='Hamed',
license='BSD',
py_modules=['my_project'],
packages=find_packages(),
package_data={'': ['*.csv', '*.yml', '*.html']},
include_package_data=True,
install_requires=requires,
long_description=open('README.md').read(),
zip_safe=False
]
},
)
python setup.py install works fine.
It seems that tox is looking for requirements in the tmp dir, but can't find it there. Is there something wrong with my configurations?
I am using tox==3.12.1, python==3.7.3, setuptools==41.0.1, and conda==4.6.9
I've tested this on Arch and SLES 12 and got the same result with both.

Based on the point from #phd, I found out that requirements.txt was not present in the source distribution. Adding requirements.txt to the MANIFEST.in solved the issue!

Complementing Hamed2005's answer:
I have my requirements split into different files (base_requirements.txt, dev_requirements.txt, etc), all of them in a requirements directory. In this case, you need to add this directory in the MANIFEST.in as
recursive-include requirements *

How to build scikit-image in travis-ci?

I am trying to build a script that makes use of scikit-image on Travis-ci. It complains that six is not up to date. Just to be sure, I tried to replicate the .travis.yml of the scikit-image project on github, but it did nothing to change the situation. I also tried to update six, but to no avail...
Does someone know how to build scikit-image on travis?
My requirements.txt:
# Requirements list:
scikit-image
My trace:
$ source ~/virtualenv/python3.3/bin/activate
$ python --version
Python 3.3.5
$ pip --version
pip 6.0.7 from /home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages (python 3.3)
5.80s$ pip install -r requirements.txt
You are using pip version 6.0.7, however version 6.0.8 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting scikit-image (from -r requirements.txt (line 2))
Downloading scikit-image-0.10.1.tar.gz (16.7MB)
100% |################################| 16.7MB 20kB/s
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 110, in <module>
check_requirements()
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 105, in check_requirements
% ((package_name, ) + min_version))
ImportError: You need `six` version 1.3 or later.
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 110, in <module>
check_requirements()
File "/tmp/pip-build-wsd_vd/scikit-image/setup.py", line 105, in check_requirements
% ((package_name, ) + min_version))
ImportError: You need `six` version 1.3 or later.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-wsd_vd/scikit-image
The command "pip install -r requirements.txt" failed and exited with 1 during .
Your build has been stopped.

If you get an error that a specific dependency isn't being met, being more specific about that dependency usually helps. I like to put dependencies that I don't directly use in a before_install section.
before_install:
pip install six>=1.3
install:
pip install -r requirements.txt
pip allows you to specify constraints on the versions of packages you install. If you had to use 1.3 exactly you could use ==.