We Keep Coding

Unable to setup "sphinx" on gitlab-ci

Unable to setup sphinx on gitlab-ci
I'm trying to setup and run sphnix on gitlab-ci. It's installed on my local machine and server works fine. But not at gitlab-ci
Here's a part of my gitlab-ci:
before_script:
- apt-get update -qq && apt-get install -yqq [......] python-pip
- pip install sphinx
# ..........
- bundle exec rake ts:index
- bundle exec rake ts:start
The output of installing sphinx:
$ pip install sphinx
Downloading/unpacking sphinx
[................]
Installing collected packages: sphinx, babel, [.............]
Compiling /tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncfilters.py ...
File "/tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncfilters.py", line 7
async def auto_to_seq(value):
^
SyntaxError: invalid syntax
Compiling /tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncsupport.py ...
File "/tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncsupport.py", line 22
async def concat_async(async_gen):
^
SyntaxError: invalid syntax
Running setup.py install for MarkupSafe
building 'markupsafe._speedups' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c markupsafe/_speedups.c -o build/temp.linux-x86_64-2.7/markupsafe/_speedups.o
markupsafe/_speedups.c:12:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
==========================================================================
WARNING: The C extension could not be compiled, speedups are not enabled.
Failure information, if any, is above.
Retrying the build without the C extension now.
==========================================================================
WARNING: The C extension could not be compiled, speedups are not enabled.
Plain-Python installation succeeded.
==========================================================================
Successfully installed sphinx babel sphinxcontrib-websupport Jinja2 alabaster imagesize Pygments snowballstemmer docutils packaging typing pytz MarkupSafe pyparsing
Cleaning up...
As you can see, it's successeded.
However, when it gets to this point, it fails:
$ bundle exec rake ts:index
Generating configuration to /builds/my_user/my_ap123/config/test.sphinx.conf
sh: 1: indexer: not found
The Sphinx indexing command failed:
Command: indexer --config "/builds/my_user/my_ap123/config/test.sphinx.conf" --all
Status: 127
Output: See above
Why is that? How to fix it?

pip install sphinx did not fully succeed. during installation, it is complaining about missing Python.h
fatal error: Python.h: No such file or directory
You need to install a development package of python as well.
For python3x use
sudo apt-get install python3-dev
For python2x use
sudo apt-get install python-dev
You can add it to before_script
before_script:
- apt-get update -qq && apt-get install -yqq [......] python-pip python3-dev
- pip install sphinx
Update:
To install sphinx search engine using gitlab ci update your before_script as follow
before_script:
- add-apt-repository ppa:builds/sphinxsearch-rel22 sphinxsearch
- apt-get update -qq && apt-get install -yqq [......] python-pip python3-dev mysql-client unixodbc libpq5
- pip install sphinxsearch

Error Installing Kivy On Ubuntu

kivy/graphics/svg.pyx:469:9: 'reload' already defined
building 'kivy.graphics.svg' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Ikivy/include -Ikivy/include -I/usr/include/python2.7 -c kivy/graphics/svg.c -o build/temp.linux- x86_64-2.7/kivy/graphics/svg.o
kivy/graphics/svg.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
I am trying to install kivy on ubuntu 16.04 . When I execute " python setup.py build_ext --inplace -f " this instruction shows above error please help me...

It is likely that you cython version is incorrect.
For the current version of kivy (1.10.0), I've had success with cython==0.26. Install failed with 0.27.1 and 0.27.3 with the same error as yours.
So in your relevant anaconda environment, uninstall the current cython:
pip uninstall cython==0.2x.x
(give the correct version of your installed cython)
or
conda uninstall cython
Install cython==0.26:
pip install cython==0.26
(there doesn't seem to be a 0.26 version of cython in the linux64 anaconda channel)
Then: pip install kivy

IMHO much easier is to use apt-get or pip.

pip install Cython #pip or pip3
pip install Kivy
EDIT:
Of course install gcc

What's your cython version? you might want to update it to 0.24 or 0.25 to build kivy.
pip install -U cython==0.25

I am using Kivy on Ubuntu 16.04 LTS and used the following installation steps. Kivy examples is optional:
$ sudo add-apt-repository ppa:kivy-team/kivy
$ sudo apt-get update
$ sudo apt-get install python3-kivy
$ sudo apt-get install python-kivy-examples
$ sudo pip install --upgrade cython==0.26

For kivy version 1.10.0, cython must be 0.26

Problems installing snappy on python on Alpine Linux

When i am trying to install Snappy on alpine linux using:
pip install snappy
I am getting the following error when it tries to install a required package called cypari.
I installed snappy from "apk add snappy"
gcc -fno-strict-aliasing -Os -fomit-frame-pointer -g -DNDEBUG -Os -fomit-frame-pointer -g -fPIC -Ilibcache/pari64/include -I/usr/include/python2.7 -c cypari_src/_pari.c -o build/temp.linux-x86_64-2.7/cypari_src/_pari.o
In file included from cypari_src/_pari.c:460:0:
cypari_src/implementation.c:47:22: fatal error: execinfo.h: No such file or directory
#include <execinfo.h>

I solved this problem by running apk add libexecinfo libexecinfo-dev
Then using apk add snappy
Then using pip install python-snappy
if you use pip install snappy it installs a completely different library which is this

Combining previous answers, this minimal Dockerfile works for me
FROM python:3.6-alpine
RUN apk add --no-cache g++ snappy-dev && \
pip install --no-cache-dir --ignore-installed python-snappy

Installing execinfo-dev suffice, since execinfo.h is in that package.
You can check it here:
http://pkgs.alpinelinux.org/contents?file=execinfo.h&path=&name=&branch=&repo=&arch=
apk add snappy
will install the package, so you don't need to:
pip install snappy

To expand on the answer from #dwardu, I also had to install the snappy-dev package. That fixed this error I was getting from pip install:
snappy-c.h: No such file or directory

Error Installing psycopg2 on Amazon Linux

When installing psycopg2 on Amazon Linux using this command: sudo pip-3.4 install psycopg2 I get this output:
gcc -pthread -Wno-unused-result -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.7 (dt dec pq3 ext)" -DPG_VERSION_NUM=90218 -I/usr/include/python3.4m -I. -I/usr/include/pgsql92 -I/usr/include/pgsql92/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.4/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from ./psycopg/replication_cursor.h:30:0,
from psycopg/psycopgmodule.c:32:
./psycopg/libpq_support.h:31:32: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘XLogRecPtr’
typedef unsigned PG_INT64_TYPE XLogRecPtr;
^
In file included from psycopg/psycopgmodule.c:32:0:
./psycopg/replication_cursor.h:47:5: error: unknown type name ‘XLogRecPtr’
XLogRecPtr write_lsn; /* LSNs for replication feedback messages */
^
./psycopg/replication_cursor.h:48:5: error: unknown type name ‘XLogRecPtr’
XLogRecPtr flush_lsn;
^
./psycopg/replication_cursor.h:49:5: error: unknown type name ‘XLogRecPtr’
XLogRecPtr apply_lsn;
^
In file included from psycopg/psycopgmodule.c:33:0:
./psycopg/replication_message.h:46:5: error: unknown type name ‘XLogRecPtr’
XLogRecPtr data_start;
^
./psycopg/replication_message.h:47:5: error: unknown type name ‘XLogRecPtr’
XLogRecPtr wal_end;
^
error: command 'gcc' failed with exit status 1
I already have gcc, postgresql-devel, postgresql-libs, and python34-devel installed. Am I missing a package or setting?
Edit: This is on an EMR node.

I have the same problem with EMR, I tried below
sudo yum -y install gcc python-setuptools python-devel postgresql-devel
sudo /usr/bin/pip install --upgrade pip
sudo /usr/local/bin/pip install psycopg2
It works when I'm running it on EMR but doesn't work when EMR is spin up from Pipeline.

If you are trying to install psycopg2 for python3 you must install python3-devel as well.
To find the correct package
yum search python3 | grep devel
In my case it was python36-devel so I did
sudo yum install python36-devel
And then I could install psycopg2 for my python3 virtualenvironment on the machine

If you're looking for psycopg2 for python3 in amazon linux, this is what worked for me:
sudo yum install -y gcc postgresql-devel
sudo curl https://bootstrap.pypa.io/ez_setup.py -o - | sudo python36
sudo pip-3.6 install -U psycopg2

How do I Install psycopg2 in Centos 7? [duplicate]

I need to install psycopg2 for openerp installation process, but I always faced that error. I've already used pip and easy_install and my gcc is the latest version. I really need your help to solve my problem.
This is my complete error with easy_install:
[root#server01 ~]# easy_install psycopg2
Searching for psycopg2
Reading http://pypi.python.org/simple/psycopg2/
Reading http://initd.org/psycopg/
Reading http://initd.org/projects/psycopg2
Best match: psycopg2 2.4.5
Downloading http://initd.org/psycopg/tarballs/PSYCOPG-2-4/psycopg2-2.4.5.tar.gz
Processing psycopg2-2.4.5.tar.gz
Running psycopg2-2.4.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-anWVvJ/psycopg2-2.4.5/egg-dist-tmp-cZbdtn
no previously-included directories found matching 'doc/src/_build' In file included from psycopg/psycopgmodule.c:27:
./psycopg/psycopg.h:31:22: error: libpq-fe.h: No such file or directory In file included from psycopg/psycopgmodule.c:29:
...
error: Setup script exited with error: command 'gcc' failed with exit status 1

You'll need to install the development package for PostgreSQL, which for instance under Ubuntu is a sudo apt-get install libpq-dev... and for CentOS it's yum install postgresql-devel

For the case of CentOS, I had this very same problem, that I solved installing these packages:
sudo yum install postgresql-libs
sudo yum install postgresql-devel
sudo yum install python-devel
only then, I was able to perform successfully:
sudo easy_install psycopg2

May Your system is missing the Python headers. To fix this, run:
sudo apt-get install python-dev
or
sudo apt-get install python3-dev

On a fresh ubuntu 14.04.2 LTS on an EC2 instance, I was able to install psycopg2 after instaling the following packages:
sudo apt-get install gcc
sudo apt-get install postgres-xc-server-dev
sudo apt-get install libpq-dev
sudo apt-get install postgresql-client-common
sudo apt-get install postgresql-common
And on a fresh AMI linux (CentOS) instance:
sudo yum install postgresql-devel
sudo yum install gcc*
Hope it helps,
Erez

i had this problem but was solved by running:
pip install -Iv psycopg2==2.7.7
There is a problem installing the latest version + python3.7

I've been trying to install pgsql on a plain Amazon Linux EC2 instance, and have been getting this error. Per Nicolás answer, I installed his suggested list of packages, but I continued to get the error. Looking more closely at the error, I noticed that the gcc command was including pgsql93 files:
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.6.1 (dt dec pq3 ext lo64)" -DPG_VERSION_HEX=0x09030A -DHAVE_LO64=1 -I/usr/include/python2.7 -I. -I/usr/include/pgsql93 -I/usr/include/pgsql93/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
Looking in /usr/include/ I see pgsql92/ but not pgsql93/. So I took a look at what pgsql-related yum packages were installed and I noticed these:
postgresql92.x86_64
postgresql92-devel.x86_64
postgresql92-libs.x86_64
postgresql93.x86_64
postgresql93-libs.x86_64
Seemed to me like there ought to be a postgresql93-devel, so I tried:
sudo yum install postgresql93-devel
And that fixed the problem for me! So I guess the theme is: Look at the gcc call, and see what dir it's pulling in. Follow the crumbs from there.

For Ubuntu 14.04, from Docker image python:3.4.3-slim this combination worked for me:
sudo apt-get update
sudo apt-get install -y build-essential
sudo apt-get install -y python3.4-dev
sudo apt-get install -y libpq-dev
pip3 install psycopg2
Note build-essential package. It was crucial in my case.

In centos 7. i'd use sudo yum install postgresql96-devel.x86_64 to solved my problem.

The invocation of gcc failed, gcc couldn't compile the source. Probably the cause is
./psycopg/psycopg.h:31:22: error: libpq-fe.h: No such file or directory In file included from psycopg/psycopgmodule.c:29:
that the header libpq-fe.h is nowhere in the search path if you have it at all.

This worked for me on Ubuntu Server 14.10 64-bit:
sudo apt-get install libpqxx-dev

For me, the missing package was
libpqxx-devel.x86_64

psycopg2 is also available on PyPI in the form of wheel packages for the most common platform (Linux, OSX, Windows): this should make you able to install a binary version of the module, not requiring the above build or runtime prerequisites.
You can use this command : pip install psycopg2-binary
Note The psycopg2-binary package is meant for beginners to start playing with Python and PostgreSQL without the need to meet the build requirements.
If you are the maintainer of a publish package depending on psycopg2 you shouldn’t use ‘psycopg2-binary’ as a module dependency. For production use you are advised to use the source distribution.

I was running 3.9-alpine for CICD and had to install these packages to get it working;
apk add postgresql-dev
apk add gcc
apk add musl-dev
Then I was able to run a pip install psycopg2 as normal.