We Keep Coding

Ansible error: "The Python 2 bindings for rpm are needed for this module"

Im trying to pip install a requirements file in my python3 environment using the following task
pip:
python3: yes
requirements: ./requirements/my_requirements.txt
extra_args: -i http://mypypi/windows/simple
I checked which version ansible is running on the controller node (RH7) and it's 3.6.8
ansible-playbook 2.9.9
config file = None
configured module search path = ['/home/{hidden}/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
executable location = /usr/local/bin/ansible-playbook
python version = 3.6.8 (default, Jun 11 2019, 15:15:01) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
No config file found; using defaults
I am however getting the following error:
fatal: [default]: FAILED! => {"changed": false, "msg": "The Python 2 bindings for rpm are needed for this module. If you require Python 3 support use the `dnf` Ansible module
instead.. The Python 2 yum module is needed for this module. If you require Python 3 support use the `dnf` Ansible module instead."}
My controller node is running RH7.
The targets are centos7 (provisioned by vagrantfiles)
Does anyonek now how to solve this?

I had a similar problem with the "Amazon Linux 2" distribution that uses yum, but does not support dnf as of this writing.
As mentioned in the comments above, my problem was in the ansible-managed nodes (AWS EC2 instances running Amazon Linux 2) and not in the controller.
Solved it by imposing the use of python2, adding ansible_python_interpreter=/usr/bin/python2 for this group of hosts in the ansible inventory file, as in the following snippet:
[amz_linux]
server2 ansible_host=ec2-xx-yy-zz-pp.eu-west-1.compute.amazonaws.com
[amz_linux:vars]
ansible_user=ec2-user
ansible_python_interpreter=/usr/bin/python2
Tried it with this playbook, adapted from a Redhat quick guide.
---
- hosts: amz_linux
become: yes
tasks:
- name: install Apache server
yum:
name: httpd
state: latest
- name: enable and start Apache server
service:
name: httpd
enabled: yes
state: started
- name: create web admin group
group:
name: web
state: present
- name: create web admin user
user:
name: webadm
comment: "Web Admin"
groups: web
append: yes
- name: set content directory group/permissions
file:
path: /var/www/html
owner: root
group: web
state: directory
mode: u=rwx,g=rwx,o=rx,g+s
- name: create default page content
copy:
content: "Welcome to {{ ansible_fqdn}} on {{ ansible_default_ipv4.address }}"
dest: /var/www/html/index.html
owner: webadm
group: web
mode: u=rw,g=rw,o=r
Actual ansible-playbook run (after using ssh-add to add the instance private key to the ssh agent.)
$ ansible-playbook -i ansible/hosts ansible/apache_amz_linux.yaml
PLAY [amz_linux] **********************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************
The authenticity of host 'ec2-xxxxxxxxxxx.eu-west-1.compute.amazonaws.com (xxxxxxxxxxx)' can't be established.
ECDSA key fingerprint is SHA256:klksjdflskdjflskdfjsldkfjslkdjflskdjf/sdkfj.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
ok: [server2]
TASK [install Apache server] **********************************************************************************************
changed: [server2]
TASK [enable and start Apache server] *************************************************************************************
changed: [server2]
TASK [create web admin group] *********************************************************************************************
changed: [server2]
TASK [create web admin user] **********************************************************************************************
changed: [server2]
TASK [set content directory group/permissions] ****************************************************************************
changed: [server2]
TASK [create default page content] ****************************************************************************************
changed: [server2]
PLAY RECAP ****************************************************************************************************************
server2 : ok=7 changed=6 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Modify
ansible-playbook
to
ansible-playbook -e ansible_python_interpreter=/usr/bin/python2

This error message can show if your ansible.cfg file's inventory entry is set to a non-existing file.

uWSGI will not run in mixed Python environment in order to operate correctly with nginx and run Django app

Environment: Ubuntu 16.04 (with system Python at 2.7.12) running in Vagrant/Virtualbox on Windows 10 host
Python Setup: System python verified by doing python -V with no virtualenv's activated. Python 3.5 is also installed, and I've done pipenv --three to create the virtualenv for this project. Doing python -V within the activated virtualenv (pipenv shell to activate) shows Python 3.5.2.
Additional Background: I'm developing a Wagtail 2 app. Wagtail 2 requires Django 2 which, of course, requires Python 3. I have other Django apps on this machine that were developed in Django 1.11/Python 2.7 and are served by Apache. We are moving to Django 2/Python 3 for development going forward and are moving to nginx/uWSGI for serving the apps.
I have gone through many tutorials/many iterations. All Vagrant port mapping is set up fine with nginx serving media/static files and passing requests upstream to the Django app on a unix socket, but this is giving a 502 Gateway not found error because uWSGI will not run correctly. Therefore, right now I'm simply running the following from the command line to try to get uWSGI to run: uwsgi --ini /etc/uwsgi/sites/my_site.com.ini. This file contains:
[uwsgi]
uid = www-data
gid = www-data
plugin = python35
# Django-related settings
# the base directory (full path)
chdir=/var/sites/my_site
# Django's wsgi file
wsgi-file = my_site.wsgi
# the virtualenv (full path)
virtualenv=/root/.local/share/virtualenvs/my_site-gmmiTMID
# process-related settings
# master
master = True
# maximum number of worker processes
processes = 10
# the socket (use the full path to be safe)
socket = /tmp/my_site.sock
# clear environment on exit
vacuum = True
I've tried installing uWSGI in the following ways:
system-wide with pip install uwsgi as well as pip3 install uwsgi
using apt-get install uwsgi uwsgi-plugin-python3
I've ensured that only one install is in place at a time by uninstalling any previous uwsgi installs. The latter install method places uwsgi-core in usr/bin and also places in usr/bin shortcuts to uwsgi, uwsgi_python3, and uwsgi_python35.
In the .ini file I've also tried plugin = python3. I've also tried from the command line:
uwsgi_python3 --ini /etc/uwsgi/sites/my_site.com.ini
uwsgi_python35 --ini /etc/uwsgi/sites/my_site.com.ini
I've tried executing the uwsgi ... .ini commands from both within the activated virtual environment and with the virtualenv deactivated. Each of the three command line uwsgi ... .ini executions (uwsgi ..., uwsgi_python3 ... and uwsgi_python35 ...) DO cause the .ini file to be executed, but each time I'm getting the following error (the last two lines of the following statements):
[uwsgi] implicit plugin requested python35
[uWSGI] getting INI configuration from /etc/uwsgi/sites/my_site.com.ini
*** Starting uWSGI 2.0.12-debian (64bit) on [Wed Mar 7 03:54:44 2018] ***
compiled with version: 5.4.0 20160609 on 31 August 2017 21:02:04
os: Linux-4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018
nodename: vagrant-ubuntu-trusty-64
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /vagrant/my_site
detected binary path: /usr/bin/uwsgi-core
setgid() to 33
setuid() to 33
chdir() to /var/sites/my_site
your processes number limit is 7743
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/my_site.sock fd 3
Python version: 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609]
Set PythonHome to /root/.local/share/virtualenvs/my_site-gmmiTMID
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
If I go into the Python command line within the activated virtualenv and do import encodings, it imports fine (no message - just comes back to command line). A search for this particular error has turned up nothing of use. Any idea why the ImportError: No module named 'encodings' is coming up?
UPDATE - PROBLEM STILL OCCURRING
I'm using pipenv, and it stores individual virtualenvs in the /home/username/.local/share/virtualenvs folder. Though I was able to start uWSGI from the command line by executing the uWSGI config file as the vagrant user (see comment below), I have still not been able start the service with /home/vagrant/.local/share/virtualenvs/my_venv in the uWSGI config file. I tried adding the vagrant user to the www-data group and adding the www-data user to the vagrant group. I also put both read and execute permission for the world on the whole path (including the individual venv), but the uWSGI service will still not start.
A HACKISH WORKAROUND
I did finally get the uWSGI service to start by copying the venv to /opt/virtualenvs/my_venv. I was then able to start the service with sudo service uwsgi start. Ownership of that whole path is root:root.
STILL LOOKING FOR A SOLUTION...
This solution is not optimal since I am now executing from a virtualenv that will have to be updated when the default virtualenv is updated since this location is not the default for pipenv, so I'm still looking for answers. Perhaps it is a Ubuntu permissions error, but I just can't find the problem.

It might be the problem with your virtual environment. Try the following
rm -rf /root/.local/share/virtualenvs/my_site-gmmiTMID
virtualenv -p python3 /root/.local/share/virtualenvs/my_site-gmmiTMID
source /root/.local/share/virtualenvs/my_site-gmmiTMID/bin/activate
pip install -r requirements.txt
and in uwsgi conf try to change
virtualenv=/root/.local/share/virtualenvs/my_site-gmmiTMID
to
home=/root/.local/share/virtualenvs/my_site-gmmiTMID
Reference:
http://uwsgi-docs.readthedocs.io/en/latest/Options.html#virtualenv

Running Windows Server Core in Docker Container

my Linux containers run like a charm, but the change to Windows Server in my Docker container makes me crazy!
My Docker file doesn't build although it is as simple as my linux Dockerfiles:
FROM microsoft/windowsservercore
#Install Chocolately
RUN #powershell -NoProfile -ExecutionPolicy unrestricted -Command "(iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex)"
ENV PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin
#Install python
RUN choco install -fy python2
RUN refreshenv
ENV PYTHONIOINPUT=UTF-8
RUN pip install -y scipy
Some times I was able to Chocolately which results in a fail to install scipy via PIP or curiously starting 5 minutes ago, even the installation of chocolately fails:
iwr : The remote name could not be resolved: 'chocolatey.org'
At line:1 char:2
+ (iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:Htt
pWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe
ll.Commands.InvokeWebRequestCommand
Here are some specs on my Docker for Windows Installation:
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 3
Server Version: 1.13.0
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: l2bridge l2tunnel nat null overlay transparent
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 14393 (14393.693.amd64fre.rs1_release.1612
Operating System: Windows 10 Education
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 7.903 GiB
Name: xxxx
ID: deleted
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: -1
Goroutines: 18
System Time: 2017-01-31T16:14:36.3753129+01:00
EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Any ideas?

I was unable to get refreshenv to work, so I used multiple powershell sessions, I've included in case it is useful to someone in the future.
#Install Chocolately, Python and Python Package Manager, each PowerShell session will reload the PATH from previous step
RUN #powershell -NoProfile -ExecutionPolicy unrestricted -Command "iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex"
RUN #powershell -NoProfile -ExecutionPolicy unrestricted -Command "choco install -y python3"

modprobe: FATAL: Module nvidia-uvm not found in directory /lib/modules/

I had an issue recently after successfully installing and testing Tensorflow compiled with GPU support.
After rebooting the machine, I got the following error Message when I tried to run a Tensorflow program:
...
('Extracting', 'MNIST_data/t10k-labels-idx1-ubyte.gz')
modprobe: FATAL: Module nvidia-uvm not found in directory /lib/modules/4.4.0-34-generic
E tensorflow/stream_executor/cuda/cuda_driver.cc:491] failed call to cuInit: CUDA_ERROR_UNKNOWN
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:140] kernel driver does not appear to be running on this host (caffe-desktop): /proc/driver/nvidia/version does not exist
I tensorflow/core/common_runtime/gpu/gpu_init.cc:92] No GPU devices available on machine.
(0, 114710.45)
(1, 95368.891)
...
(98, 56776.922)
(99, 57289.672)
Screencapture of error
Code: https://github.com/llSourcell/autoencoder_demo
Question: Why would restarting a Ubuntu 16.04 machine break Tensorflow?

I actually solved my own problem and wanted to share the solution which worked for me.
The magic Google search was:
"modprobe: FATAL: Module nvidia-uvm not found in directory /lib/modules/"
Which led me to the following answer on askubuntu:
https://askubuntu.com/a/496146
That answer's author, Sneetsher, did a really good job of explaining so if the link doesn't 404 I would start there.
Cliff Notes
Diagnosis: I suspected that Ubuntu may have installed a kernel update when I rebooted.
Solution: Reinstalling the NVIDIA driver fixed the error.
Problem: NVIDIA drivers cannot be installed with X server running
Two different ways to fix the NVIDIA Driver
1) Keyboard and Monitor:
Paraphrasing the askubuntu answer:
1) Switch to text-only console (Ctrl+Alt+F1 or any to F6).
2) Build driver modules for the current kernel (which just installed) sudo ./<DRIVER>.run -K
credit "Sneetsher" : https://askubuntu.com/a/496146
I don't have a keyboard or monitor attached to this PC so here's the "headless" approach I actually used:
2) Over SSH:
Following this guide to reboot to console:
http://ubuntuhandbook.org/index.php/2014/01/boot-into-text-console-ubuntu-linux-14-04/
$ sudo cp -n /etc/default/grub /etc/default/grub.orig
$ sudo nano /etc/default/grub
$ sudo update-grub
edit the grub file according to above link(3 changes):
Comment the line GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash”, by adding # at the beginning, which will disable the Ubuntu purple screen.
Change GRUB_CMDLINE_LINUX=”" to GRUB_CMDLINE_LINUX=”text”, this makes Ubuntu boot directly into Text Mode.
Uncomment this line #GRUB_TERMINAL=console, by removing the # at the beginning, this makes Grub Menu into real black & white Text Mode (without background image)
UPDATE: (If running Ubuntu 16.04 If
$ sudo systemctl set-default multi-user.target
Reboot into console
$ sudo shutdown -r now
$ sudo service lightdm stop
$ sudo ./<DRIVER>.run
follow the NVIDIA driver installer
$ sudo mv /etc/default/grub /etc/default/grub.textonly
$ sudo mv /etc/default/grub.orig /etc/default/grub
$ sudo update-grub
$ sudo shutdown -r now
Results (What things look like now the GPU was successfully detected)
...
('Extracting', 'MNIST_data/t10k-labels-idx1-ubyte.gz')
I tensorflow/core/common_runtime/gpu/gpu_init.cc:118] Found device 0 with properties:
name: GeForce GTX 970
major: 5 minor: 2 memoryClockRate (GHz) 1.342
pciBusID 0000:01:00.0
Total memory: 3.94GiB
Free memory: 3.88GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:138] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:148] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 970, pci bus id: 0000:01:00.0)
(0, 113040.92)
(1, 94895.867)
...
Screencapture of the same

One simple solution to "Problem: NVIDIA drivers cannot be installed with X server running:
Access ubuntu from another computer using SSH
Remove the screen (display device) of Ubuntu computer
Reboot computer using sudo reboot, then access it again

port upgrade outdated fails in Yosemite

I am trying to upgrade my MacPorts. I ran sudo port -d selfupdate successfully after updating my xcode command line, but the problem is persisting. How can I successfully upgrade my outdated installations? I am running Mac Yosemite.
$ sudo port upgrade outdated
---> Fetching distfiles for python25
Error: python25 is not supported on Yosemite or later.
Error: org.macports.fetch for port python25 returned: unsupported platform
Please see the log file for port python25 for details:
/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python25/python25/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
http://guide.macports.org/#project.tickets
logfile:
$ cat /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python25/python25/main.log
version:1
:debug:main Executing org.macports.main (python25)
:debug:main changing euid/egid - current euid: 0 - current egid: 0
:debug:main egid changed to: 501
:debug:main euid changed to: 502
:debug:fetch fetch phase started at Fri Jan 9 22:13:21 PST 2015
:notice:fetch ---> Fetching distfiles for python25
:debug:fetch Executing proc-pre-org.macports.fetch-fetch-0
:error:fetch python25 is not supported on Yosemite or later.
:error:fetch org.macports.fetch for port python25 returned: unsupported platform
:debug:fetch Error code: NONE
:debug:fetch Backtrace: unsupported platform
while executing
"proc-pre-org.macports.fetch-fetch-0 org.macports.fetch"
("eval" body line 1)
invoked from within
"eval $pre $targetname"
:info:fetch Warning: targets not executed for python25: org.macports.destroot org.macports.fetch org.macports.checksum org.macports.extract org.macports.patch org.macports.configure org.macports.build
:notice:fetch Please see the log file for port python25 for details:
/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python25/python25/main.log