We Keep Coding

Run a python script in node-red running on Docker

I'm trying to run a python script saved on my local system in node-red which is running as a docker container. I copied the python script into the docker container as the exec node was unable to locate the file using this command -
cat /local/file/path | docker exec -i <running-container-id> sh -c 'cat > /inside/docker/file/path'
But now I'm getting the following error - Traceback (most recent call last):
File "outlier.py", line 2, in
from pandas import read_csv
ModuleNotFoundError: No module named 'pandas'
I had installed pandas on my local but it's not being found by the exec node. Any help is appreciated, thanks.

When applications run inside a Docker container they only have access to the libraries/modules included inside the container. They have no access to anything in the host machine.
So if you want to run Python scripts that have dependencies on Python modules you will need to create a custom Docker container that extends the official Node-RED container and then installs those modules.
Node-RED provides doc about extending it's container here

Getting an output from Python that gets executed by PHP

So my issue is that I have an AWS EC2 instance running Ubuntu server 18.04 and Apache2, when a button is pressed it posts some variables which are passed to a function which then uses shell_exec() to execute a Python script. The Python script then takes these variables (direction and angle) as command line arguments, it will then attempt to write then to a file called cmds.txt. When doing this however I get this error reported back to me:
Traceback (most recent call last):
File "/home/server/cmdWriter.py", line 45, in
main()
File "/home/server/cmdWriter.py", line 41, in main
servoCmds(direction, angle)
File "/home/server/cmdWriter.py", line 28, in servoCmds
cmdFile = open("cmds.txt", "a")
PermissionError: [Errno 13] Permission denied: 'cmds.txt'
After some looking I think this is because PHP executes as user "www-data" which doesn't have write privileges, so after looking at other questions I tried settung up permissions so that www-data has read and write privilages to the folder and the python file. For whatever reason this does not work! I've been pulling my hair out trying to get this to work, trying suggestions from several other questions, can anyone help me here?

You are getting the error with respect to 'opening' the file. You have to make sure that the permissions on the .txt file are at 755 or higher. If the server you are using is Linux, then you have two alternatives to modify the permissions:
If you have cPanel, you can use the cPanel interface and just change permissions by going to the file, click on Change Permissions and then set the permissions to 777.
or
From the linux command line, use chmod command - like this:
chmod 777 cmds.txt
I can't speak to the security issues associated with same without knowing more.
The other possibility is that you are using the 'a' mode, but the file does not already exist. If you are not certain the file exists when the command is executed, you may try 'a+' as that will create the file if one does not already exist.
Does that work?

I cannot find output files even though the are saved in the container apparently

I can't find output files of arpeggio using Windows Power Shell (the algorithm was created for Linux)
I've tries to add > to create an output text file , but this only creates an output file of this:
INFO//08:56:42.365//Program begin.
INFO//08:56:42.402//Loaded PDB structure (BioPython)
INFO//08:56:42.403//Detected that the input structure contains hydrogens. Hydrogen addition will be skipped.
INFO//08:56:42.630//Loaded PDB structure (OpenBabel)
INFO//08:56:42.640//Mapped OB to BioPython atoms and vice-versa.
INFO//08:56:42.780//Typed atoms.
INFO//08:56:42.789//Determined atom explicit and implicit valences, bond orders, atomic numbers, formal charge and number of bound hydrogens.
INFO//08:56:42.826//Initialised SIFts.
WARNING//08:56:42.830//Chain termini could not be determined for chain E. It may not be a polypeptide chain.
INFO//08:56:42.831//Determined polypeptide residues, chain breaks, termini
INFO//08:56:43.009//Percieved and stored rings.
INFO//08:56:43.034//Perceived and stored amide groups.
INFO//08:56:43.039//Added hydrogens to BioPython atoms.
INFO//08:56:43.045//Added VdW radii.
INFO//08:56:43.050//Added covalent radii.
INFO//08:56:43.053//Completed NeighborSearch.
INFO//08:56:43.058//Assigned rings to residues.
INFO//08:56:43.061//Made selection.
INFO//08:56:43.173//Expanded to binding site.
INFO//08:56:43.174//Flagged selection rings.
INFO//08:56:43.175//Completed new NeighbourSearch.
INFO//08:56:43.245//Calculated pairwise contacts.
INFO//08:56:43.386//Program End. Maximum memory usage was 83.82 MB.
As you can see, the file is created and saved somewhere but when I ran cat I couldn't find any of the files suggested as output files by the developer.
docker run --rm -v C:\directory:/<path to file> -it arpeggio python arpeggio.py /file -s RESNAME:xxx -v
After this, I should be able to run python show_contacts.py file.pdb -s to show the interactions on PyMOL
PS Z:\> docker run arpeggio python show_contacts.py fie.pdb -s
But, this is the outcome
docker : Traceback (most recent call last):
At line:1 char:1
+ docker run arpeggio python show_contacts.py file.pdb -s
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Traceback (most recent call last)::String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
File "show_contacts.py", line 509, in <module>
with open(contacts_filename, 'rb') as fo:
IOError: [Errno 2] No such file or directory: '/arpeggio/file.contacts'
So, no output file was created
Any idea why?

I was able to solve this issue by running this code in PowerShell:
docker run --rm -v :/<host-directory>:/<container-path> -u uid:gid -it arpeggio python arpeggio.py /container-path/file.pdb -s RESNAME:xxx -v
I was able to get my uid and gid after running ubuntu 18.04 and pulling docker ce on Windows as a subsystem and I have to clarify that the same code is useful to use in docker if you add -H localhost:2375 and select "Expose daemon" after doing a right click on Docker and click on Settings.
Hopefully, this might be useful for anyone having the same issue.

Python module import failure in Jenkins

I have a project I'm trying to test and run on Jenkins. On my machine it works fine, but when I try to run it in Jenkins, it fails to find a module in the workspace.
In the main workspace directory, I run the command:
python xtests/app_verify_auto.py
And get the error:
+ python /home/tomcat7/.jenkins/jobs/exit103/workspace/xtests/app_verify_auto.py
Traceback (most recent call last):
File "/home/tomcat7/.jenkins/jobs/exit103/workspace/xtests/app_verify_auto.py", line 19, in <module>
import exit103.data.db as db
ImportError: No module named exit103.data.db
Build step 'Execute shell' marked build as failure
Finished: FAILURE
The directory exit103/data exists in the workspace and is a correct path, but python can't seem to find it.
This error exists both with and without virtualenv.

It's may caused by your PATH setting not right in jenkins environment.In fact , the environments for your default user and jenkins-user are not the same.
You may try to find what are the PATH and PYTHONPATH in your jenkins-user environments .
Try to run "shell commands" in jenkins "echo $path" and so on to see what's them are.
In most of time , you need to set the PATH by yourself.
You may reference this answer.
Jenkins: putting my Python module on the PYTHONPATH

Faced the same issue.
For others who are reading this, Run the build in your master node. It fixed the problem for me.
Running the build in the slave node doesn't give proper access to all the python modules and other commands such as jq to the workspace.

Auto Starting a python script on boot (RPi)

I have a python script on my RPi that needs to run on boot
I added it to rc.local, and it used to work fine
A few days ago, I added a functionality to the program, and it now uses open() to read a txt file
Now every time I restart the Pi, python gives me and error:
File "home/pi/client.py", line 13, in <module>
stats=open('stats.txt')
IOError: [Errno 2] No such file or directory: 'stats.txt'
When I manually launch the script with:
sudo python client.py
it works fine with no problems.
Any suggestions?

Your rc.local probably does not start your script in the correct directory. So you should either:
use something like cd my/dir && python /path/to/home/pi/client.py
call os.chdir("/path/to/some_dir") in your script
use an absolute path when opening the file: stats = open('/path/to/stats.txt')