I'm trying to run a python script saved on my local system in node-red which is running as a docker container. I copied the python script into the docker container as the exec node was unable to locate the file using this command -
cat /local/file/path | docker exec -i <running-container-id> sh -c 'cat > /inside/docker/file/path'
But now I'm getting the following error - Traceback (most recent call last):
File "outlier.py", line 2, in
from pandas import read_csv
ModuleNotFoundError: No module named 'pandas'
I had installed pandas on my local but it's not being found by the exec node. Any help is appreciated, thanks.
When applications run inside a Docker container they only have access to the libraries/modules included inside the container. They have no access to anything in the host machine.
So if you want to run Python scripts that have dependencies on Python modules you will need to create a custom Docker container that extends the official Node-RED container and then installs those modules.
Node-RED provides doc about extending it's container here
Related
i am trying to build a container for my express.js application. The express.js-app makes use of python via the npm package PythonShell.
I have plenty of python-code, which is in a subfolder of my express-app and with npm start everything works perfectly.
However, i am new to docker and i need to containerize the app. My Dockerfile looks like this:
FROM node:18
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3001
CMD ["node", "./bin/www"]
I built the Image with:
docker build . -t blahblah-server and ran it with docker run -p 8080:3001 -d blahblah-server.
I make use of imports at the top of the python-script like this:
import datetime
from pathlib import Path # Used for easier handling of auxiliary file's local path
import pyecma376_2 # The base library for Open Packaging Specifications. We will use the OPCCoreProperties class.
from assi import model
When the pythonscript is executed (only in the container!!!) I get following error-message:
/usr/src/app/public/javascripts/service/pythonService.js:12
if (err) throw err;
^
PythonShellError: ModuleNotFoundError: No module named 'pyecma376_2'
at PythonShell.parseError (/usr/src/app/node_modules/python-shell/index.js:295:21)
at terminateIfNeeded (/usr/src/app/node_modules/python-shell/index.js:190:32)
at ChildProcess.<anonymous> (/usr/src/app/node_modules/python-shell/index.js:182:13)
at ChildProcess.emit (node:events:537:28)
at ChildProcess._handle.onexit (node:internal/child_process:291:12)
----- Python Traceback -----
File "/usr/src/app/public/pythonscripts/myPython/wtf.py", line 6, in <module>
import pyecma376_2 # The base library for Open Packaging Specifications. We will use the OPCCoreProperties class. {
traceback: 'Traceback (most recent call last):\n' +
' File "/usr/src/app/public/pythonscripts/myPython/wtf.py", line 6, in <module>\n' +
' import pyecma376_2 # The base library for Open Packaging Specifications. We will use the OPCCoreProperties class.\n' +
"ModuleNotFoundError: No module named 'pyecma376_2'\n",
executable: 'python3',
options: null,
script: 'public/pythonscripts/myPython/wtf.py',
args: null,
exitCode: 1
}
If I comment the first three imports out, I get the same error:
PythonShellError: ModuleNotFoundError: No module named 'assi'
Please notice, that assi actually is from my own python-code, which is included in the expressjs-app-directory
Python seems to be installed in the container correctly. I stepped inside the container via docker exec -it <container id> /bin/bash and there are the python packages in the #/usr/lib-directory.
I really have absolute no idea how all this works together and why python doesn't find this modules...
You are trying to use libs that are not in Standard Python Library. It seems that you are missing to run pip install , when you build the docker images.
Try adding RUN docker commands that can do this for you. Example:
RUN pip3 install pyecma376_2
RUN pip3 install /path/to/assi
Maybe, that can solve your problem. Don't forget to check if python are already installed in your container, it semms that it is. And if you have python2 and pyhton3 installed, make sure that you use pip3 instead of only pip.
I am completely new to docker (on windows 10 machine). I intend to setup a python development environment as a docker container. And most of the reading that I did involved the use of Dockerfile. I want to do it from scratch instead purely using commands.
What I intend to do is very basic requirement: To have python docker image present with me and that I should be able to install more libraries in that image and commit these updates to that image. But I want to do it completely using commands (not via a Dockerfile).
I am using Docker Desktop on windows 10 machine. I did docker pull python:latest and it pulled the image like so:
C:\Users\MyHomeDirectory>docker pull python:latest
latest: Pulling from library/python
d960726af2be: Pull complete
e8d62473a22d: Pull complete
8962bc0fad55: Pull complete
65d943ee54c1: Pull complete
532f6f723709: Pull complete
1334e0fe2851: Pull complete
062ada600c9e: Pull complete
aec2e3a89371: Pull complete
1ec7c3bcb4b2: Pull complete
Digest: sha256:65367d1d3eb47f62127f007ea1f74d1ce11be988044042ab45d74adc6cfceb21
Status: Downloaded newer image for python:latest
docker.io/library/python:latest
Then I did docker images and it showed that python latest image is present with a size of 886 MB.
C:\Users\Tejas.Khajanchee>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
python latest 5b3b4504ff1f 47 hours ago 886MB
I am also able to enter the interactive python by doing docker run -it python and it generates the interactive shell:
Python 3.9.5 (default, May 12 2021, 15:26:36)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import gc
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>>
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'
>>>
>>> import openpyxl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'openpyxl'
>>>
But as evident, some of the libraries are not installed. But this is where I get stuck. How do I install libraries into the python image and have the image updated. Also, if this shell is the only thing that I am allowed to do till now, what does the 886 MB content represent? Also I want to be able to run scripts using this docker image. When I attempt to do this on a very basic hello world script, the following error comes up:
C:\Users\MyHomeDirectory\Downloads>docker run -it python a.py
docker: Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: exec: "a.py": executable file not found in $PATH: unknown.
I want to be able to do this purely with commands and not a Dockerfile. Please help.
First, looks you confuse the concept of image & container.
Docker image: read only, used as basis of container
Docker container: overlay a writeable layer upon the read only layer of docker image, all container will use image as basis
Second, for you, you mentioned you want to install numpy in the image, the best way for this is to customized a Dockerfile like next:
Dockerfile:
FROM python
RUN pip install numpy
Then, build a new image with docker build -t newpython .
BUT, you mentioned you don't want to use Dockerfile, then the replacement is next:
Install numpy in a container:
docker run -it python /bin/bash
# pip install numpy
Use docker ps -a to get the container id, e.g: 0a6b4df8e2c2, then commit this container which already have numpy installed to a new image:
docker commit 0a6b4df8e2c2 newpython
Finally, all new container need to run base on newpython image not python image, as only the newpython image has numpy installed:
docker run --rm newpython python -c "import numpy; print(numpy.__version__)"
1.20.3
Additional, for docker run -it python a.py, I think you misunderstand the concept. Container command like python a.py means the command will executed in container, so the a.py should be in container, not in host machine.
I created a docker container of a Python program and I need to run it on a virtual machine.
I prepared a Dockerfile to create the container, where I launch the main script by using:
CMD python3 /home/project_name_folder/script_00.py
built it as usual:
docker build -t registry_address/image_name -f folder_name/Dockerfile_01 .
and pushed the image to a registry. Then, I pulled and ran the image on the VM by:
docker run -p 5555:8050 registry_address/image_name
The problem is that, on the VM, I'm getting an error which never occurs when I run a container generated from the same image on my PC.
The error looks something like this:
Traceback (most recent call last):
File "/home/project_name_folder/script_02.py", line 188, in <module>
infile = open(sys.argv[1], 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'temp/3004601723038773'
So, script_00.py is executed, creates the file temp/3004601723038773, and calls another script_02 passing the file name as an argument. However, the container on the VM seems to be unable to find the corresponding file.
What surprises me is that this is never happening when running the container on my local computer, so I assume it's not due to the code itself. Can it be related to the docker version, or something similar?
P.S. sorry if the terminology I'm using is not correct, I'm a beginner when it comes to Docker!
I have a project I'm trying to test and run on Jenkins. On my machine it works fine, but when I try to run it in Jenkins, it fails to find a module in the workspace.
In the main workspace directory, I run the command:
python xtests/app_verify_auto.py
And get the error:
+ python /home/tomcat7/.jenkins/jobs/exit103/workspace/xtests/app_verify_auto.py
Traceback (most recent call last):
File "/home/tomcat7/.jenkins/jobs/exit103/workspace/xtests/app_verify_auto.py", line 19, in <module>
import exit103.data.db as db
ImportError: No module named exit103.data.db
Build step 'Execute shell' marked build as failure
Finished: FAILURE
The directory exit103/data exists in the workspace and is a correct path, but python can't seem to find it.
This error exists both with and without virtualenv.
It's may caused by your PATH setting not right in jenkins environment.In fact , the environments for your default user and jenkins-user are not the same.
You may try to find what are the PATH and PYTHONPATH in your jenkins-user environments .
Try to run "shell commands" in jenkins "echo $path" and so on to see what's them are.
In most of time , you need to set the PATH by yourself.
You may reference this answer.
Jenkins: putting my Python module on the PYTHONPATH
Faced the same issue.
For others who are reading this, Run the build in your master node. It fixed the problem for me.
Running the build in the slave node doesn't give proper access to all the python modules and other commands such as jq to the workspace.
I'm trying to run a simple python script that takes a json string as an argument in a docker container. However I get the following error:
Traceback (most recent call last):
File "/root/simple.py", line 2, in <module>
import json
ImportError: No module named json
I'm running the standard ubuntu:12.04 image. Here's how I call up the container:
docker run -v $(pwd)/:/root/ ubuntu:12.04 python /root/simple.py '[{"hi":"bye"}]'
My simple.py script is just:
import sys
import json
configs = json.loads(sys.argv[1])
print configs
def read_option_keys(json_file):
json_file[0]["new"] = None
print json.dumps(json_file)
read_option_keys(configs)
Any idea why it's not returning the following as expected:
[{u'hi': u'bye'}]
[{"hi": "bye", "new": null}]
I was able to solve the issue myself. Ubuntu image is super bare-bones. I pulled the dockerfile/python image and now it works.
Docker is a containerized instance, so your python script has no knowledge of the outside world. You are currently running the Ubuntu image, but it doesn't contain the python standard module. You should run a python image instead or add python to your Ubuntu image