We Keep Coding

bash shell commands run through python to be universal with windows, mac, and linux

I need to have bash shell commands run through python in order to be universal with pc and mac/linux. ./bin/production doesn't work in powershell and putting 'bash' in front would give an error that it doesn't recognize 'docker' command
./bin/production contents:
#!/bin/bash
docker run --rm -it \
--volume ${PWD}/prime:/app \
$(docker build -q docker/prime) \
npm run build
This is the python script:
import subprocess
from python_on_whales import docker
cmd = docker.run('docker run --rm -it --volume ${PWD}/prime:/app $(docker build -q docker/prime) npm run build')
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
out, err = p.communicate()
print(out)
This is the error I get when running the python script:
python_on_whales.exceptions.NoSuchImage: The docker command executed was C:\Program Files\Docker\Docker\resources\bin\docker.EXE image inspect docker run --rm -it --volume ${PWD}/prime:/app $(docker build -q docker/prime) npm run build.
It returned with code 1
The content of stdout is '[]
'
The content of stderr is 'Error response from daemon: no such image: docker run --rm -it --volume ${PWD}/prime:/app $(docker build -q docker/prime) npm run build: invalid reference format: repository name must be lowercase
'
Running the command, docker run --rm -it--volume ${PWD}/prime:/app $(docker build -q docker/prime) npm run build in one long line in powershell works but we want a universal standard command for both pc and mac/linux

The Python on Whales docker.run() function doesn't take a docker run ... command line. It is a native Python API where you need to express the various Docker options as function parameters.
In principle you could rewrite this Python script using that API:
from pathlib import Path
from python_on_whales import docker
# build the image, returns an Image object
image = docker.build(Path.cwd() / 'docker' / 'prime')
# start the container; like `docker run ...`
docker.run(image,
command=['npm', 'run', 'build'],
volumes=[(Path.cwd() / 'prime', '/app')], # -v $(PWD)/prime:/app
interactive=True, # -i (required?)
tty=True, # -t (required?)
remove=True) # --rm
The return value from docker.run() (without detach=True) is the container's stdout, and the examples print() that data.

This might not be what you're looking for but you can always try this:
import platform
import subprocess
import os
cur_os = platform.system()
if cur_os == "Windows":
print("You are on windows")
os.system('Command here') # for windows
elif cur_os == "Darwin":
print("You are on mac")
subprocess.call('Command goes here') # for mac
Edit:
I'm intermediate with python so don't judge, if I did something wrong please give me feedback. Thanks.

Docker run inside a container

I have a Python method code using docker and I try to understand it. The method is here,
def exec(self, container_target, command, additional_options=""):
""" execte docker exec commmand and return the stdout or None when error"""
cmd = """docker exec -i "%s" sh -c '%s' %s""" % (
container_target, command, additional_options)
if self.verbose:
print(cmd)
try:
cp = subprocess.run(cmd,
shell=True,
check=True,
stdout=subprocess.PIPE)
return cp.stdout.decode("utf-8").strip()
except Exception as e:
print(f"Docker exec failed command {e}")
return None
I get the screenshot at the time of debugging,
The cmd value is found,
'docker exec -i "craft_p2-2" sh -c \'cd craft && composer show
--name-only | grep nerds-and-company/schematic | wc -l\' '
My understanding is the code using the shell of the container named craft_p2-2 and enters a folder named craft. Then, it checks if the Schematic plugin is installed. Is that correct?
This might be obvious for some, but, I don't come with a wealth of container knowledge and need to be sure of what's going on.

Run sequential commands in Python with subprocess

hope you can help. I need, in my Python script, to run the software container Docker with a specific image (Fenics in my case) and then to pass him a command to execute a script.
I've tried with subprocess:
cmd1 = 'docker exec -ti -u fenics name_of_my_container /bin/bash -l'
cmd2 = 'python2 shared/script_to_be_executed.py'
process = subprocess.Popen(shlex.split(cmd1),
stdout=subprocess.PIPE,stdin=subprocess.PIPE, stderr =
subprocess.PIPE)
process.stdin.write(cmd2)
print(first_process.stdout.read())
But it doesn't do anything. Suggestions?

Drop the -it flags in your call do docker, you don't want them. Also, don't try to send the command to execute into the container via stdin, but just pass the command to run in your call do docker exec.
I don't have a container running, so I'll use docker run instead, but the code below should give you a clue:
import subprocess
cmd = 'docker run python:3.6.4-jessie python -c print("hello")'.split()
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
out, err = p.communicate()
print(out)
This will run python -c print("hello") in the container and capture the output, so the Python (3.6) script will itself print
b'hello\n'
It will also work in Python 2.7, I don't know which version you're using on the host machine :)
Regarding communicating with a subprocess, see the official docs subprocess.Popen.communicate. Since Python 3.5 there's also subprocess.run, which makes your life even easier.
HTH!

You can use subprocess to call Fenics as an application, section 4.4 here.
docker run --rm -v $(pwd):/home/fenics/shared -w /home/fenics/shared quay.io/fenicsproject/stable "python3 my-code.py"

Popen can find the existing tool

I'm trying to run the following;
def conn(ad_group):
result = Popen(["sudo -S /opt/quest/bin/vastool", "-u host/ attrs 'AD_GROUP_NAME' | grep member"], stdout=PIPE)
return result.stdout
on a RedHat machine in a python script but I'm getting FileNotFoundError: [Errno 2] No such file or directory: 'sudo -S /opt/quest/bin/vastool'
I can run the command(sudo -S /opt/quest/bin/vastool -u host/ attrs 'AD_GROUP_NAME' | grep member) at the command line without a problem.
I'm sure I've messed up something in the function but I need an other set of eyes.
Thank you

You need to make the entire command a single string, and use the shell=True option because you're using a shell pipeline.
result = Popen("sudo -S /opt/quest/bin/vastool -u host/ attrs 'AD_GROUP_NAME' | grep member", stdout=PIPE, shell=True)

Python and nfqueue error

I'm trying a python script downloaded from a blog to send fake echo-replies after a ping from a machine.
The problem is that when I run the script, it gives me this error:
File "/usr/lib/python2.7/dist-packages/nfqueue.py", line 96, in
create_queue def create_queue(self,*args): return
_nfqueue.queue_create_queue(self, *args) RuntimeError: error during nfq_create_queue()
This is the part where it binds the queue:
import nfqueue
q = None
q = nfqueue.queue()
q.open()
q.bind(socket.AF_INET)
q.set_callback(cb)
q.create_queue(0)
try:
q.try_run()
except KeyboardInterrupt:
print "Exiting..."
q.unbind(socket.AF_INET)
q.close()
The error is on the q.create_queue(0), but I don't know what to do!

The obtained message may derive from an already running execution of your python script.
Assuming your script file is pyscriptname.py, run the following command to check if another instance of your script is already running:
ps -aux | grep "pyscriptname.py" | grep -v grep | wc -l
In case something returned value is greater than 0, you can solve the issue by running the following command:
kill -9 `ps aux | grep "pyscriptname.py" | grep -v grep | awk '{print $2}'`
Then, you can run again your python script:
python pyscriptname.py

nfqueue needs root privileges. So run the script as root or run it under sudo