We Keep Coding

Bash script to run python script that trigger a c++ executable

I have a c++ code that is compiled and can be executed.
Let's say the output file after compiling is executable.x.
I also have python script that can call this executable and run it.
pythonScript.py:
# lets say the path is absolute for simplicity.
file_path = 'C:/MyProject/code/executable.x'
# I need to pass an argument to main.cpp
subprocess.check_call([file_path, '-switch1'])
I can run the python script from terminal, and it runs the executable without any issue.
Then there is a shell command to run the python script.
myShell.sh
#!/bin/sh
pwd
(cd pythonScriptDirectory && python3 pythonScript.py)
pwd
By running the sh script, it sets the working directory (like how I run python script from terminal), and then it runs the python script. It seems it also finds the executable.x, but it always return with some error.
Is there any suggestion what might be wrong here, or what would be the debugging approach.
The return value specifies the error code 3221225785 in decimal, which is C000 0139 Hex. My assumption is that the executable file can be selected to run, but a working directory issue causes that libraries being used by executable cannot be loaded.
Here is directory structure:

Azure pipeline command line agent job seems to ignore the commands following the python environment activation?

I am using foll. code in azure pipeline command line agent job. But the commands after Venv_Project\scripts\activate do not show on the output. What could be the issue?
ECHO START
SET var=%cd%
ECHO %var%
python -m venv Venv_Project
SET var=%cd%
ECHO %var%
ECHO Venv created and now activating
Venv_Project\scripts\activate
SET var=%cd%
ECHO %var%
ECHO END

Replace the Venv_Project\scripts\activate with call Venv_Project\scripts\activate
Reference: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/command-line?view=azure-devops&tabs=yaml#running-batch-and-cmd-files
Running batch and .CMD files
Azure Pipelines puts your inline script
contents into a temporary batch file (.cmd) in order to run it. When
you want to run a batch file from another batch file in Windows CMD,
you must use the call command, otherwise the first batch file is
terminated. This will result in Azure Pipelines running your intended
script up until the first batch file, then running the batch file,
then ending the step. Additional lines in the first script wouldn't be
run. You should always prepend call before executing a batch file in
an Azure Pipelines script step.
Important
You may not realize you're running a batch file. For example, npm on
Windows, along with any tools that you install using npm install -g,
are actually batch files. Always use call npm to run NPM
commands in a Command Line task on Windows.

Azure pipeline command line agent job seems to ignore the commands following the python environment activation?
I could reproduce this issue on side. In order to further verify if the command was executed or not, or it executed but not display in the output.
I created a simple python script to create a .txt file, so we can verify that the command is executed by verifying the existence of the .txt file:
file = open('D:/a/1/s/Test.txt', 'w');
file.write('hello, \n world!')
When I build the command line task with private agent without python environment activation, it created the .txt file successfully. But if I activate the python environment activation, it will not create that file.
So, this further determines that the command is not being executed with Azure devops. But I have no way to further figure out the reason why Azure devops does not execute commands in the python virtual environment.
To resolve this issue, I would like suggest you report this issue on Developer Community:
https://developercommunity.visualstudio.com/spaces/21/index.html
which is our main forum for product issues. Thank you for helping us build a better Azure DevOps.
Hope this helps.

Cordova command not running through cron job

I am trying to make one cron job script in Python . For the start what i did i just add the code to run the cordova and show its version the same python file works if i run it through shell but when its run through cron job it gives me this error
env: node: No such file or directory
Python file has this code
#!/usr/bin/python
import os
import subprocess
subprocess.call('/usr/local/bin/cordova -v',shell=True)
Is there any solution for that ? What i get to understand that maybe when i run code through cronjob the global variables are not accessable . Is there anyway that i can get access to command line terminal through pipe and get access to all the global variables ?

Please use env command from your terminal and cronjob script (e.g.: env > cron_output file), it will give the environment variables available for normal terminal and cronjob. Compare that variables and export the required missing variables in your script.

How does one make sure that the python submission script in slurm is in the location from where the sbatch command was given?

I have a python submission script that I run with sbatch using slurm:
sbatch batch.py
when I do this things do not work properly because I assume, the batch.py process does not inherit the right environment variables. Thus instead of running batch.py from where the sbatch command was done, its ran from somewhere else (/ I believe). I have managed to fix this by doing wrapping the python script with a bash script:
#!/usr/bin/env bash
cd path/to/scripts
python script.py
this temporary hack sort of works it seems though it seems that it avoids the question all together rather than addressing it. Does someone know how to fix this in a better way?
I know for example, that in docker the -w or -WORKDIR exists so that the docker container knows where its suppose to be at. I was wondering if something like that existed for slurm.

Slurm is designed to push the user's environment at submit time to the job, except for variables explicitly disabled by the user or the system administrator.
But the way the script is run is as follows: the script is copied on the master node of the allocation in a Slurm specific directory and run from there, with the $PWD set to the directory where the sbatch command was run.
You can see that with a simple script like this one:
$ cat t.sh
#!/bin/bash
#
#SBATCH --job-name=test_ms
#SBATCH --output=res_ms.txt
echo $PWD
dirname $(readlink -f "$0")
$ sbatch t.sh
Submitted batch job 1109631
$ cat res_ms.txt
/home/damienfrancois/
/var/spool/slurm/job1109631
One consequence is that Python scripts that import modules in the current directory fail to do so. The workaround is then to explicitly add sys.path.append(os.getcwd()) before the failing imports.

Zsh/Bash: Path isn't what it should be

I stumbled upon something I just can't figure out. Following situation: I downloaded the python frontend to control dropbox via command line (dropbox.py). I put this file in the folder:
/home/username1/.dropbox-dist/dropbox.py
I made a simple bash script in /usr/bin called "dropbox":
#!/bin/bash
python /home/username1/.dropbox-dist/dropbox.py
Now when i run it following happens:
The whereis for the file:
root#linux_remote /home/username1 # whereis dropbox
dropbox: /usr/bin/dropbox
When i run it:
root#linux_remote /home/username1 # dropbox
zsh: no such file or directory: /home/username2/.dropbox-dist/dropboxd
Yeah. It tells me another username. To be specific: I'm logged in via SSH on this linuxbox. On the remote shell there is byobu running. In byobu runs zsh. Username2 equals the user that I'm currently logged in with on my local linuxbox, with which I connected:
username2#linux_local /home/username2 # ssh username1#linux_remote
Thats how I am connected.
So there must be a variable which was passed to my remote shell from my local shell, and python seems to read it, but I can't figure out which it would be.
Now.. look at that: When I type in the command that I wrote into the bash script:
username2#linux_remote /home/username2 # python /home/username1/.dropbox-dist/dropbox.py
Dropbox command-line interface
So it runs if I do it manually.
Another thing: If I run it with the whole path it works too:
root#linux_remote /home/username1 # /usr/bin/dropbox
Dropbox command-line interface
And it does work if I run it via login-shell, for example using "bash -l" and then trying to run "dropbox".
It doesn't work either if I change the hashbang to "#!/usr/bin/zsh"
Any ideas on this?

whereis doesn't do what you think: it searches a specific set of directories, not $PATH. which searches $PATH so you need to use which to find out which executable will be executed by a given name.
Edit: which as an external program (for shells that do not have a builtin command, such as bash) will not give a right answer for some cases, e.g. shell aliases. The type builtin should be used instead (it also should be available more widely as it's mandated by POSIX, though not necessarily as a builtin).