We Keep Coding

Print bash script output in Python

I have a Python code that will run a script file.The script file will output the version tag of specific git repository.
My script file named 'start.sh' is as follows:
#!/bin/sh
git clone https://xxxxxxxxxx:x-oauth-basic#github.com/xxxxxxx/xxxxx.git
cd xxxxxxxx
git config --global user.email "xxxxxxxx"
git config --global user.name "xxxxxxxxx"
IMAGE_TAG=`echo \`git describe --tags\``
echo $IMAGE_TAG
My Python code is as follows:
import os
git_tag = os.popen('sh start.sh')
print(git_tag)
When I run the script file separately, it will return me the git tag. But, Whenever I try to print it in the Python code, it's not working.
How can I solve this issue?

Since you are using Python anyway, you could think of using GitPython as an alternative. You can list all your tags with:
from git import Repo
repo = Repo("path/to/repo")
print(repo.tags)

Try like this
from subprocess import check_output
out = check_output(['sh', 'start.sh'])
print(out)

According to python's documentation, os.popen() function has been deprecated since version 2.6. You might want to use subprocess module.
P.S: I wanted to comment but couldn't hence this answer.

For python3-X:
from subprocess
git_tag = subprocess.run(['sh', 'start.sh'], capture_output=True, text=True)
print(git_tag.stdout)

How to add environment variables to the bash opened by subprocess module?

I need to use the wget in a Python script with the subprocess.call function, but it seems the "wget" command cannot be identified by the bash subprocess opened by python.
I have added the environment variable (the path where wget is):
export PATH=/usr/local/bin:$PATH
to the ~/.bashrc file and the ~/.bash_profile file on my mac and guaranteed to have sourced them.
And the python script looks like:
import subprocess as sp
cmd = 'wget'
process = sp.Popen(cmd ,stdout=sp.PIPE, stdin=sp.PIPE,
stderr=sp.PIPE, shell=True ,executable='/bin/bash')
(stdoutdata, stderrdata) = process.communicate()
print stdoutdata, stderrdata
The expected output should be like
wget: missing URL
Usage: wget [OPTION]... [URL]...
But the result is always
/bin/bash: wget: command not found
Interestingly I can get the help output if I type in wget directly in a bash terminal, but it never works in the python script. How could it be?
PS:
If I change the command to
cmd = '/usr/local/bin/wget'
then it works. So I am sure I got wget installed.

You can pass an env= argument to the subprocess functions.
import os
myenv = os.environ.copy
myenv['PATH'] = '/usr/local/bin:' + myenv['PATH']
subprocess.run(..., env=myenv)
However, you probably want to avoid running a shell at all, and instead augment the PATH that Python uses to find the binary to run in the subprocess call.
import subprocess as sp
import os
os.environ['PATH'] = '/usr/local/bin:' + os.environ['PATH']
cmd = 'wget'
# use run instead of Popen
# don't needlessly use a shell
# and thus put [cmd] as a list
process = sp.run([cmd], stdout=sp.PIPE, stdin=sp.PIPE,
stderr=sp.PIPE,
universal_newlines=True)
print(process.stdout, process.stderr)
Running Bash commands in Python explains the changes I made in more detail.
However, there is no good reason to use an external utility for this; Python requests does pretty everything wget does, often more naturally and with more control over what exactly it does.

How do I make a python script executable?

How can I run a python script with my own command line name like myscript without having to do python myscript.py in the terminal?

Add a shebang line to the top of the script:
#!/usr/bin/env python
Mark the script as executable:
chmod +x myscript.py
Add the dir containing it to your PATH variable. (If you want it to stick, you'll have to do this in .bashrc or .bash_profile in your home dir.)
export PATH=/path/to/script:$PATH

The best way, which is cross-platform, is to create setup.py, define an entry point in it and install with pip.
Say you have the following contents of myscript.py:
def run():
print('Hello world')
Then you add setup.py with the following:
from setuptools import setup
setup(
name='myscript',
version='0.0.1',
entry_points={
'console_scripts': [
'myscript=myscript:run'
]
}
)
Entry point format is terminal_command_name=python_script_name:main_method_name
Finally install with the following command.
pip install -e /path/to/script/folder
-e stands for editable, meaning you'll be able to work on the script and invoke the latest version without need to reinstall
After that you can run myscript from any directory.

I usually do in the script:
#!/usr/bin/python
... code ...
And in terminal:
$: chmod 755 yourfile.py
$: ./yourfile.py

Another related solution which some people may be interested in. One can also directly embed the contents of myscript.py into your .bashrc file on Linux (should also work for MacOS I think)
For example, I have the following function defined in my .bashrc for dumping Python pickles to the terminal, note that the ${1} is the first argument following the function name:
depickle() {
python << EOPYTHON
import pickle
f = open('${1}', 'rb')
while True:
try:
print(pickle.load(f))
except EOFError:
break
EOPYTHON
}
With this in place (and after reloading .bashrc), I can now run depickle a.pickle from any terminal or directory on my computer.

The simplest way that comes to my mind is to use "pyinstaller".
create an environment that contains all the lib you have used in your code.
activate the environment and in the command window write pip install pyinstaller
Use the command window to open the main directory that codes maincode.py is located.
remember to keep the environment active and write pyinstaller maincode.py
Check the folder named "build" and you will find the executable file.
I hope that this solution helps you.
GL

I've struggled for a few days with the problem of not finding the command py -3 or any other related to pylauncher command if script was running by service created using Nssm tool.
But same commands worked when run directly from cmd.
What was the solution? Just to re-run Python installer and at the very end click the option to disable path length limit.
I'll just leave it here, so that anyone can use this answer and find it helpful.

How to implement full wget command into python

Is there a way you can use a full wget command into python?
I know that we can do this: os.system('wget %s' %%url)
But I want a full command with all of the data saved into a directory:
wget -r --accept "*.exe,*.dll,*.zip,*.msi,*.rar,*.iso" ftp://ftp.apple.asimov.com/ -P e:\e

There is the subprocess module for that (this is what os.system calls but with a bit more flexibility). Specifically, you can use the call function in the following way to execute any command
import subprocess
subprocess.call(r'wget -r --accept "*.exe,*.dll,*.zip,*.msi,*.rar,*.iso" ftp://ftp.apple.asimov.com/ -P e:\e', shell=True)
Alternatively, you can pass individual arguments as a list omitting the shell flag:
subprocess.call(['wget', '-r', ...])
Also check the return value for errors. For details, see the standard library documentation on subprocess.

pexpect.run can not run a long command

I am using pexpect.run to execute a command. See below:
cmd = "grep -L killed /dir/dumps/*MAC-66.log"
output = pexpect.run(cmd)
When I run this, output equals to:
grep: /dir/dumps/*MAC-66.log: No such file or directory
But when I run the same command in my shell, it works, everytime. I don't see the problem. Any help is appreciated! Does pexpect.run require the command to be split in some fancy way?

Your shell is interpreting the glob, pexpect is not. You could either use python's glob.glob() function to evaluate the glob yourself, or run it through your shell, for example:
cmd = "bash -c 'grep -L killed /dir/dumps/*MAC-66.log'"
Also, if all you're after is output of this command, you ought to check out the subprocess module.