We Keep Coding

How to run python script from command line or from another script

I downloaded a script written in Python, called let's say 'myScript.py', but I don't know how to run it.
It has the following structure:
import numpy as np
import argparse
parser = argparse.ArgumentParser(description='manual to this script')
parser.add_argument('--file', type=str, default = None)
parser.add_argument('--timeCor', type=bool, default = False)
parser.add_argument('--iteration', type=str, default = 'Newton')
args = parser.parse_args()
def func1(file):
...
def func2(file):
...
def calculate(data, timeCor=False, iteration='Newton'):
...
if __name__ == "__main__":
print('\n--- Calculating ---\nN file:',args.file)
print('Time correction =',args.timeCor,
'\nIteration strategy =',args.iteration,'\n')
rawdata,data = func2(args.file)
pos = calculate(data,timeCor=args.timeCor,iteration=args.iteration)
for each in pos:
print('Pos:',format(np.uint8(each[0]),'2d'),each[1:-1])
np.savetxt('pos.csv',pos,delimiter=',')
print('--- Save file as "pos.csv" in the current directory ---')
How can I run it from command line? And from another script?
Thank you in advance!

If I understood your question correctly, you are asking about executing this python program from another python script. This can be done by making use of subprocess.
import subprocess
process = subprocess.run([“python3”, “path-to-myScript.py”, “command line arguments here”], capture_output=True)
output = process.stdout.decode() # stdout is bytes
print(output)
If you do not want to provide the command in a list, you can add shell=True as an argument to subprocess.run(), or you can use shlex.split().
If you are on windows, replace python3 with python.
However this solution is not very portable, and on a more general note, if you are not strictly needing to run the script, I would recommend you to import it and call its functions directly instead.
To run the python script from command line:
python myScript.py --arguments
And as #treuss has said, if you are on a Unix system (macOS or Linux) you can add a shebang to the top of the script, but I recommend to use the following instead:
#!/usr/bin/env python3
for portability sake, as the actual path of python3 may vary.
To run the edited program:
chmod +x myScript # to make file executable, and note that there is no longer a .py extension as it is not necessary
./myScript --arguments

Run it by executing:
python myScript.py
Alternatively, on systems which support it (UNIX-Like systems), you can add what's called a she-bang to the first line of the file:
#!/usr/bin/python
Note that /usr/bin/python should be the actual path where your python is located. After that, make the file executable:
chmod u+x myScript.py
and run it either from the current directory:
./myScript.py
or from a different directory with full or relative path:
/path/to/python/scripts/myScript.py

Sending python command line arguments to IPython

How can we start ipython REPL and instruct it to pass some command-line arguments to the underlying python interpreter?
For example, we can open a python REPL with increased verbosity by using
python -v
But I could not see how to pass that flag through when opening IPython.

I'd say the best way to do that would be to explicitly launch ipython with python:
python /usr/bin/ipython
as the ipython executable is just a python script ; or you can launch ipython by telling python to load the ipython library:
python -m IPython.frontend.terminal.ipapp
and then you can add all the native python arguments:
python -v /usr/bin/ipython
python -v -m IPython.frontend.terminal.ipapp
HTH

You could write your own ipython shebang script.
Here I copied my ipython script and added the -v
#!/usr/local/bin/python3.5 -v
# -*- coding: utf-8 -*-
import re
import sys
from IPython import start_ipython
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(start_ipython())
Now when I execute ./vipython I get many pages of import information upon startup and shutdown.
I gather from other SO questions that I might not be able to add multiple options to such a shebang line.
How to use multiple arguments with a shebang (i.e. #!)?
So for example
#!/usr/local/bin/python3.5 -vv
works, but
#!/usr/local/bin/python3.5 -v -v
doesn't - it gives me
1008:~/mypy$ ./vipython
Unknown option: -
usage: /usr/local/bin/python3.5 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.

That -v option affects the behavior of the interpreter itself, not just the REPL. You get the extra import information regardless of whether you add the -i option.
Here's the default script that launches ipython (or at least one version of that)
1522:~/mypy$ cat /usr/local/bin/ipython3.5
#!/usr/local/bin/python3.5
# -*- coding: utf-8 -*-
import re
import sys
from IPython import start_ipython
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(start_ipython())
and from within an ipython session:
In [1153]: from IPython import start_ipython
In [1154]: start_ipython??
String form: <function start_ipython at 0xb697edac>
File: /usr/lib/python3/dist-packages/IPython/__init__.py
Definition: start_ipython(argv=None, **kwargs)
Source:
def start_ipython(argv=None, **kwargs):
"..."
from IPython.terminal.ipapp import launch_new_instance
return launch_new_instance(argv=argv, **kwargs)
Eventually the argv are passed to an argparse parser. Ipython populates that parser with arguments derived from its config files. So you have several options for setting parameters - default config, profile config, and commandline. But all of this is after the interpreter has been launched. Some things are acted on in the same was a with an interpreter REPL, but not all (-m, but not -v).
When -v is used as zmo suggests, we see all the imports of ipython code - which are quite a few. Are you interested in those, or are you more interested in imports related to your own script?
I use ipython a lot to test answers, especially for numpy. In fact my default ipython call has the --pylab flag. But to test stand alone scripts I use plain python (often called from a terminal window in my editor). Sometimes though I'll %run a script from within ipython. That loads the module into the main workspace, making it easy to perform %timeit tests on functions.
Other ipython scripts use
#!/usr/bin/python
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point(...)
I don't write much code using this style, but I don't see how that kind initiation can pass argv on through to the interpreter. By the time the module has been loaded and start running, the interpreter is already running.
In general it looks like ipython handles options like -i, -m, -c in basically the same way as the regular python. It may doing so with its own code, rather than delegating to the interpreter. But things like -v, -O, -t apply to the interpreter, not the REPL, and aren't handled by ipython code.

Python subprocess calling another py not producing output

I have 2 programs, one is calling the other through subprocess. Running this in pyCharm. My issue is that the call to the second program doesn't print out the desired string (see programs). What am I doing wrong, or is my understanding of the subprocess wrong?
this is something.py:
import subprocess
def func():
print("this is something")
sb = subprocess.call("diff.py", shell=True)
return sb
if __name__=="__main__":
func()
this is diff.py:
print("this is diff running")
def caller():
print("this is diff running called from name main")
if __name__=="__main__":
caller()
I decided to try subprocessing instead of importing for the purpose of running the calls concurrently in diff threads in the future. For now I just wanted to make sure I grasp subprocessing but I'm stuck at this basic level with this issue and get figure it out.

You must use python to run python files.
import subprocess
def func():
print("this is something")
sb = subprocess.call("python diff.py", shell=True)
# It is also important to keep returns in functions
return sb
if __name__=="__main__":
func()
I would be careful to understand the layout of how pycharm saves files. Maybe consider trying to run a program that already exists for the Windows command line if you are just trying to learn about the subprocess module.
import subprocess
print("this is where command prompt is located")
sb = subprocess.call("where cmd.exe", shell=True)
returns
this is where command prompt is located
C:\Windows\System32\cmd.exe

Thank you.
subprocess.call("python something.py", shell=True) now works as intended but for some reason the very same call from pyCharm does not return the second string from diff.py I assume the issue is with pyCharm then

To run diff.py script from the current directory using the same Python interpreter that runs the parent script:
#!/usr/bin/env python
import sys
from subprocess import check_call
check_call([sys.executable, 'diff.py'])
do not use shell=True unless it is necessary e.g., unless you need to run an internal command such as dir, you don't need shell=True in most cases
use check_call() instead of call() to raise an exception if the child script returns with non-zero exit code
Why[When] I try python something.py pyCharm fires up to interpret it.
You should associate .py extension with py (Python launcher). Though if running the command:
T:\> python something.py
literally brings up PyCharm with the file something.py opened instead of running the script using a Python interpreter then something is really broken. Find out what program is run then you type python (without arguments).
Make sure you understand the difference between:
subprocess.Popen(['python', 'diff.py'])
subprocess.Popen('diff.py')
subprocess.Popen('diff.py', shell=True)
os.startfile('diff.py')
os.startfile('diff.py', 'edit')
Try to run it from the command-line (cmd.exe) and from IDLE (python3 -m idlelib) and see what happens in each case.
You should prefer to import the Python module and use multiprocessing, threading modules if necessary to run the corresponding functions instead of running the Python script as a child process via subprocess module.

python module and command line program

I have a code which I'd like people to be able to use as a stand alone python program, or to import as a module for their own codes. Is there a way to package a module which can also include a program that can be run from the command line?
I.e. from the command-line:
> ZHermesCode
or within ipython or a python script:
import ZHermesCode

Look up Setuptools automatic script creation. For example, python-scss uses this technique to make scss an executable shell command.
In setup.py:
setup(
# ...
entry_points={
'console_scripts': [
'scss = scss.tool:main',
]
},
)
Then defining a function main in scss/tool.py. It also uses the technique mentioned by Loocid to make the tool.py file itself directly executable (as opposed to the wrapper script that is publicly installed by setuptools according to the recipe above):
if __name__ == '__main__':
main()

If you use:
if name == '__main__':
The code held in that 'if' statement will only be ran if someone runs your program from the command line.
If someone was to import your module instead, the code will not run.
Eg
def test(st):
return(st)
if name == "__main__":
print(test("hello"))
If you run this program from the command line, it will print "hello". However, someone could also import this module and have access to the "test" function to use in their own programs.

Python erratic behaviour: /usr/bin/python: No module named hotspotd.hotspotd

I'm quite new to python and just created my first opensource python project - a daemon to create wifi hotspots on linux.
I've used distutils to build the package. To start the daemon once installation is done, I've registered the below script that simply starts the daemon by calling the corresponding python module:
#!/bin/bash
python -m hotspotd $*
And this is the setup.py that registers it:
#INSTALL IT
from distutils.core import setup
s = setup(name='hotspotd',
version='0.1',
description='Small daemon to create a wifi hotspot on linux',
license='MIT',
author='Prahlad Yeri',
author_email='prahladyeri#yahoo.com',
url='https://github.com/prahladyeri/hotspotd',
#py_modules=['hotspotd','cli'],
packages=['hotspotd'],
package_dir={'hotspotd': ''},
package_data={'hotspotd': ['run.dat']},
scripts=['hotspotd']
#data_files=[('config',['run.dat'])],
)
Now, this works fine on my machine and some other machines I've tested. However, as indicated by the open issue on the github, some users are unable to run that script. It gives the error:
No module named hotspotd.main; 'hotspotd' is a package and cannot be directly executed
Apparently it expects the entire package.module syntax which is hotspotd.hotspotd on their setups. However, on my machine the full syntax doesn't work, and only hotspotd works. Whats going on here?

I had to change my script, so that instead of directly passing the module as argument, I import this module in the script itself and call the hotspotd.main function from there:
#!/usr/bin/env python
##author: Prahlad Yeri
##description: Small daemon to create a wifi hotspot on linux
##license: MIT
#python -m hotspotd $*
import hotspotd.main
import sys
import os
import argparse
if __name__ == "__main__":
#check root or not
if os.getenv('USER') != 'root':
print "You need root permissions to do this, sloth!"
sys.exit(1)
parser = argparse.ArgumentParser(description='A small daemon to create a wifi hotspot on linux')
parser.add_argument('-v', '--verbose', required=False, action='store_true')
parser.add_argument('command', choices=['start', 'stop', 'configure'])
args = parser.parse_args()
hotspotd.main.main(args)