I'm new to python and programming in general. I want to assign options to a variable by using optparse module in ipython. My code is as follows:
import sys
import optparse
parser = optparse.OptionParser()
parser.add_option('-v', action="store_true", dest='verbose', default=False)
(options, others) = parser.parse_args()
print options.verbose
if options.verbose:
print "Not yet"
else:
print "Done"
I saved them in a file and I can run it in bash like this:
$ python filename.py -verbose
Now I want to assign the whole code to a variable. I hope it can be run like this:
$ myvar -verbose
How can I do that? Thanks.
One way to do this would be to make your python script executable with a shebang. Add this as the first line in your filename.py file
#!/usr/bin/env python
Next you need to change the file permissions to be executable. At the command line, execute:
$ chmod +x filename.py
Then you can execute filename.py directly:
$ ./filename.py -verbose
You could of course rename filename.py to myvar, or make a symolic link like this:
$ ln -s filename.py myvar
Now you can do:
$ ./myvar -verbose
If you don't want to add the /. (which just tells the shell that the executable is in the current directory), or want to be able to use the "myvar" command line from anywhere, you could add your working directory to the environment PATH:
export PATH=$PATH:$PWD
You might want to read a good tutorial on command line usage for more on this sort of thing. A little time invested can be really rewarding!
Aliases are your friend:
alias myvar="python filename.py"
Just assign to the shell variable:
myvar='python filename.py'
$myvar -verbose
Note the leading $, which would not be required if you used an alias (#Nick Edward's solution).
By the way, -verbose will set options v, e, r, b, o, s, and e. Maybe you mean a long option, like --verbose?
Related
I downloaded a script written in Python, called let's say 'myScript.py', but I don't know how to run it.
It has the following structure:
import numpy as np
import argparse
parser = argparse.ArgumentParser(description='manual to this script')
parser.add_argument('--file', type=str, default = None)
parser.add_argument('--timeCor', type=bool, default = False)
parser.add_argument('--iteration', type=str, default = 'Newton')
args = parser.parse_args()
def func1(file):
...
def func2(file):
...
def calculate(data, timeCor=False, iteration='Newton'):
...
if __name__ == "__main__":
print('\n--- Calculating ---\nN file:',args.file)
print('Time correction =',args.timeCor,
'\nIteration strategy =',args.iteration,'\n')
rawdata,data = func2(args.file)
pos = calculate(data,timeCor=args.timeCor,iteration=args.iteration)
for each in pos:
print('Pos:',format(np.uint8(each[0]),'2d'),each[1:-1])
np.savetxt('pos.csv',pos,delimiter=',')
print('--- Save file as "pos.csv" in the current directory ---')
How can I run it from command line? And from another script?
Thank you in advance!
If I understood your question correctly, you are asking about executing this python program from another python script. This can be done by making use of subprocess.
import subprocess
process = subprocess.run([“python3”, “path-to-myScript.py”, “command line arguments here”], capture_output=True)
output = process.stdout.decode() # stdout is bytes
print(output)
If you do not want to provide the command in a list, you can add shell=True as an argument to subprocess.run(), or you can use shlex.split().
If you are on windows, replace python3 with python.
However this solution is not very portable, and on a more general note, if you are not strictly needing to run the script, I would recommend you to import it and call its functions directly instead.
To run the python script from command line:
python myScript.py --arguments
And as #treuss has said, if you are on a Unix system (macOS or Linux) you can add a shebang to the top of the script, but I recommend to use the following instead:
#!/usr/bin/env python3
for portability sake, as the actual path of python3 may vary.
To run the edited program:
chmod +x myScript # to make file executable, and note that there is no longer a .py extension as it is not necessary
./myScript --arguments
Run it by executing:
python myScript.py
Alternatively, on systems which support it (UNIX-Like systems), you can add what's called a she-bang to the first line of the file:
#!/usr/bin/python
Note that /usr/bin/python should be the actual path where your python is located. After that, make the file executable:
chmod u+x myScript.py
and run it either from the current directory:
./myScript.py
or from a different directory with full or relative path:
/path/to/python/scripts/myScript.py
I don't really know how to ask this question but I can describe what I want to achieve. I would update any edits that would be suggested.
I have a python module that makes use of some command line arguments. Using the module requires some initial setup outside of the python interpreter. The python file that does the setup runs fine, but the problem is that I have to dig through the python installation to find where that file is located i.e. I have to do python full-path-to-setup-script.py -a argA -b argB etc.I would like to call the setup script like this
some-setup-command -a argA -b argB etc.
I want to achieve something like
workon environmnent_name as in the virtualenv module or
pipenv install as in the pipenv module.
I know both of the above commands call a script of some kind (whether bash or python). I've tried digging through the source codes of virtualenv and pipenv without any success.
I would really appreciate if someone could point me to any necessary resource for coding such programs.
If full-path-to-setup-script.py is executable and has a proper shebang line
#! /usr/bin/env python
then you can
ln -s full-path-to-setup-script.py ~/bin/some-command
considering ~/bin exists and is in your PATH,
and you'll be able to invoke
some-command -a argA -b argB
It's a bit difficult to understand what you're looking for, but python -m is my best guess.
For example, to make a new Jupyter kernel, we call
python -m ipykernel arg --option --option
Where arg is the CLI argument and option is a CLI option, and ipykernel is the module receiving the args and options.
Commands that are callable from the command prompt are located in one of the directories in your system's PATH variable. If you are on Windows, you see the locations via:
echo %PATH%
Or if you want a nicer readout:
powershell -c "$env:path -split(';')"
One solution is to create a folder, add it to your system's PATH, and then create a callable file that you can run. In this example we will create a folder in your user profile, add it to the path, then create a callable file in that folder.
mkdir %USERPROFILE%\path
set PATH=%PATH%%USERPROFILE%\path;
setx PATH %PATH%
In the folder %USERPROFILE%\path, we create a batch file with following content:
# file name:
# some-command.bat
#
python C:\full\path\to\setup-script.py %*
Now you should be able to call
some-command -a argA -b argB
And the batch file will call python with python script and pass the arguments you added.
Looking at the above answers, I see no one has mentioned this:
You can of course compile the python file and give executable permissions with
chmod +x filename.py
and then run it as
./filename.py -a argA -b argB ...
Moreover, you can also remove the extention .py (since it is an executable now) and then run it only as
./filename -a argA -b argB ...
I have a python script (e.g. test.py) and a commands.txt file which contains a custom bash function (e.g. my_func) along with its parameters, e.g.
my_func arg1 arv2; my_func arg3 arg4; my_func arg5 arg6;
This function is included in the ~/.bash_profile.
What I have tried to do is:
subprocess.call(['.', path/to/commands.txt], shell=True)
I know this is not the best case, in terms of setting the shell argument into True, but I cannot seem to implement it even in this way. What I get when I run test.py is:
my_func: command not found
You will need to invoke bash directly, and instruct it to process both files.
At the command-line, this is:
bash -c '. ~/.bash_profile; . commands.txt'
Wrapping it in python:
subprocess.call(['bash', '-c', '. ~/.bash_profile; . commands.txt'])
You could also source ~/.bash_profile at the top of commands.txt. Then you'd be able to run a single file.
It may make sense to extract the function to a separate file, since .bash_profile is intended to be used by login shells, not like this.
If the first line of your commands.txt file had the correct shebang (for example #!/bin/bash) making your file executable (chmod +x commands.txt) will be enough :
subprocess.call("path/to/commands.txt")
I'm making a python script for Travis CI.
.travis.yml
...
script:
- support/travis-build.py
...
The python file travis-build.py is something like this:
#!/usr/bin/env python
from subprocess import check_call
...
check_call(r"mv !(my_project|cmake-3.0.2-Darwin64-universal) ./my_project/final_folder", shell=True)
...
When Travis building achieves that line, I'm getting an error:
/bin/sh: 1: Syntax error: "(" unexpected
I just tried a lot of different forms to write it, but I get the same result. Any idea?
Thanks in advance!
Edit
My current directory layout:
- my_project/final_folder/
- cmake-3.0.2-Darwin64-universal/
- fileA
- fileB
- fileC
I'm trying with this command to move all the current files fileA, fileB and fileC, excluding my_project and cmake-3.0.2-Darwin64-universal folders into ./my_project/final_folder. If I execute this command on Linux shell, I get my aim but not through check_call() command.
Note: I can't move the files one by one, because there are many others
I don't know which shell Travis are using by default because I don't specify it, I only know that if I write the command in my .travis.yml:
.travis.yml
...
script:
# Here is the previous Travis code
- mv !(my_project|cmake-3.0.2-Darwin64-universal) ./my_project/final_folder
...
It works. But If I use the script, it fails.
I found this command from the following issue:
How to use 'mv' command to move files except those in a specific directory?
You're using the bash feature extglob, to try to exclude the files that you're specifying. You'll need to enable it in order to have it exclude the two entries you're specifying.
The python subprocess module explicitly uses /bin/sh when you use shell=True, which doesn't enable the use of bash features like this by default (it's a compliance thing to make it more like original sh).
If you want to get bash to interpret the command; you have to pass it to bash explicitly, for example using:
subprocess.check_call(["bash", "-O", "extglob", "-c", "mv !(my_project|cmake-3.0.2-Darwin64-universal) ./my_project/final_folder"])
I would not choose to do the job in this manner, though.
Let me try again: in which shell do you expect your syntax !(...) to work? Is it bash? Is it ksh? I have never used it, and a quick search for a corresponding bash feature led nowhere. I suspect your syntax is just wrong, which is what the error message is telling you. In that case, your problem is entirely independent form python and the subprocess module.
If a special shell you have on your system supports this syntax, you need to make sure that Python is using the same shell when invoking your command. It tells you which shell it has been using: /bin/sh. This is usually just a link to the real shell executable. Does it point to the same shell you have tested your command in?
Edit: the SO solution you referenced contains the solution in the comments:
Tip: Note however that using this pattern relies on extglob. You can
enable it using shopt -s extglob (If you want extended globs to be
turned on by default you can add shopt -s extglob to .bashrc)
Just to demonstrate that different shells might deal with your syntax in different ways, first using bash:
$ !(uname)
-bash: !: event not found
And then, using /bin/dash:
$ !(uname)
Linux
The argument to a subprocess.something method must be a list of command line arguments. Use e.g. shlex.split() to make the string be split into correct command line arguments:
import shlex, subprocess
subprocess.check_call( shlex.split("mv !(...)") )
EDIT:
So, the goal is to move files/directories, with the exemption of some file(s)/directory(ies). By playing around with bash, I could get it to work like this:
mv `ls | grep -v -e '\(exclusion1\|exclusion2\)'` my_project
So in your situation that would be:
mv `ls | grep -v -e '\(myproject\|cmake-3.0.2-Darwin64-universal\)'` my_project
This could go into the subprocess.check_call(..., shell=True) and it should do what you expect it to do.
I've tried googling the answer but with no luck.
I need to use my works supercomputer server, but for my python script to run, it must be executed via a shell script.
For example I want job.sh to execute python_script.py
How can this be accomplished?
Just make sure the python executable is in your PATH environment variable then add in your script
python path/to/the/python_script.py
Details:
In the file job.sh, put this
#!/bin/sh
python python_script.py
Execute this command to make the script runnable for you : chmod u+x job.sh
Run it : ./job.sh
Method 1 - Create a shell script:
Suppose you have a python file hello.py
Create a file called job.sh that contains
#!/bin/bash
python hello.py
mark it executable using
$ chmod +x job.sh
then run it
$ ./job.sh
Method 2 (BETTER) - Make the python itself run from shell:
Modify your script hello.py and add this as the first line
#!/usr/bin/env python
mark it executable using
$ chmod +x hello.py
then run it
$ ./hello.py
Save the following program as print.py:
#!/usr/bin/python3
print('Hello World')
Then in the terminal type:
chmod +x print.py
./print.py
You should be able to invoke it as python scriptname.py e.g.
# !/bin/bash
python /home/user/scriptname.py
Also make sure the script has permissions to run.
You can make it executable by using chmod u+x scriptname.py.
Imho, writing
python /path/to/script.py
Is quite wrong, especially in these days. Which python? python2.6? 2.7? 3.0? 3.1? Most of times you need to specify the python version in shebang tag of python file. I encourage to use #!/usr/bin/env python2 #or python2.6 or python3 or even python3.1 for compatibility.
In such case, is much better to have the script executable and invoke it directly:
#!/bin/bash
/path/to/script.py
This way the version of python you need is only written in one file. Most of system these days are having python2 and python3 in the meantime, and it happens that the symlink python points to python3, while most people expect it pointing to python2.
This works for me:
Create a new shell file job. So let's say:
touch job.sh and add command to run python script (you can even add command line arguments to that python, I usually predefine my command line arguments).
chmod +x job.sh
Inside job.sh add the following py files, let's say:
python_file.py argument1 argument2 argument3 >> testpy-output.txt && echo "Done with python_file.py"
python_file1.py argument1 argument2 argument3 >> testpy-output.txt && echo "Done with python_file1.py"
Output of job.sh should look like this:
Done with python_file.py
Done with python_file1.py
I use this usually when I have to run multiple python files with different arguments, pre defined.
Note: Just a quick heads up on what's going on here:
python_file.py argument1 argument2 argument3 >> testpy-output.txt && echo "completed with python_file.py" .
Here shell script will run the file python_file.py and add multiple command-line arguments at run time to the python file.
This does not necessarily means, you have to pass command line arguments as well.
You can just use it like: python python_file.py, plain and simple.
Next up, the >> will print and store the output of this .py file in the testpy-output.txt file.
&& is a logical operator that will run only after the above is executed successfully and as an optional echo "completed with python_file.py" will be echoed on to your cli/terminal at run time.
This works best for me:
Add this at the top of the script:
#!c:/Python27/python.exe
(C:\Python27\python.exe is the path to the python.exe on my machine)
Then run the script via:
chmod +x script-name.py && script-name.py
I use this and it works fine
#/bin/bash
/usr/bin/python python python_script.py
Since the other posts say everything (and I stumbled upon this post while looking for the following).
Here is a way how to execute a python script from another python script:
Python 2:
execfile("somefile.py", global_vars, local_vars)
Python 3:
with open("somefile.py") as f:
code = compile(f.read(), "somefile.py", 'exec')
exec(code, global_vars, local_vars)
and you can supply args by providing some other sys.argv
Here I have demonstrated an example to run python script within a shell script. For different purposes you may need to read the output from a shell command, execute both python script and shell command within the same file.
To execute a shell command from python use os.system() method. To read output from a shell command use os.popen().
Following is an example which will grep all processes having the text sample_program.py inside of it. Then after collecting the process IDs (using python) it will kill them all.
#!/usr/bin/python3
import os
# listing all matched processes and taking the output into a variable s
s = os.popen("ps aux | grep 'sample_program.py'").read()
s = '\n'.join([l for l in s.split('\n') if "grep" not in l]) # avoiding killing the grep itself
print("To be killed:")
print(s)
# now manipulating this string s and finding the process IDs and killing them
os.system("kill -9 " + ' '.join([x.split()[1] for x in s.split('\n') if x]))
References:
Execute a python program from within a shell script
Assign output of os.system to a variable and prevent it from being displayed on the screen
If you have a bash script and you need to run inside of it a python3 script (with external modules), I recommend that you point in your bash script to your python path like this.
#!/usr/bin/env bash
-- bash code --
/usr/bin/python3 your_python.py
-- bash code --