We Keep Coding

How to use a python module in combination with a command from shell

I would like to use a python module from shell (to be exact: indirect from gnuplot). I do not want to write for every call an extra script or implement some I/O logic.
Let's say as a minimal working example, I have a python module module_foo.py with
#!/usr/bin/python
def bar():
print(1,2,3)
My question is:
Why isn't it possible to use a python module combining module loading and command execution like here?:
$ python -m module_foo -c 'bar()'
When executed, nothing happens. But what does work, is using only a command call like this
$ python -c 'import module_foo; module_foo.bar()'
1 2 3
or this
$ python -c 'from module_foo import *; bar()'
1 2 3
As soon as I load a module before, even a syntactically errorneous command is “accepted” – not executed, I suppose (the bracked of the call to bar isn't closed):
$ python -m module_foo -c 'bar('
$
It is, however, possible to use the -m module option using a python unit test (from the python docs):
python -m unittest test_module1 test_module2
The python manpage says for both options:
-c command
Specify the command to execute (see next section). This terminates the option
list (following options are passed as arguments to the command).
-m module-name
Searches sys.path for the named module and runs the corresponding .py file as
a script.
So I'd expect to be able to use path options in this -m ... -c ..., but not in reverse order -c ... -m ...'. Am I missing something obvious?

If you want your Python module to be executable and to call function bar(), you should add this to the end of the python file:
if __name__ == "__main__": # this checks that the file is "executed", rather than "imported"
bar() # call the function you want to call
Then call:
python module_foo.py
If you want more control, you can pass arguments to the script and access them from sys.argv.
For even more flexibility in arguments passed to the script, see argparse module.

profiling a python module from the command line

I have a python project where I execute the app as a module using the -m flag. So something like:
python -m apps.validate -i input.mp4
Now, I want to profile it using the command line. So the inline examples suggest invoking cProfile itself a module. However, I cannot do something like:
python -m cProfile apps.validate -i input.mp4
However, this results in the error "No such file or directory". I cannot just go to the apps directory and launch validate.py due to relative imports.
Is there a way to profile a module on the command line?

Instead of running cProfile in shell, maybe you can use cProfile in your python script by adding some code in apps.validate or creating a new script and import apps.validate like this. Maybe some typo below :)
import cProfile
import sys
def run_validate(args):
# run your apps.validate code with shell arguments args here
pass
if __name__ == '__main__':
pr = cProfile.Profile()
pr.enable()
run_validate(*sys.argv)
pr.disable()
pr.print_stats()
then just run the original script: python -m apps.validate -i input.mp4

How can I write python code in a shell script such that it imports a module and runs a test on it?

How can I run python code in a shell script that imports a python module and tests that specific module?
For example:
from _some_path_ import File_We_Want_To_Test
if(File_We_Want_To_Test.Function_We_Want_To_Test(a, b) == True):
print("great!")
else:
print("bad!")

You can use Python in your shell like this:
python -c command
Where command is a small Python program (a quoted string).
Example:
python -c "import my_module; my_module.function()"
See the page Command line and environment

run program in Python shell

I have a demo file: test.py.
In the Windows Console I can run the file with: C:\>test.py
How can I execute the file in the Python Shell instead?

Use execfile for Python 2:
>>> execfile('C:\\test.py')
Use exec for Python 3
>>> exec(open("C:\\test.py").read())

If you're wanting to run the script and end at a prompt (so you can inspect variables, etc), then use:
python -i test.py
That will run the script and then drop you into a Python interpreter.

It depends on what is in test.py. The following is an appropriate structure:
# suppose this is your 'test.py' file
def main():
"""This function runs the core of your program"""
print("running main")
if __name__ == "__main__":
# if you call this script from the command line (the shell) it will
# run the 'main' function
main()
If you keep this structure, you can run it like this in the command line (assume that $ is your command-line prompt):
$ python test.py
$ # it will print "running main"
If you want to run it from the Python shell, then you simply do the following:
>>> import test
>>> test.main() # this calls the main part of your program
There is no necessity to use the subprocess module if you are already using Python. Instead, try to structure your Python files in such a way that they can be run both from the command line and the Python interpreter.

For newer version of python:
exec(open(filename).read())

If you want to avoid writing all of this everytime, you can define a function :
def run(filename):
exec(open(filename).read())
and then call it
run('filename.py')

From the same folder, you can do:
import test

Why different behaviors between running Python module as an executable vs. Python [filename]

So, I created a simple python module, test.py
import commands
def main():
cmd = 'ls -l'
(status, output) = commands.getstatusoutput(cmd)
print status, output
if __name__ == '__main__':
main()
When I ran it using "Python test.py", I got the result that I expected. But when I ran it as an executable (yes, it has the 'x' permission), the program didn't respond at all and I had to Ctrl+C to quit it. Why is that? Shouldn't both ways give the same result?

Add a hash-bang line to the top:
#!/usr/bin/env python
import commands
...
This tells your system what interpreter to use to execute the script. Without it it doesn't know if it's a shell script, Perl script, Python script, what.

You need the hashbang to be the first line of your script, referencing the path of the Python interpreter. Otherwise, all the OS knows is that you're trying to execute a script, and it has no idea how to go about doing that.