Code
import sys
def main()
print(sys.argv)
Version - 3.3
File name Pytest.py
running the file with syntax pytest.py aaa bbb ccc
But it didn't print anything and not giving any error also
You never call main().
Python has no special main function that is run automatically, so instead, you can place the code that you want to run when the file is called from the command line into a special if block:
import sys
def main():
print(sys.argv)
if __name__ == '__main__':
main()
To elaborate #Blender's answer: Python functions does not compile like in C. Functions are statements - they are being executed when the control encounters them, and it begins at the first line of the file.
The following code is perfectly legal:
# get b somehow
if b:
def foo(): return 1
else:
def foo(): return 2
print(foo())
Related
I wrote a package that is using multiprocessing.Pool inside one of its functions.
Due to this reason, it is mandatory (as specified in here under "Safe importing of main module") that the outermost calling function can be imported safely e.g. without starting a new process. This is usually achieved using the if __name__ == "__main__": statement as explicitly explained at the link above.
My understanding (but please correct me if I'm wrong) is that multiprocessing imports the outermost calling module. So, if this is not "import-safe", this will start a new process that will import again the outermost module and so on recursively, until everything crashes.
If the outermost module is not "import-safe" when the main function is launched it usually hangs without printing any warning, error, message, anything.
Since using if __name__ == "__main__": is not usually mandatory and the user is usually not always aware of all the modules used inside a package, I would like to check at the beginning of my function if the user complied with this requirement and, if not, raise a warning/error.
Is this possible? How can I do this?
To show this with an example, consider the following example.
Let's say I developed my_module.py and I share it online/in my company.
# my_module.py
from multiprocessing import Pool
def f(x):
return x*x
def my_function(x_max):
with Pool(5) as p:
print(p.map(f, range(x_max)))
If a user (not me) writes his own script as:
# Script_of_a_good_user.py
from my_module import my_function
if __name__ == '__main__':
my_function(10)
all is good and the output is printed as expected.
However, if a careless user writes his script as:
# Script_of_a_careless_user.py
from my_module import my_function
my_function(10)
then the process hangs, no output is produces, but no error message or warning is issued to the user.
Is there a way inside my_function, BEFORE opening Pool, to check if the user used the if __name__ == '__main__': condition in its script and, if not, raise an error saying it should do it?
NOTE: I think this behavior is only a problem on Windows machines where fork() is not available, as explained here.
You can use the traceback module to inspect the stack and find the information you're looking for. Parse the top frame, and look for the main shield in the code.
I assume this will fail when you're working with a .pyc file and don't have access to the source code, but I assume developers will test their code in the regular fashion first before doing any kind of packaging, so I think it's safe to assume your error message will get printed when needed.
Version with verbose messages:
import traceback
import re
def called_from_main_shield():
print("Calling introspect")
tb = traceback.extract_stack()
print(traceback.format_stack())
print(f"line={tb[0].line} lineno={tb[0].lineno} file={tb[0].filename}")
try:
with open(tb[0].filename, mode="rt") as f:
found_main_shield = False
for i, line in enumerate(f):
if re.search(r"__name__.*['\"]__main__['\"]", line):
found_main_shield = True
if i == tb[0].lineno:
print(f"found_main_shield={found_main_shield}")
return found_main_shield
except:
print("Coulnd't inspect stack, let's pretend the code is OK...")
return True
print(called_from_main_shield())
if __name__ == "__main__":
print(called_from_main_shield())
In the output, we see that the first called to called_from_main_shield returns False, while the second returns True:
$ python3 introspect.py
Calling introspect
[' File "introspect.py", line 24, in <module>\n print(called_from_main_shield())\n', ' File "introspect.py", lin
e 7, in called_from_main_shield\n print(traceback.format_stack())\n']
line=print(called_from_main_shield()) lineno=24 file=introspect.py
found_main_shield=False
False
Calling introspect
[' File "introspect.py", line 27, in <module>\n print(called_from_main_shield())\n', ' File "introspect.py", lin
e 7, in called_from_main_shield\n print(traceback.format_stack())\n']
line=print(called_from_main_shield()) lineno=27 file=introspect.py
found_main_shield=True
True
More concise version:
def called_from_main_shield():
tb = traceback.extract_stack()
try:
with open(tb[0].filename, mode="rt") as f:
found_main_shield = False
for i, line in enumerate(f):
if re.search(r"__name__.*['\"]__main__['\"]", line):
found_main_shield = True
if i == tb[0].lineno:
return found_main_shield
except:
return True
Now, it's not super elegant to use re.search() like I did, but it should be reliable enough. Warning: since I defined this function in my main script, I had to make sure that line didn't match itself, which is why I used ['\"] to match the quotes instead of using a simpler RE like __name__.*__main__. Whatever you chose, just make sure it's flexible enough to match all legal variants of that code, which is what I aimed for.
I think the best you can do is to try execute the code and provide a hint if it fails. Something like this:
# my_module.py
import sys # Use sys.stderr to print to the error stream.
from multiprocessing import Pool
def f(x):
return x*x
def my_function(x_max):
try:
with Pool(5) as p:
print(p.map(f, range(x_max)))
except RuntimeError as e:
print("Whoops! Did you perhaps forget to put the code in `if __name__ == '__main__'`?", file=sys.stderr)
raise e
This is of course not a 100% solution, as there might be several other reasons the code throws a RuntimeError.
If it doesn't raise a RuntimeError, an ugly solution would be to explicitly force the user to pass in the name of the module.
# my_module.py
from multiprocessing import Pool
def f(x):
return x*x
def my_function(x_max, module):
"""`module` must be set to `__name__`, for example `my_function(10, __name__)`"""
if module == '__main__':
with Pool(5) as p:
print(p.map(f, range(x_max)))
else:
raise Exception("This can only be called from the main module.")
And call it as:
# Script_of_a_careless_user.py
from my_module import my_function
my_function(10, __name__)
This makes it very explicit to the user.
I have the following file: up.py
in this file:
def main(a_param, b_param, c_param):
// Code
if __name__ == '__main__':
exit(main())
I want to run this python file via the CMD, so I write this line:
python up.py False True False
But I get the next error:
TypeError: main() takes exactly 3 arguments (0 given)
This has nothing to do with CMD. Your main function expects three arguments, but you aren't passing any; you call it directly from your if __name__ == '__main__' block with just main().
Either get the arguments (eg from sys.argv) within that block and pass them to main, or remove the arguments from the function signature and get them within main.
You are trying to call your main function without arguments event though it requires 3 (a_param, b_param and c_param).
The command line parameters are stored in sys.argv. To call the main function with the first 3 command line parameters, you could this:
import sys
if __name__ == '__main__':
main(*sys.argv[1:4])
To clarify, * unpacks the argument list so main(*sys.argv[1:4]) is equivalent to main(sys.argv[1], sys.argv[2], sys.argv[3])
This code works for me
def main(a_param, b_param, c_param):
# Code
if __name__ == '__main__':
exit(main())
then:
$ python up.py False True False
I am trying to understand the usage of #main annotation in python.
With the below python program,
def cube(x):
return x * x * x
def run_tests():
printf("Should be 1:", cube(1))
printf("Should be 8:", cube(2))
printf("Should be 27:", cube(3))
#main
def main():
print("Starting")
run_tests()
print("Ending.")
I get the following error:
PS C:\Users\MOHET01\Desktop> python.exe -i .\cube.py
Traceback (most recent call last):
File ".\cube.py", line 9, in <module>
#main
NameError: name 'main' is not defined
>>>
Function that is imported from ucb is as shown below:
def main(fn):
"""Call fn with command line arguments. Used as a decorator.
The main decorator marks the function that starts a program. For example,
interact()
#main
def my_run_function():
# function body
Use this instead of the typical __name__ == "__main__" predicate.
"""
if inspect.stack()[1][0].f_locals['__name__'] == '__main__':
args = sys.argv[1:] # Discard the script name from command line
print(args)
print(*args)
print(fn)
fn(*args) # Call the main function
return fn
My question:
Despite i define function with intrinsic name main, Why do i see this error?
I should use this:
def main():
#Do something
if __name__ == "__main__":
#Here use the method that will be the main
main()
I hope this helps
The #main decorator is implemented in a file your course provides, but you have not imported it. The page you linked says to use
from ucb import main, interact
to import the ucb.py features in your program.
As for why the error says name 'main' is not defined, that's because the function definition doesn't actually finish defining anything until the decorators execute. The reuse of the name main for both the decorator and the decorated function is confusing; the main in #main is a different function from the main you're defining in def main(): .... The main in #main is defined to run the decorated function if the file is run as a script, while the main in def main(): ... is the function to be run.
I would strongly recommend not using anything like this decorator when you don't have to. The standard way to perform the task the decorator performs is to write
if __name__ == '__main__':
whatever_function_you_would_have_put_the_decorator_on()
or if you want to handle command line arguments like the decorator would,
if __name__ == '__main__':
import sys
whatever_function_you_would_have_put_the_decorator_on(*sys.argv[1:])
The decorator is an attempt to hide the issues of sys.argv and __name__ so you don't have to know about them, but it has a problem. If you try to write something like this:
#main
def hello():
print(hello_string)
hello_string = 'Hi there.'
you'll get a NameError, because hello_string won't be assigned until after the decorator runs. If you continue to write Python beyond this course, you'll find that using if __name__ == '__main__' is less bug-prone and more understandable to other programmers than using a decorator for this.
You are using the function before it is defined. In other words, you need to define the main function higher up (in the document) than where you use it as a decorator:
def main():
pass
#main
def somefunction():
pass
The #main notation means the main function is being used to "decorate", or modify, another function. There are various articles on python decorators:
http://simeonfranklin.com/blog/2012/jul/1/python-decorators-in-12-steps/
http://www.artima.com/weblogs/viewpost.jsp?thread=240808
http://www.jeffknupp.com/blog/2013/11/29/improve-your-python-decorators-explained/
You can only use a decorator on a different function. Example:
def foo(f):
def inner():
print("before")
f()
print("after")
return inner
#foo
def bar():
print("bar")
if __name__ == "__main__":
bar()
Output:
before
bar
after
For example let's say I have a file called myscript.py
This file contains the following code.
foo(var):
return var
How would I call the function foo with argument var on command line.
I know that I can go to the directory myscript.py is placed in and type.
>>> python myscript.py
Which will run myscript.py. The only problem is myscript.py doesn't automatically call foo when it is run.
I have tried using
if __name__ == "__main__":
foo( )
Which does not work for me. For some reason when I do that nothing happens. I get no error message and nothing is called.
You don't get any output because you don't generate any. Try calling print:
def foo(var):
print(var)
if __name__ == '__main__':
foo('Hello, world')
You have to use the sys module to pass arguments from the command line.
You can do this:
import sys
def foo(var):
return var
if __name__ == '__main__':
# arguments check
if len(sys.argv) != 2:
print "USAGE: %s <value>" % sys.argv[0]
sys.exit(1)
# get the agument so as to use it to the function call
arg = sys.argv[1]
# call the function using this argument
val = foo(arg)
# print the returned value
print val
Then you can run your python script by this command:
python myscript.py 3
giving as argument e.g. the number 3
Say I have a module with the following:
def main():
pass
if __name__ == "__main__":
main()
I want to write a unit test for the bottom half (I'd like to achieve 100% coverage). I discovered the runpy builtin module that performs the import/__name__-setting mechanism, but I can't figure out how to mock or otherwise check that the main() function is called.
This is what I've tried so far:
import runpy
import mock
#mock.patch('foobar.main')
def test_main(self, main):
runpy.run_module('foobar', run_name='__main__')
main.assert_called_once_with()
I will choose another alternative which is to exclude the if __name__ == '__main__' from the coverage report , of course you can do that only if you already have a test case for your main() function in your tests.
As for why I choose to exclude rather than writing a new test case for the whole script is because if as I stated you already have a test case for your main() function the fact that you add an other test case for the script (just for having a 100 % coverage) will be just a duplicated one.
For how to exclude the if __name__ == '__main__' you can write a coverage configuration file and add in the section report:
[report]
exclude_lines =
if __name__ == .__main__.:
More info about the coverage configuration file can be found here.
Hope this can help.
You can do this using the imp module rather than the import statement. The problem with the import statement is that the test for '__main__' runs as part of the import statement before you get a chance to assign to runpy.__name__.
For example, you could use imp.load_source() like so:
import imp
runpy = imp.load_source('__main__', '/path/to/runpy.py')
The first parameter is assigned to __name__ of the imported module.
Whoa, I'm a little late to the party, but I recently ran into this issue and I think I came up with a better solution, so here it is...
I was working on a module that contained a dozen or so scripts all ending with this exact copypasta:
if __name__ == '__main__':
if '--help' in sys.argv or '-h' in sys.argv:
print(__doc__)
else:
sys.exit(main())
Not horrible, sure, but not testable either. My solution was to write a new function in one of my modules:
def run_script(name, doc, main):
"""Act like a script if we were invoked like a script."""
if name == '__main__':
if '--help' in sys.argv or '-h' in sys.argv:
sys.stdout.write(doc)
else:
sys.exit(main())
and then place this gem at the end of each script file:
run_script(__name__, __doc__, main)
Technically, this function will be run unconditionally whether your script was imported as a module or ran as a script. This is ok however because the function doesn't actually do anything unless the script is being ran as a script. So code coverage sees the function runs and says "yes, 100% code coverage!" Meanwhile, I wrote three tests to cover the function itself:
#patch('mymodule.utils.sys')
def test_run_script_as_import(self, sysMock):
"""The run_script() func is a NOP when name != __main__."""
mainMock = Mock()
sysMock.argv = []
run_script('some_module', 'docdocdoc', mainMock)
self.assertEqual(mainMock.mock_calls, [])
self.assertEqual(sysMock.exit.mock_calls, [])
self.assertEqual(sysMock.stdout.write.mock_calls, [])
#patch('mymodule.utils.sys')
def test_run_script_as_script(self, sysMock):
"""Invoke main() when run as a script."""
mainMock = Mock()
sysMock.argv = []
run_script('__main__', 'docdocdoc', mainMock)
mainMock.assert_called_once_with()
sysMock.exit.assert_called_once_with(mainMock())
self.assertEqual(sysMock.stdout.write.mock_calls, [])
#patch('mymodule.utils.sys')
def test_run_script_with_help(self, sysMock):
"""Print help when the user asks for help."""
mainMock = Mock()
for h in ('-h', '--help'):
sysMock.argv = [h]
run_script('__main__', h*5, mainMock)
self.assertEqual(mainMock.mock_calls, [])
self.assertEqual(sysMock.exit.mock_calls, [])
sysMock.stdout.write.assert_called_with(h*5)
Blam! Now you can write a testable main(), invoke it as a script, have 100% test coverage, and not need to ignore any code in your coverage report.
Python 3 solution:
import os
from importlib.machinery import SourceFileLoader
from importlib.util import spec_from_loader, module_from_spec
from importlib import reload
from unittest import TestCase
from unittest.mock import MagicMock, patch
class TestIfNameEqMain(TestCase):
def test_name_eq_main(self):
loader = SourceFileLoader('__main__',
os.path.join(os.path.dirname(os.path.dirname(__file__)),
'__main__.py'))
with self.assertRaises(SystemExit) as e:
loader.exec_module(module_from_spec(spec_from_loader(loader.name, loader)))
Using the alternative solution of defining your own little function:
# module.py
def main():
if __name__ == '__main__':
return 'sweet'
return 'child of mine'
You can test with:
# Override the `__name__` value in your module to '__main__'
with patch('module_name.__name__', '__main__'):
import module_name
self.assertEqual(module_name.main(), 'sweet')
with patch('module_name.__name__', 'anything else'):
reload(module_name)
del module_name
import module_name
self.assertEqual(module_name.main(), 'child of mine')
I did not want to exclude the lines in question, so based on this explanation of a solution, I implemented a simplified version of the alternate answer given here...
I wrapped if __name__ == "__main__": in a function to make it easily testable, and then called that function to retain logic:
# myapp.module.py
def main():
pass
def init():
if __name__ == "__main__":
main()
init()
I mocked the __name__ using unittest.mock to get at the lines in question:
from unittest.mock import patch, MagicMock
from myapp import module
def test_name_equals_main():
# Arrange
with patch.object(module, "main", MagicMock()) as mock_main:
with patch.object(module, "__name__", "__main__"):
# Act
module.init()
# Assert
mock_main.assert_called_once()
If you are sending arguments into the mocked function, like so,
if __name__ == "__main__":
main(main_args)
then you can use assert_called_once_with() for an even better test:
expected_args = ["expected_arg_1", "expected_arg_2"]
mock_main.assert_called_once_with(expected_args)
If desired, you can also add a return_value to the MagicMock() like so:
with patch.object(module, "main", MagicMock(return_value='foo')) as mock_main:
One approach is to run the modules as scripts (e.g. os.system(...)) and compare their stdout and stderr output to expected values.
I found this solution helpful. Works well if you use a function to keep all your script code.
The code will be handled as one code line. It doesn't matter if the entire line was executed for coverage counter (though this is not what you would actually actually expect by 100% coverage)
The trick is also accepted pylint. ;-)
if __name__ == '__main__': \
main()
If it's just to get the 100% and there is nothing "real" to test there, it is easier to ignore that line.
If you are using the regular coverage lib, you can just add a simple comment, and the line will be ignored in the coverage report.
if __name__ == '__main__':
main() # pragma: no cover
https://coverage.readthedocs.io/en/coverage-4.3.3/excluding.html
Another comment by # Taylor Edmiston also mentions it
My solution is to use imp.load_source() and force an exception to be raised early in main() by not providing a required CLI argument, providing a malformed argument, setting paths in such a way that a required file is not found, etc.
import imp
import os
import sys
def mainCond(testObj, srcFilePath, expectedExcType=SystemExit, cliArgsStr=''):
sys.argv = [os.path.basename(srcFilePath)] + (
[] if len(cliArgsStr) == 0 else cliArgsStr.split(' '))
testObj.assertRaises(expectedExcType, imp.load_source, '__main__', srcFilePath)
Then in your test class you can use this function like this:
def testMain(self):
mainCond(self, 'path/to/main.py', cliArgsStr='-d FailingArg')
To import your "main" code in pytest in order to test it you can import main module like other functions thanks to native importlib package :
def test_main():
import importlib
loader = importlib.machinery.SourceFileLoader("__main__", "src/glue_jobs/move_data_with_resource_partitionning.py")
runpy_main = loader.load_module()
assert runpy_main()