Please explain the output?
# maindir/config/config.py
foo = 0
# maindir/worker/worker.py
import maindir.config.config
def print_foo():
print maindir.config.config.foo
# maindir/main.py
import config.config
import worker.worker
if __name__ == '__main__':
print config.config.foo
config.config.foo = 5
print config.config.foo
worker.worker.print_foo()
OUTPUT:
$ env PYTHONPATH=. python maindir/main.py
0
5
0 # was expecting 5 ...
On the other hand if we switch to import config.config in worker.py, then there is no surprise:
# maindir/worker/worker.py
import config.config
def print_foo():
print config.config.foo
OUTPUT:
$ env PYTHONPATH=./maindir python maindir/main.py
0
5
5
Related
This question already has answers here:
Python coverage.py exclude_lines
(2 answers)
is there a python-version specific "#pragma nocover" available for python coverage tool?
(4 answers)
Closed last year.
I wrote the following code.
https://gitlab.com/ksaito11/click-test
$ cat commands/cmd.py
import click
from commands.hello import hello
def print_version(ctx, param, value):
if not value or ctx.resilient_parsing:
return
click.echo('Version 1.0')
ctx.exit()
#click.group()
#click.option('--opt1')
#click.option('--version', is_flag=True, callback=print_version,
expose_value=False, is_eager=True)
#click.pass_context
def cmd(ctx, **kwargs):
ctx.obj = kwargs
def main():
cmd.add_command(hello)
cmd(auto_envvar_prefix='HELLOCLI')
if __name__ == '__main__':
main()
$ cat commands/hello.py
import click
#click.command()
def hello():
click.echo('Hello World!')
The code works correctly.
$ export PYTHONPATH=.
$ python commands/cmd.py
Usage: cmd.py [OPTIONS] COMMAND [ARGS]...
Options:
--opt1 TEXT
--version
--help Show this message and exit.
Commands:
hello
$ python commands/cmd.py --version
Version 1.0
$ python commands/cmd.py hello
Hello World!
I wrote the following test case.
$ cat tests/test_cmd.py
from click.testing import CliRunner
import click
import pytest
from commands.cmd import cmd, main
from commands.hello import hello
def test_version():
runner = CliRunner()
result = runner.invoke(cmd, ["--version"])
assert result.exit_code == 0
def test_help():
runner = CliRunner()
result = runner.invoke(cmd)
assert result.exit_code == 0
def test_hello():
runner = CliRunner()
result = runner.invoke(hello)
assert result.exit_code == 0
I measured the coverage with the following command.
$ pytest --cov-branch --cov=commands
================================================================ test session starts ================================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/ksaito/ghq/gitlab.com/ksaito11/click-test
plugins: cov-3.0.0
collected 3 items
tests/test_cmd.py ... [100%]
----------- coverage: platform linux, python 3.9.9-final-0 -----------
Name Stmts Miss Branch BrPart Cover
--------------------------------------------------------
commands/__init__.py 0 0 0 0 100%
commands/cmd.py 18 5 4 2 68%
commands/hello.py 4 0 0 0 100%
--------------------------------------------------------
TOTAL 22 5 4 2 73%
================================================================= 3 passed in 0.15s =================================================================
I didn't know how to write the code to test the part below and couldn't get 100% coverage.
def cmd(ctx, **kwargs):
ctx.obj = kwargs
def main():
cmd.add_command(hello)
cmd(auto_envvar_prefix='HELLOCLI')
The code below may not be needed when using "# click.group", but I couldn't determine.
def print_version(ctx, param, value):
if not value or ctx.resilient_parsing:
return
Please give me advice.
By adding the following settings, code that does not need to be included in coverage is excluded.
$ cat .coveragerc
[run]
branch = True
[report]
exclude_lines =
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:
def main
ctx.obj = kwargs
I deleted the code below because I thought it was unnecessary.
if not value or ctx.resilient_parsing:
return
The coverage is now 100%.
$ pytest --cov-branch --cov=commands
================================================================ test session starts ================================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/ksaito/ghq/gitlab.com/ksaito11/click-test
plugins: cov-3.0.0
collected 3 items
tests/test_cmd.py ... [100%]
----------- coverage: platform linux, python 3.9.9-final-0 -----------
Name Stmts Miss Branch BrPart Cover
--------------------------------------------------------
commands/__init__.py 0 0 0 0 100%
commands/cmd.py 10 0 0 0 100%
commands/hello.py 4 0 0 0 100%
--------------------------------------------------------
TOTAL 14 0 0 0 100%
================================================================= 3 passed in 0.22s =================================================================
I have this simple master/slave scripts, using locustio==0.13.5. This is the master:
#!/usr/bin/env python3
import logging
import argparse
import os
import sys
import time
import urllib3
import locust
import utils
class TestSomething(locust.TaskSet):
#locust.task(1)
def get_hosts_small(self):
print(self.locust.message)
return self.client.get(url='http://localhost', verify=False)
class TheSomething(locust.HttpLocust):
task_set = TestSomething
wait_time = locust.constant(0)
urllib3.disable_warnings()
logging.basicConfig(level=logging.DEBUG)
options = argparse.Namespace()
options.host = "http://localhost"
options.num_clients = 1
options.hatch_rate = options.num_clients
options.num_requests = 10
options.stop_timeout = 1
options.step_load = False
options.reset_stats = False
options.test_duration = 3
options.master_host = 'localhost'
options.master_port = 5557
options.master_bind_host = '*'
options.master_bind_port = 5557
options.heartbeat_liveness = 3
options.heartbeat_interval = 1
options.expect_slaves = 1
test_set = TheSomething
test_set.message = 'Hello'
locust_runner = locust.runners.MasterLocustRunner([test_set], options)
while len(locust_runner.clients.ready) < options.expect_slaves:
logging.info("Waiting for slaves to be ready, %s of %s connected", len(locust_runner.clients.ready), options.expect_slaves)
time.sleep(1)
locust_runner.start_hatching(locust_count=options.num_clients, hatch_rate=options.hatch_rate)
time.sleep(options.test_duration)
locust_runner.quit()
locusts.events.quitting.fire(reverse=True)
print(locust_runner.stats) # actually using custom function to format results
and this is the slave:
#!/usr/bin/env python3
import logging
import argparse
import os
import sys
import time
import locust
class TestSomething(locust.TaskSet):
#locust.task(1)
def get_hosts_small(self):
print(self.locust.message)
return self.client.get(url='http://localhost', verify=False)
class TheSomething(locust.HttpLocust):
task_set = TestSomething
wait_time = locust.constant(0)
logging.basicConfig(level=logging.DEBUG)
options = argparse.Namespace()
options.host = "http://localhost"
options.num_clients = 1
options.hatch_rate = options.num_clients
options.num_requests = 10
options.stop_timeout = 1
options.step_load = False
options.reset_stats = False
options.test_duration = 3
options.master_host = 'localhost'
options.master_port = 5557
options.master_bind_host = '*'
options.master_bind_port = 5557
options.heartbeat_liveness = 3
options.heartbeat_interval = 1
test_set = TheSomething
test_set.message = 'Hello'
locust_runner = locust.runners.SlaveLocustRunner([test_set], options)
locust_runner.worker()
When I start master and slave, I can see how master waits for a slave to come up, then how slave is executing the test and I see report printed by master before it finished. But slave does not finishes - it hangs running, doing nothing (I assume).
I would like slave to either exit or to restart and attempt to connect to master again in case I just rerun the master script. Does anybody have any idea on how to do that please?
I usually just set any parameters as environment variables and read them from the script (os.environ['MY_ENV_VAR'])
If you're running the slaves on the same server that should be easy (just run export MY_ENV_VAR=Hello before starting the processes), if you are running slaves on different machines it would be a little more complicated but check out locust-swarm that does the work for you (https://github.com/SvenskaSpel/locust-swarm)
As for the "do stuff after the test" there is a "quitting" event that you can subscribe to:
https://docs.locust.io/en/0.14.5/api.html#available-hooks
Or, for the upcoming 1.0 version:
https://docs.locust.io/en/latest/api.html#locust.event.Events.quitting
Given an object produced during importing the code, produce the set of imports that are needed to execute that object creation code.
Case 1:
some_obj = module.submodule.Class(42)
get_imports for_object(some_obj)
>>> "import module.submodule"
Case 2 (Sometimes the root module does not import submodules automatically (e.g. Airflow operators)):
some_obj = submodule.Class(42)
get_imports for_object(some_obj)
>>> "from module import submodule"
Case 3 (stretch goal):
some_obj = submodule.Class(sub2.Class2(42))
get_imports for_object(some_obj)
>>> ["from module import submodule", "from module2 import sub2"]
The goal is to produce import lines such that prepending them to object instantiation code will make the instantiation work.
This'll do:
def get_object_imports(obj, sub_obj_class_name=None):
sub_obj_modules = []
if sub_obj_class_name is not None:
for _, attribute_value in obj.__dict__.items():
value_str = str(getattr(attribute_value,'__class__',''))
if ('.' + sub_obj_class_name) in value_str:
sub_obj_modules.append(attribute_value.__module__)
if sub_obj_modules != []:
sub_module_imports = [('import ' + sub_obj_module) for sub_obj_module
in sub_obj_modules]
return ['import ' + obj.__module__] + sub_module_imports
else:
return 'import ' + obj.__module__
Cases (1) & (2) are equivalent, in that running either imports the same module. Note that with above, objects with same class names but different module sources will be included.
Demo:
from module import class1
from other_module import submodule
obj1 = class1()
obj2 = obj1(submodule.class2())
print(get_object_imports(obj2, 'class2'))
# ['import module', 'import other_module.submodule']
I am trying to run a simple script named test using python test in the terminal in order to call two other scripts.
This is the script I am running
#!/usr/bin/env python
import sys
import rospy
import subprocess
import time
from geometry_msgs.msg import Twist
from std_msgs.msg import Empty
subprocess.call("./takeoff_sim")
time.sleep(2)
subprocess.call("./forward_sim" + sys.argv[5.0] + sys.argv[2.0])
But I am getting the following error :
Traceback (most recent call last):
File "test", line 15, in <module>
subprocess.call(["./forward_sim"] + sys.argv[5.0] + sys.argv[2.0])
TypeError: list indices must be integers, not float
In this case both arguments need to a float thus, I cant just change them to int 0.5 should be the first argument and 2.0 the second one. Here is the forward_sim script:
#!/usr/bin/env python
import sys
import rospy
from geometry_msgs.msg import Twist
def commander(speed, time):
movement_publisher = rospy.Publisher('cmd_vel', Twist , queue_size=10)
rospy.init_node("bebop_commander") #Use log_level=rospy.DEBUG to see debug messages
rate = rospy.Rate(20) # 20hz
counter = 0.0
movement_cmd = Twist()
while not rospy.is_shutdown():
movement_cmd.linear.x = speed
movement_cmd.linear.y = 0
movement_cmd.linear.z = 0
movement_cmd.angular.x = 0
movement_cmd.angular.y = 0
movement_cmd.angular.z = 0
rospy.logdebug("Publishing")
counter += 0.05
movement_publisher.publish(movement_cmd)
rate.sleep()
if counter >= time:
break
if __name__ == '__main__':
speed = float(sys.argv[0])
time = float(sys.argv[1])
rospy.logdebug("Adelante") # Use rospy.logdebug() for debug messages.
print("Adelante")
if speed > 0:
rospy.logdebug("Velocidad = %s m/s", speed)
print("Velocidad =",speed," m/s",)
else:
raise ValueError("Falta parametro de velocidad o el valor es incorrecto")
if time > 0 :
rospy.logdebug("Tiempo = %s s", time)
print("Tiempo =" ,time, " s")
else:
raise ValueError("Falta parametro de tiempo o el valor es incorrecto")
try:
commander(speed, time)
except rospy.ROSInterruptException:
pass
sys.argv is only for receiving arguments. To pass values as arguments to another script, you need only convert them to strings and add them directly to the command line.
subprocess.call(["./forward_sim", str(5.0), str(2.0)])
When the other script receives them, it needs to convert each string back to a float:
# This is forward_sim
arg1 = float(sys.argv[1])
arg2 = float(sys.argv[2])
Note that sys.argv[0] is the name of the script itself, not the first argument.
I would like to find a way where the execution of Python scripts automatically write the result of the expressions in the top level, as is done in interactive mode.
For instance, if I have this script.py:
abs(3)
for x in [1,2,3]:
print abs(x)
abs(-4)
print abs(5)
and execute python script.py, I will get
1
2
3
5
but I would rather have
3
1
2
3
4
5
which is what one would get executing it interactively (modulo prompts).
More or less, I would like to achieve the contrary of Disable automatic printing in Python interactive session . It seems that the code module could help me, but I got no no success with it.
Well, I'm not seriously proposing using something like this, but you could (ab)use ast processing:
# -*- coding: utf-8 -*-
import ast
import argparse
_parser = argparse.ArgumentParser()
_parser.add_argument('file')
class ExpressionPrinter(ast.NodeTransformer):
visit_ClassDef = visit_FunctionDef = lambda self, node: node
def visit_Expr(self, node):
node = ast.copy_location(
ast.Expr(
ast.Call(ast.Name('print', ast.Load()),
[node.value], [], None, None)
),
node
)
ast.fix_missing_locations(node)
return node
def main(args):
with open(args.file) as source:
tree = ast.parse(source.read(), args.file, mode='exec')
new_tree = ExpressionPrinter().visit(tree)
exec(compile(new_tree, args.file, mode='exec'))
if __name__ == '__main__':
main(_parser.parse_args())
Output for your example script.py:
% python2 printer.py test2.py
3
1
2
3
4
5