Hi I want to see what is being passed to a rule that is defined in a rules.bzl file. So I would like to add just a logging statement inside of either the BUILD file or the rules.bzl file just to see what parameter was passed. I realize there is a whole --execution_log_binary_file and there is a good deal of stuff to read up on, but I am definitely looking to do something just once and quick and dirty. Do I import logging module? Is it something else? Thanks
Use the builtin Starlark print() function in BUILD or .bzl files to dump values Bazel's console.
bazel query --output=build //my/package:all will print out the definitions of all the targets created by macros called from my/package/BUILD by any .bzl file. This includes all the attributes passed to to various rules to create those targets.
Related
Is there a way to perform a python import which is not atomic?
For instance, I have a file as follows:
# Filename: a.py
myvariable = 1
mylist = [1, 2, 3]
raise ImportError
donotimportthis = 5
I then have a separate file which does the following:
import a
a.myvariable == 1 # This is okay as it imported it
a.donotimportthis # <-- raise an exception as this is not imported.
I have a file which contains some python code, this follows the format of:
...variables...
import X
I do not have X installed nor do I want it however I do want the variables.
Note: This file is autogenerated not by me but by a tool whose version is frozen.
Two choices, in descending order of preference:
Change the autogeneration process. Instead of invoking proprietary_autogen_process, invoke custom_autogen_wrapper. This wrapper in turn first invokes the proprietary third-party tool, and then modifies the produced module source code by searching for the code that imports module X, and deletes everything after it.
This is relatively straightforward. You just need to take some care to not introduce false positives or false negatives by performing too loose (or too strict) matching of the import code. Ideally you’d use an AST rewriter but that’s probably overkill; a regular expression search for import X might work, although it will yield wrong results if this text appears inside a comment, string literal or inside a method which isn’t executed.
Generate a stub module X in a location where it will be found by the autogenerated module when importing the latter. I don’t recommend this because it’s tedious: You probably can’t just generate an empty module, since the autogenerated module will want to use X. You need to generate meaningful method stubs.
You can do specific imports with
from a import myvariable
EDIT: The above won't work if anything that is flat in the file raises an error. If you have no way to edit the imported file then I don't know if there is a (resonable) solution to this. Sorry didn't realise.
(an unreasonable solution would be to read in the file as text, slice it, and then run eval on it).
Or, as mentioned in the comments, put the stuff you don't want under
if __name__=="__main__":
<here>
Then it will only be invoked if you run the file directly.
What you can do is removing the donotimportthis variable at the end of the module, as follows: del donotimportthis. I hope it helps
I've the read pytest documentation. Section 7.4.3 gives instructions for registering markers. I have followed the instructions exactly, but it doesn't seem to have worked for me.
I'm using Python 2.7.2 and pytest 2.5.1.
I have a pytest.ini file at the root of my project. Here is the entire contents of that file:
[pytest]
python_files=*.py
python_classes=Check
python_functions=test
rsyncdirs = . logs
rsyncignore = docs archive third_party .git procs
markers =
mammoth: mark a test as part of the Mammoth regression suite
A little background to give context: The folks that created the automation framework I am working on no longer work for the company. They created a custom plugin that extended the functionality of the default pytest.mark. From what I understand, the only thing the custom plugin does is make it so that I can add marks to a test like this:
#pytest.marks(CompeteMarks.MAMMOTH, CompeteMarks.QUICK_TEST_A, CompeteMarks.PROD_BVT)
def my_test(self):
instead of like this:
#pytest.mark.mammoth
#pytest.mark.quick_test_a
#pytest.mark.prod_bvt
def my_test(self):
The custom plugin code remains present in the code base. I do not know if that has any negative effect on trying to register a mark, but thought it was worth mentioning if someone knows otherwise.
The problem I'm having is when I execute the following command on a command-line, I do NOT see my mammoth mark listed among the other registered marks.
py.test --markers
The output returned after running the above command is this:
#pytest.mark.skipif(condition): skip the given test function if eval(condition) results in a True value. Evaluation happens within the module global context. Example: skipif('sys.platform == "win32"') skips the test if we are on the win32 platform. see http://pytest.org/latest/skipping.html
#pytest.mark.xfail(condition, reason=None, run=True): mark the the test function as an expected failure if eval(condition) has a True value. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. See http://pytest.org/latest/skipping.html
#pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: #parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see http://pytest.org/latest/parametrize.html for more info and examples.
#pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see http://pytest.org/latest/fixture.html#usefixtures
#pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
#pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
What am I doing wrong and how can I get my mark registered?
One more piece of info, I have applied the mammoth mark to a single test (shown below) when I ran the py.test --markers command:
#pytest.mark.mammoth
def my_test(self):
If I understand your comments correctly the project layout is the following:
~/customersites/
~/customersites/automation/
~/customersites/automation/pytest.ini
Then invoking py.test as follows:
~/customersites$ py.test --markers
will make py.test look for a configuration file in ~/customersites/ and subsequently all the parents: ~/, /home/, /. In this case this will not make it find pytest.ini.
However when you invoke it with one or more arguments, py.test will try to interpret each argument as a file or directory and start looking for a configuration file from that directory and it's parents. It then iterates through all arguments in order until it found the first configuration file.
So with the above directory layout invoking py.test as follows will make it find pytest.ini and show the markers registered in it:
~/customersites$ py.test automation --markers
as now py.test will first look in ~/customersites/automation/ for a configuration file before going up the directory tree and looking in ~/customersites/. But since it finds one in ~/customersites/automation/pytest.ini it stops there and uses that.
Have you tried here?
From the docs:
API reference for mark related objects
class MarkGenerator[source]
Factory for MarkDecorator objects - exposed as a pytest.mark singleton
instance.
Example:
import py
#pytest.mark.slowtest
def test_function():
pass
will set a slowtest MarkInfo object on the test_function object.
class MarkDecorator(name, args=None, kwargs=None)[source]
A decorator for test functions and test classes. When applied it will
create MarkInfo objects which may be retrieved by hooks as item keywords.
MarkDecorator instances are often created like this:
mark1 = pytest.mark.NAME # simple MarkDecorator
mark2 = pytest.mark.NAME(name1=value) # parametrized MarkDecorator
and can then be applied as decorators to test functions:
#mark2
def test_function():
pass
I come from a PHP (as well as a bunch of other stuff) background and I am playing around with Python. In PHP when I want to include another file I just do include or require and everything in that file is included.
But it seems the recommended way to do stuff in python is from file import but that seems to be more for including libraries and stuff? How do you separate your code amongst several files? Is the only way to do it, to have a single file with a whole bunch of function calls and then import 15 other files?
Things are totally different between PHP and Python, and there are many reasons why.
But it seems the recommended way to do stuff in python is from file import but that seems to be more for including libraries and stuff?
Indeed, import statements are for importing objects from another module to current module. You can either import all the objects of the imported module to current module:
import foo
print foo.bar
or you can select what you want from that module:
from foo import bar
print bar
and even better, if you import a module twice, it will be only imported once:
>> import foo as foo1
>> import foo as foo2
>> foo1 is foo2
True
How do you separate your code amongst several files?
You have to think about your code... That's called software design, and here are a few rules:
you never write an algorithm at the module's level; instead make it a function, and call that function
you never instantiate an object at the module's level; you shall embed it in the function, and call that function
if you need an object in several different functions, create a class and encapsulate that object in that class, then use it in your functions bound to that class (so they now are called methods)
The only exception is when you want to launch a program from command line, you append:
if __name__ == "__main__":
at the end of the module. And my best advice would be to just call your first function afterwards:
if __name__ == "__main__":
main()
Is the only way to do it, to have a single file with a whole bunch of function calls and then import 15 other files?
It's not the only way to do it, but it's the best way to do it. You make all your algorithms into libraries of functions and objects, and then import exactly what you need in other libraries etc.. That's how you create a whole universe of reusable code and never have to reinvent the wheel! So forget about files, and think about modules that contains objects.
Finally, my best advice to you learning python is to unlearn every habit and usage you had while coding PHP, and learn those things again, differently. In the end, that can only make you a better software engineer.
I guess I understand what you are tring to say and to do.
Here is the random include example from PHP:
File #1 - vars.php
<?php
$color = 'green';
$fruit = 'apple';
?>
File #2 - main.php
<?php
echo "A $color $fruit"; // A
include 'vars.php';
echo "A $color $fruit"; // A green apple
?>
The fist echo command will print just "A" string, for it does not have any values assigned to the vars. The next echo will print a full string thanks to your include before it.
Python's "import", however, imports a module or it's part, so you could work with it in your current module.
Here is a python example:
File 1 - echo.py
apple = 'apple'
color = 'green'
File 2 - main.py
import echo
def func():
print "A "+echo.color+" "+echo.fruit
if __name__ == '__main__':
func()
In other words - you import some functionality from one module and then use it in your other module.
The example above is not really good from programming standarts or best practises, but I think it gives you some understanding.
Interesting question. As you know, in PHP, you can separate your code by using include, which literally takes all the code in the included file and puts it wherever you called include. This is convenient for writing web applications because you can easily divide a page into parts (such as header, navigation, footer, etc).
Python, on the other hand, is used for way more than just web applications. To reuse code, you must rely on functions or good old object-oriented programming. PHP also has functions and object-oriented programming FYI.
You write functions and classes in a file and import it in another file. This lets you access the functions or use the classes you defined in the other file.
Lets say you have a function called foo in file file1.py. From file2.py, you can write
import file1. Then, call foo with file1.foo(). Alternatively, write from file1 import foo and then you can call foo with foo(). Note that the from lets you call foo directly. For more info, look at the python docs.
On a technical level, a Python import is very similar to a PHP require, as it will execute the imported file. But since Python isn't designed to ultimately generate an HTML file, the way you use it is very different.
Typically a Python file will on the module level not include much executable code at all, but definitions of functions and classes. You them import them and use them as a library.
Hence having things like header() and footer() makes no sense in Python. Those are just functions. Call them like that, and the result they generate will be ignored.
So how do you split up your Python code? Well, you split it up into functions and classes, which you put into different files, and then import.
There's an execfile() function which does something vaguely comparable with PHP's include, here, but it's almost certainly something you don't want to do. As others have said, it's just a different model and a different programming need in Python. Your code is going to go from function to function, and it doesn't really make a difference in which order you put them, as long as they're in an order where you define things before you use them. You're just not trying to end up with some kind of ordered document like you typically are with PHP, so the need isn't there.
I have a dictionary of addresses with their usernames and passwords listed that looks something like this:
address_dict = {'address1':{'username':'abc', 'password':'123'}, 'address2':{'username':'xyz', 'password':'456'}}
Is there a way to make this dictionary accessible for multiple scripts to read from and possibly write to? Like save it as seperate python file and import it or something?
Yes, you can do just that:
# module.py
address_dict = {'address1':{'username':'abc', 'password':'123'}, 'address2':{'username':'xyz', 'password':'456'}}
# main.py
import module
print(module.address_dict)
If you don't like the module. prefix, you could import the dictionary like so:
from module import address_dict
print(address_dict)
To access it and modify it at runtime, you can just define it in a module and then import it. But if you want your changes to be persistent (i.e. see the changed version next time you run the script) you need something else, like a database.
The simplest to use in this case would probably be the shelve module, which is based on pickle. You can also use pickle itself if you wish.
take a look at pickle :)
http://docs.python.org/2/library/pickle.html
You can use it to dump objects to files and also to read them back in with any other python script.
I have a situation where users can pick another method from another clases and use it in thier own class using the .im_func. i give an example below
import foo1
import foo2
foo1.ClassX.methodX = foo2.ClassX.methodX.im_func
Where methodX could be implemented differently in both modules.
When i instantiate the object say foo1.Class(), methodX from modulefoo2` is used.
My problem is how to save the changes made maybe as foo3.py to a new source code file.
saving it as new py could be a problem but you can easily use serialization for it (pickle module)
see: http://docs.python.org/library/pickle.html
The source code can be retrieved with inspect module. However, the problem with that is, that it's the original source code, not source code of dynamically modified object.
Have you considered using parser combined with the aforementioned inspect to do this? In this situation it might be better to simply go with text processing rather than attempting to use imported modules.
EDIT: An example of using the parser to print the file:
with open('foo1.py','r') as fh:
st = parser.suite(fh.read())
src1 = parser.st2list(st)
with open('foo2.py','r') as fh:
st = parser.suite(fh.read())
src2 = parser.st2list(st)
You'd have to then do some tricky programming to merge the methods from the source code and write it to a file. But then again I have the strange feeling I'm not quite understanding the question...