Im working with matplotlib and for a certain program I need to change the matplotlib parameters only in a context, not for all the document, so I neeed to use this context with lot of parameters,
with plt.rc_context(rc={'figure.figsize':(10,9),
'font.size':17,
....
'xtick.top':True,
'ytick.right':True,
'xtick.minor.visible':True,
'ytick.minor.visible':True }):
As there are lot of parameters in order to clean the code I have the intention of put all this code on a separated archive and later import it as a function,
def context():
with plt.rc_context( ... ):
But it cant be made in this way, gives an error because the function is not "closed", (SyntaxError: unexpected EOF while parsing).
My question is if there is some way to pass this "context" inside a function.
I think what you might be asking is how to create your own context manager that you can use with with (this is what this comment was suggesting). But actually there's an even simpler solution than that.
rc_context itself is already a context manager. If you want to make your own "version" of it that bakes in a specific context, you can do that with partial application using functools.partial:
from functools import partial
my_mpl_context = partial(plt.rc_context, rc={
# fill in your mpl settings here
})
And then you can use it like:
with my_mpl_context():
# make plots, etc...
To make your code work you need to use yield in the with block. And to make context a context manager you can decorate it with contextlib.contextmanager:
from contextlib import contextmanger
#contextmanager
def context():
with plt.rc_context( ... ):
yield
Related
I'm trying to debug Hy's use of bytecode. In particular, each time a module is imported, I want to see the path it was actually imported from, whether source or bytecode. Under the hood, Hy manages modules with importlib. It doesn't explicitly read or write bytecode; that's taken care of by importlib.machinery.SourceFileLoader. So it looks like what I want to do is monkey-patch Python's importing system to print the import path each time an import happens. How can I do that? I should be able to figure out how to do it for Hy once I understand how to do it for Python.
The easiest way that does not involve coding, is to start Python with two(!) verbose flags:
python -vv myscript.py
you'll get a lot of output, including all the import statements and all the files Python tries to access in order to load the module. In this example I have a simple python script that does import json:
lots of output!
[...]
# trying /tmp/json.cpython-310-x86_64-linux-gnu.so
# trying /tmp/json.abi3.so
# trying /tmp/json.so
# trying /tmp/json.py
# trying /tmp/json.pyc
# /usr/lib/python3.10/json/__pycache__/__init__.cpython-310.pyc matches /usr/lib/python3.10/json/__init__.py
# code object from '/usr/lib/python3.10/json/__pycache__/__init__.cpython-310.pyc'
[...]
Alternatively but more complex: you could change the import statement itself. For that, you can overwrite __import__, which is invoked by the import statement itself. This way you could print out all the files import actually opens.
Seems like a good option would be to dynamically patch importlib.machinery.SourceFileLoader(fullname, path) and importlib.machinery.SourcelessFileLoader(fullname, path) to each print or write to a variable (a) the calling method and (b) the argument passed to the function.
If all you need to do is:
I want to see the path it was actually imported from, whether source or bytecode
And you don't need the import to "work properly", perhaps you can do a modified version of something like this. For example, I quickly modified their sample code to get this, I have not tested it so it may not work exactly, but it should get you on the right track:
# custom class to be the mock return value
class MockSourceLoader:
# mock SourceFileLoader method always returns that the module was loaded from source and its path
def SourceFileLoader(fullname, path):
return {"load type": "source", "fullname": fullname, "path": path}
def check_how_imported(monkeypatch):
# Any arguments may be passed and mock_get() will always return our
# mocked object
def mock_get(*args, **kwargs):
return MockSourceLoader
# apply the monkeypatch
monkeypatch.setattr(importlib.machinery, SourceFileLoader, SourceFileLoader)
You would of course provide a similar mock for Sourceless file loading for SourcelessFileLoader
For reference:
https://docs.python.org/3/library/importlib.html#:~:text=importlib.machinery.SourceFileLoader(fullname%2C%20path)%C2%B6
https://docs.python.org/3/library/importlib.html#:~:text=importlib.machinery.SourcelessFileLoader(fullname%2C%20path)
I have the following code that I'm trying to test:
great_report.py
from retry import retry
#retry((ReportNotReadyException), tries=3, delay=10, backoff=3)
def get_link(self):
report_link = _get_report_link_from_3rd_party(params)
if report_link:
return report_link
else:
stats.count("report_not_ready", 1)
raise ReportNotReadyException
I've got my testing function which mocks _get_report_link_from_3rd_party which tests everything but I don't want this function to actually pause execution during when I run tests..
#mock.patch('repo.great_report._get_report_link_from_3rd_party', return_value=None)
test_get_link_raises_exception(self, mock_get_report_link):
self.assertRaises(ReportNotReadyException, get_link)
I tried mocking the retry parameters but am running into issues where get_link keeps retrying over and over which causes long build times instead of just raising the exception and continuing. How can I mock the parameters for the #retry call in my test?
As hinted here, an easy way to prevent the actual sleeping is by patching the time.sleep function. Here is the code that did that for me:
#patch('time.sleep', side_effect = lambda _: None)
There is no way to change decorators parameters after load the module. Decorators decorate the original function and change it at the module load time.
First I would like encourage you to change your design a little to make it more testable.
If you extract the body of get_link() method test the new method and trust retry decorator you will obtain your goal.
If you don't want add a new method to your class you can use a config module that store variables that you use when call retry decorator. After that you can use two different module for testing and production.
The last way is the hacking way where you replace retry.api.__retry_internal by a your version that invoke the original one by changing just the variables:
import unittest
from unittest.mock import *
from pd import get_link, ReportNotReadyException
import retry
orig_retry_internal = retry.api.__retry_internal
def _force_retry_params(new_tries=-1, new_delay=0, new_max_delay=None, new_backoff=1, new_jitter=0):
def my_retry_internals(f, exceptions, tries, delay, max_delay, backoff, jitter, logger):
# call original __retry_internal by new parameters
return orig_retry_internal(f, exceptions, tries=new_tries, delay=new_delay, max_delay=new_max_delay,
backoff=new_backoff, jitter=new_jitter, logger=logger)
return my_retry_internals
class MyTestCase(unittest.TestCase):
#patch("retry.api.__retry_internal", side_effect=_force_retry_params(new_tries=1))
def test_something(self, m_retry):
self.assertRaises(ReportNotReadyException, get_link, None)
IMHO you should use that hacking solution only if you are with the back on the wall and you have no chance to redesign you code to make it more testable. The internal function/class/method can change without notice and your test can be difficult to maintain in the future.
I have a third-party Python library that allows me to register a callback function that it called later.
While the code works okay with functions, when I tried to pass a method it fails, the callback is never called.
I have no control over the third party library (source code not available).
def old_callbackFunction(param, data):
print data
class MyClass():
def callbackFunction(self, param, data):
print data
myObj = MyClass()
# old_setCallback(myObj.callbackFunction, param = "x") # this would work
setCallback(myObj.callbackFunction, param = "x") # this is never called
Sorin actually figured this out himself, with help from my comment, but he indicated that he wanted me to post the original comment as an answer. I was reluctant to post this originally because I'm unsure of the precise behavior of the setCallback and callbackFunction code; use at your own risk and modify as reason dictates.
The best way to wrap a function is to use functools.partial:
from functools import partial
setCallback(partial(myObj.callbackFunction), param="x")
You may also use a lambda (but you'll lose style points with the pythonistas):
setCallback(lambda param, data: myObj.callbackFunction(param, data), param="x")
imagine you have an io heavy function like this:
def getMd5Sum(path):
with open(path) as f:
return md5(f.read()).hexdigest()
Do you think Python is flexible enough to allow code like this (notice the $):
def someGuiCallback(filebutton):
...
path = filebutton.getPath()
md5sum = $getMd5Sum()
showNotification("Md5Sum of file: %s" % md5sum)
...
To be executed something like this:
def someGuiCallback_1(filebutton):
...
path = filebutton.getPath()
Thread(target=someGuiCallback_2, args=(path,)).start()
def someGuiCallback_2(path):
md5sum = getMd5Sum(path)
glib.idle_add(someGuiCallback_3, md5sum)
def someGuiCallback_3(md5sum):
showNotification("Md5Sum of file: %s" % md5sum)
...
(glib.idle_add just pushes a function onto the queue of the main thread)
I've thought about using decoraters, but they don't allow me to access the 'content' of the function after the call. (the showNotification part)
I guess I could write a 'compiler' to change the code before execution, but it doesn't seam like the optimal solution.
Do you have any ideas, on how to do something like the above?
You can use import hooks to achieve this goal...
PEP 302 - New Import Hooks
PEP 369 - Post Import Hooks
... but I'd personally view it as a little bit nasty.
If you want to go down that route though, essentially what you'd be doing is this:
You add an import hook for an extension (eg ".thpy")
That import hook is then responsible for (essentially) passing some valid code as a result of the import.
That valid code is given arguments that effectively relate to the file you're importing.
That means your precompiler can perform whatever transformations you like to the source on the way in.
On the downside:
Whilst using import hooks in this way will work, it will surprise the life out of any maintainer or your code. (Bad idea IMO)
The way you do this relies upon imputil - which has been removed in python 3.0, which means your code written this way has a limited lifetime.
Personally I wouldn't go there, but if you do, there's an issue of the Python Magazine where doing this sort of thing is covered in some detail, and I'd advise getting a back issue of that to read up on it. (Written by Paul McGuire, April 2009 issue, probably available as PDF).
Specifically that uses imputil and pyparsing as it's example, but the principle is the same.
How about something like this:
def performAsync(asyncFunc, notifyFunc):
def threadProc():
retValue = asyncFunc()
glib.idle_add(notifyFunc, retValue)
Thread(target=threadProc).start()
def someGuiCallback(filebutton):
path = filebutton.getPath()
performAsync(
lambda: getMd5Sum(path),
lambda md5sum: showNotification("Md5Sum of file: %s" % md5sum)
)
A bit ugly with the lambdas, but it's simple and probably more readable than using precompiler tricks.
Sure you can access function code (already compiled) from decorator, disassemble and hack it. You can even access the source of module it's defined in and recompile it. But I think this is not necessary. Below is an example using decorated generator, where yield statement serves as a delimiter between synchronous and asynchronous parts:
from threading import Thread
import hashlib
def async(gen):
def func(*args, **kwargs):
it = gen(*args, **kwargs)
result = it.next()
Thread(target=lambda: list(it)).start()
return result
return func
#async
def test(text):
# synchronous part (empty in this example)
yield # Use "yield value" if you need to return meaningful value
# asynchronous part[s]
digest = hashlib.md5(text).hexdigest()
print digest
Im a coding a library including textual feedback that I need to translate.
I put the following lines in a _config.py module that I import everywhere in my app :
import gettext, os, sys
pathname = os.path.dirname(sys.argv[0])
localdir = os.path.abspath(pathname) + "/locale"
gettext.install("messages", localdir)
I have the *.mo files in ./locale/lang_LANG/LC_MESSAGES and I apply the _() function to all the strings that need to be translated.
Now I just added a feature for the user, supposedly a programmer, to be able to create his own messages. I don't want him to care about the underlying implementation, so I want him to be able to make it something straightforward like :
lib_object.message = "My message"
I used properties to make it clean, but what if my user whats to translate his own code (that uses mine) and does something like :
import gettext, os, sys
pathname = os.path.dirname(sys.argv[0])
localdir = os.path.abspath(pathname) + "/locale"
gettext.install("user_app", localdir)
lib_object.message = _("My message")
Is it a problem ? What can I do to avoid troubles without bothering my user ?
You can use the class based gettext api to isolate message catalogs. This is also what is recommended in the python gettext documentation.
The drawback is that you, or the other dev, will have to use the gettext method or define the _() method in the local scope, bound to the specific gettext class. An example of a class with its own string catalog:
import gettext
class MyClass(object):
def __init__(self, locale_for_instance):
self.lang = gettext.translation("appname", localedir, \
locale=locale_for_instance)
def some_method(self, arg):
return self.lang.gettext("You called some method")
def other_method(self, arg): # does the same thing
_ = self.lang.gettext
return _("You called some method")
You could stick the code for adding the _() in a decorator, so all the methods that need it is prefixed with something like #with_local_gettext
(Note, I've not tested the above could but It Should Work Just Fine(tm) )
If the goal is to not bother your user (and he's not very good) I guess you could use the class based approach in your code and let the user use the global one.
You can only gettext.install() once. In general it's useless for library work -- gettext.install() will only do the right thing if the module calling it is in charge of the whole program, since it will only provide you with one catalog to load from. Library code should do something akin to what Mailman does: have their own wrapper for gettext() that passes the right arguments for this module, then imports that as '_' in each module that wants to use it.