I am reverse engineering for the purpose of fixing a script in Python. There is nothing about
from asyncio.windows_events import NULL
on the internet, and the documentation of asyncio is a little ambiguous. What does this line do? If I remove this line, will this affect the functionality of the code in any meaningful way?
from asyncio.windows_events import NULL
means exactly to import the name NULL from the module asyncio.windows_events.
That name is a re-export of _winapi.NULL.
_winapi is an extension module that is only compiled on Windows, and _winapi.NULL is exactly the integer 0.
In other words, you could probably replace that line with
NULL = 0
Anything that looks like
from some.package import something
means that the name something (it could be a class, a function, a variable, or etc) is being imported from some.package, and you will need to refer to this package's documentation (or in the worst case its source code) to figure out what exactly the declaration does. Though often, the code which does the import will make its purpose clear enough, even if you don't know exactly how the name is eventually defined.
Removing the import will break your code unless something was unused in the first place. Now Python won't know what something refers to.
Related
Short question: I have a module with objects. How can I do that if someone imports an object from my module, my specified exception is raised?
What I want to do: I write an architectural framework. A class for output depends on jinja2 external library. I want the framework to be usable without this dependency as well. In the package's __init__.py I write conditional import of my class RenderLaTeX (if jinja2 is available, the class is imported, otherwise not).
The problem with this approach is that I have some code which uses this class RenderLaTeX, but when I run it on a fresh setup, I receive an error like Import error: no class RenderLaTeX could be imported from output. This error is pretty unexpected and ununderstandable before I recall that jinja2 must be installed beforehand.
I thought about this solution: if the class is not available, __init__.py can create a string with this name. If a user tries to instantiate this object with the usual class constructor, they'll get a more meaningful error. Unfortunately, simple import
from output import RenderLaTeX
won't raise an error in this case.
What would you suggest, hopefully with the description of benefits and drawbacks?
Important UPD: I package my classes in modules and import them to the module via __init__.py, so that I import 'from lena.flow import ReadROOTFile', rather than 'from lena.flow.read_root_file import ReadROOTFile.'
When Python imports a module all of the code inside the file from which you are importing is executed.
If your RenderLaTeX class is therefore placed into a seperate file you can freely add logic which would prevent it from being imported (or run) if required dependencies are missing.
For example:
try:
import somethingidonthave
except ImportError:
raise Exception('You need this module!')
class RenderLaTeX(object):
pass
You can also add any custom message you want to the exception to better describe the error. This should work in both Python2 and Python3.
After a year of thinking, the solution appeared.
First of all, I think that this is pretty meaningless to overwrite an exception's type. The only good way would be to add a useful message for a missing import.
Second, I think that the syntax
from framework.renderers import MyRenderer
is really better than
from framework.renderers.my_renderer import MyRenderer
because it hides implementation details and requires less code from user (I updated my question to reflect that). For the former syntax to work, I have to import MyRenderer in __init__.py in the module.
Now, in my_renderer.py I would usually import third-party packages with
import huge_specific_library
in the header. This syntax is required by PEP 8. However, this would make the whole framework.renderers module depend on huge_specific_library.
The solution for that is to violate PEP 8 and import the library inside the class itself:
class MyRenderer():
def __init__(self):
import huge_specific_library
# ... use that...
Here you can catch the exception if that is important, change its message, etc. There is another benefit for that: there exist guides how to reduce import time, and they propose this solution (I read them a long time ago and forgot). Large modules require some time to be loaded. If you follow PEP 8 Style Guide (I still think that you usually should), this may lead to large delays just to make all imports to your program, not having done anything useful yet.
The only caveat is this: if you import the library in __init__, you should also import that to other class methods that use it, otherwise it won't be visible there.
For those who still doubt, I must add that since Python imports are cached, this doesn't affect performance if your method that uses import is not called too often.
Is there a way to perform a python import which is not atomic?
For instance, I have a file as follows:
# Filename: a.py
myvariable = 1
mylist = [1, 2, 3]
raise ImportError
donotimportthis = 5
I then have a separate file which does the following:
import a
a.myvariable == 1 # This is okay as it imported it
a.donotimportthis # <-- raise an exception as this is not imported.
I have a file which contains some python code, this follows the format of:
...variables...
import X
I do not have X installed nor do I want it however I do want the variables.
Note: This file is autogenerated not by me but by a tool whose version is frozen.
Two choices, in descending order of preference:
Change the autogeneration process. Instead of invoking proprietary_autogen_process, invoke custom_autogen_wrapper. This wrapper in turn first invokes the proprietary third-party tool, and then modifies the produced module source code by searching for the code that imports module X, and deletes everything after it.
This is relatively straightforward. You just need to take some care to not introduce false positives or false negatives by performing too loose (or too strict) matching of the import code. Ideally you’d use an AST rewriter but that’s probably overkill; a regular expression search for import X might work, although it will yield wrong results if this text appears inside a comment, string literal or inside a method which isn’t executed.
Generate a stub module X in a location where it will be found by the autogenerated module when importing the latter. I don’t recommend this because it’s tedious: You probably can’t just generate an empty module, since the autogenerated module will want to use X. You need to generate meaningful method stubs.
You can do specific imports with
from a import myvariable
EDIT: The above won't work if anything that is flat in the file raises an error. If you have no way to edit the imported file then I don't know if there is a (resonable) solution to this. Sorry didn't realise.
(an unreasonable solution would be to read in the file as text, slice it, and then run eval on it).
Or, as mentioned in the comments, put the stuff you don't want under
if __name__=="__main__":
<here>
Then it will only be invoked if you run the file directly.
What you can do is removing the donotimportthis variable at the end of the module, as follows: del donotimportthis. I hope it helps
If you type this:
import somemodule
help(somemodule)
it will print out paged package description. I would need to get the same description as a string but without importing this package to the current namespace. Is this possible? It surely is, because anything is possible in Python, but what is the most elegant/pythonic way of doing so?
Side note: by elegant way I mean without opening a separate process and capturing its stdout... ;)
In other words, is there a way to peek into a unimported but installed package and get its description? Maybe something with importlib.abc.InspectLoader? But I have no idea how to make it work the way I need.
UPDATE: I need not just not polluting the namespace but also do this without leaving any traces of itself or dependent modules in memory and in sys.modules etc. Like it was never really imported.
UPDATE: Before anyone asks me why I need it - I want to list all installed python packages with their description. But after this I do not want to have them imported in sys.modules nor occupying excessive space in memory because there can be a lots of them.
The reason that you will need to import the module to get a help string is that in many cases, the help strings are actually generated in code. It would be pointlessly difficult to parse the text of such a package to get the string since you would then have to write a small Python interpreter to reconstruct the actual string.
That being said, there are ways of completely deleting a temporarily imported modules based on this answer, which summarizes a thread that appeared on the Python mailing list around 2003: http://web.archive.org/web/20080926094551/http://mail.python.org/pipermail/python-list/2003-December/241654.html. The methods described here will generally only work if the module is not referenced elsewhere. Otherwise the module will be unloaded in the sense that import will reload it from scratch instead of using the existing sys.modules entry, but the module will still live in memory.
Here is a function that does approximately what you want and even prints a warning if the module does not appear to have been unloaded. Unlike the solutions proposed in the linked answer, this function really handles all the side-effects of loading a module, including the fact that importing one package may import other external packages into sys.modules:
import sys, warnings
def get_help(module_name):
modules_copy = sys.modules.copy()
module = __import__(module_name)
h = help(module)
for modname in list(sys.modules):
if modname not in modules_copy:
del sys[modname]
if sys.getrefcount(module) > 1:
warnings.warn('Module {} is likely not to be completely wiped'.format(module_name))
del module
return h
The reason that I make a list of the keys in the final loop is that it is inadvisable to modify a dictionary (or any other iterable) as you iterate through it. At least in Python 3, dict.keys() returns an iterable that is backed by the dictionary itself, not a frozen copy. I am not sure if h = ... and return h are even necessary, but in the worst case, h is just None.
Well, if you are only worried about keeping the global namespace tidy, you could always import in a function:
>>> def get_help():
... import math
... help(math)
...
>>> math
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'math' is not defined
I would suggest a different approach, if i understand you correctly, you wish to read a portion of a package, without importing it (even within a function with local scope). I would suggest a method to do so would be via accessing the (python_path)/Lib/site-packages/(package_name)/ and reading the contents of the respective files as an alternative to importing the module so Python can.
I have a plypython function which does some json magic. For this it obviously imports the json library.
Is the import called on every call to the function? Are there any performance implication I have to be aware of?
The import is executed on every function call. This is the same behavior you would get if you wrote a normal Python module with the import statement inside a function body as oppposed to at the module level.
Yes, this will affect performance.
You can work around this by caching your imports like this:
CREATE FUNCTION test() RETURNS text
LANGUAGE plpythonu
AS $$
if 'json' in SD:
json = SD['json']
else:
import json
SD['json'] = json
return json.dumps(...)
$$;
This is admittedly not very pretty, and better ways to do this are being discussed, but they won't happen before PostgreSQL 9.4.
The declaration in the body of a PL/Python function will eventually become an ordinary Python function and will thus behave as such. When a Python function imports a module for the first time the module is cached in the sys.modules dictionary (https://docs.python.org/3/reference/import.html#the-module-cache). Subsequent imports of the same module will simply bind the import name to the module object found in the dictionary. In a sense, what I'm saying may cast some doubt on the usefulness of the tip given in the accepted answer, since it makes it somewhat redundant, as Python already does a similar caching for you.
To sum things up, I'd say that if you import in the standard way of simply using the import or from [...] import constructs, then you need not worry about repeated imports, in functions or otherwise, Python has got you covered.
On the other hand, Python allows you to bypass its native import semantics and to implement your own (with the __import__() function and importlib module). If this is what you're doing, maybe you should review what's available in the toolbox (https://docs.python.org/3/reference/import.html).
I'm creating a class to extend a package, and prior to class instantiation I don't know which subset of the package's namespace I need. I've been careful about avoiding namespace conflicts in my code, so, does
from package import *
create problems besides name conflicts?
Is it better to examine the class's input and import only the names I need (at runtime) in the __init__ ??
Can python import from a set [] ?
does
for name in [namespace,namespace]:
from package import name
make any sense?
I hope this question doesn't seem like unnecessary hand-ringing, i'm just super new to python and don't want to do the one thing every 'beginnger's guide' says not to do (from pkg import * ) unless I'm sure there's no alternative.
thoughts, advice welcome.
In order:
It does not create other problems - however, name conflicts can be much more of a problem than you'd expect.
Definitely defer your imports if you can. Even though Python variable scoping is simplistic, you also gain the benefit of not having to import the module if the functionality that needs it never gets called.
I don't know what you mean. Square brackets are used to make lists, not sets. You can import multiple names from a module in one line - just use a comma-delimited list:
from awesome_module import spam, ham, eggs, baked_beans
# awesome_module defines lots of other names, but they aren't pulled in.
No, that won't do what you want - name is an identifier, and as such, each time through the loop the code will attempt to import the name name, and not the name that corresponds to the string referred to by the name variable.
However, you can get this kind of "dynamic import" effect, using the __import__ function. Consult the documentation for more information, and make sure you have a real reason for using it first. We get into some pretty advanced uses of the language here pretty quickly, and it usually isn't as necessary as it first appears. Don't get too clever. We hates them tricksy hobbitses.
When importing * you get everything in the module dumped straight into your namespace. This is not always a good thing as you could accentually overwrite something like;
from time import *
sleep = None
This would render the time.sleep function useless...
The other way of taking functions, variables and classes from a module would be saying
from time import sleep
This is a nicer way but often the best way is to just import the module and reference the module directly like
import time
time.sleep(3)
you can import like from PIL import Image, ImageDraw
what is imported by from x import * is limited to the list __all__ in x if it exists
importing at runtime if the module name isn't know or fixed in the code must be done with __import__ but you shouldn't have to do that
This syntax constructions help you to avoid any name collision:
from package import somename as another_name
import package as another_package_name