I need to import a single class (not the whole file) from a python file starting with number.
There was a topic on importing a whole module and it works, but can't find my way around this one.
(In python, how to import filename starts with a number)
Normally it would be:
from uni_class import Student
though the file is called 123_uni_class.
Tried different variations of
importlib.import_module("123_uni_class")
and
uni_class=__import__("123_uni_class")
Error:
from 123_uni_class import Student
^
SyntaxError: invalid decimal literal
importlib.import_module("123_uni_class") returns the module after importing it, you must give it a valid name in order to reuse it:
import importlib
my_uni_class = importlib.import_module("123_uni_class")
Then you can access your module under the name 'my_uni_class'.
This would be equivalent to import 123_uni_class as my_uni_class if 123_uni_class were valid in this context.
It works for me:
Python 3.6.8 (default, Feb 14 2019, 22:09:48)
[GCC 7.4.0] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib
>>> importlib.import_module('123_a')
<module '123_a' from '/path/to/123_a.py'>
>>> __import__('123_a')
<module '123_a' from '/path/to/123_a.py'>
You would not see an actual syntax error containing the literal text "from 123_uni_class import ..." unless you actually have some source code containing that line.
If you must, you can also bypass the import system entirely by reading the contents of the file and exec()-ing them, possibly into a namespace you provide. For example:
mod = {}
with open('123_uni_class.py') as fobj:
exec(fobj.read(), mod)
Student = mod['Student']
This general technique is used, for example, to read config files written in Python and things like that. I would discourage it though for normal usage and suggest you just use a valid module name.
Related
This question already has answers here:
from ... import OR import ... as for modules
(6 answers)
Closed 4 years ago.
I've always used from a import b but recently a team at work decided to move a module into a new namespace, and issued a warning notice telling people to replace import b with import a.b as b.
I've never used import as and the only documentation I can find seems to suggest it doesn't support import a.b as b, though clearly it does.
but is there actually a difference, and if so what?
As far as I know, correct me if I am wrong.
First, import a.b must import a module(a file) or a package(a directory contains __init__.py).
For example, you can import tornado.web as web but you cannot import flask.Flask as Flask as Flask is an object in package flask.
Second, import a.b also import the namespace a which from a import b won't. You can check it by globals().
So what's the influence? For example:
import tornado.web as web
Now you have access to namespace tornado, but you cannot access tornado.options even though tornado has this module. But as python's global package management, if you from tornado import options, you will not only get access to options but also add it to namespace tornado. So now you can also access options by tornado.options.
I am going to provide only a partial answer, and I will speculate a bit.
1) Sometimes I have observed that the second way works while the first does not. On my system:
Python 3.6.3 (default, Oct 3 2017, 21:45:48)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tensorflow import keras # <- this works
>>>
>>> import tensorflow.keras as K # <- this fails
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorflow.keras'
>>>
2) I usually don't see a difference between these two approaches to importing. I haven't investigated WHY there is a difference with TensorFlow. It may have to do with what names are imported to the top level by the various TensorFlow subfolder init.py files (which are completely empty in most cases, but the one in ../dist_packages/tensorflow/python is pretty long).
I am running python 3.6.4 (anaconda, spyder).
Do I need to reload a user-defined module in order to capture the changes?
For example, suppose I wrote the simple function and save it in test.py file:
def plus5(x):
return x + 5
Then in the IPython console I type
import test as t
and then I change the user-defined function to:
def plus5(x):
return x + 500
Then when I type in IPython console
t.plus5(0)
it returns 500 without re-importing or reloading the module first.
If I change the function name from plus5 to something else then I have to re-import the module to see the change. But when I change the function statements then it automatically captures the changes without re-importing the module
From the Python documentation:
Note: For efficiency reasons, each module is only imported once per interpreter session. Therefore, if you change your modules, you must restart the interpreter – or, if it’s just one module you want to test interactively, use importlib.reload()
e.g. import importlib; importlib.reload(modulename).
This is a feature in the IPython interpreter name autoreload. It has the magic command %autoreload which allows for activating or deactivating this feature. It seems to be on by default, but I was not able to find something proving that.
As Megalng explained, this is a built in feature of IPython interpreter and in default Python interpreter you have to use importlib to reload the module. Here is default python interpreter execution,
Python 3.6.2 (default, Sep 5 2017, 17:37:49)
[GCC 4.6.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>>
>>> import test as t
>>> t.plus5(0)
5
>>>
>>>
>>> #Changed function body to return x + 500
...
>>> t.plus5(0)
5
>>> import test as t
>>> t.plus5(0)
5
>>> #It had no affect, even importing again doesn't work.
...
>>> import importlib; importlib.reload(t)
<module 'test' from '~/test.py'>
>>>
>>> t.plus5(0)
500
>>> #Now it works !
...
>>>
As you can see, even after changing function body to return x + 500, it still generated a result of 5 for t.plus5(0), even importing test module again did not help. It only started working when importlib was used to reload test module.
I'm trying to get to grips with a Python module I've installed called 'mido' for handling MIDI I/O.
The function mido.get_output_names should tell me what output ports are available, but, when I try to use it in the interactive interpreter, I get the following error(s):
Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from mido import *
>>> mido.get_output_names()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'mido' is not defined
>>> get_output_names()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'get_output_names' is not defined
>>>
I've seen other questions with similar issues, but the suggested solution seems to be to name the package before the call (in this case 'mido.') but as you can see that doesn't seem to make a difference here.
I've also tried putting the code in a .py file and interpreting/running that and I get the same error messages (for with and without the '.mido' respectively)
Can anyone help me work out what I've missed?
I've also tried from mido.port import * and calls to port.get_output_names() in as many combinations as I could think of, with similar equivalent NameError messages.
Looking at the __init__.py file of the mido module you can see that it prevents star * imports by setting __all__ to an empty list:
# Prevent splat import.
__all__ = []
__all__ is the list of names picked up by from mod import *, setting it to [] makes sure nothing gets imported.
it also sets a couple of additional functions (like get_output_names) in the module dictionary via use of the set_backend helper function.
So, either import mido directly and use get_output_names by prefixing the module name:
import mido
mido.get_output_names(...)
or, import the name from the module and use it directly:
from mido import get_output_names
get_output_names(...)
seems strange, maybe try this way:
import mido
then when calling functions from the package, use:
mido.get_output_names()
you can also import this way:
import mido as md
then:
md.get_output_names()
Also:
- try to get to the package directory, and have a look in the files
- try to get help from the package from terminal when imported:
import mido
help(mido)
I have a strange error using sep, file, (etc.) arguments of python's print() function.
I tried to google it out, dag around stackoverflow, and read python's documentation but I came up with nothing.
I have attached a simple snippet, I would deeply appreciate any help.
# python
Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print("blah"*10, sep=" | ")
File "<stdin>", line 1
print("blah"*10, sep=" | ")
^
SyntaxError: invalid syntax
Try:
from __future__ import print_function
first
In the 2.x series, print is a statement, while in 3.x it's a function. If you want in 2.6+ to have print as a function, you use from __future__ import print_function as the first import statement.
Expect code to break though
The print function is specific to Python 3.
You have two solutions here:
Write
from __future__ import print_function
so you can use it as specified by cdarke.
Or you use print as a simple statement as it should be with older versions of Python (print "Hello World").
I need to read all modules (pre-compiled) from a zipfile (built by py2exe compressed) into memory and then load them all.
I know this can be done by loading direct from the zipfile but I need to load them from memory.
Any ideas? (I'm using python 2.5.2 on windows)
TIA Steve
It depends on what exactly you have as "the module (pre-compiled)". Let's assume it's exactly the contents of a .pyc file, e.g., ciao.pyc as built by:
$ cat>'ciao.py'
def ciao(): return 'Ciao!'
$ python -c'import ciao; print ciao.ciao()'
Ciao!
IOW, having thus built ciao.pyc, say that you now do:
$ python
Python 2.5.1 (r251:54863, Feb 6 2009, 19:02:12)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> b = open('ciao.pyc', 'rb').read()
>>> len(b)
200
and your goal is to go from that byte string b to an importable module ciao. Here's how:
>>> import marshal
>>> c = marshal.loads(b[8:])
>>> c
<code object <module> at 0x65188, file "ciao.py", line 1>
this is how you get the code object from the .pyc binary contents. Edit: if you're curious, the first 8 bytes are a "magic number" and a timestamp -- not needed here (unless you want to sanity-check them and raise exceptions if warranted, but that seems outside the scope of the question; marshal.loads will raise anyway if it detects a corrupt string).
Then:
>>> import types
>>> m = types.ModuleType('ciao')
>>> import sys
>>> sys.modules['ciao'] = m
>>> exec c in m.__dict__
i.e: make a new module object, install it in sys.modules, populate it by executing the code object in its __dict__. Edit: the order in which you do the sys.modules insertion and exec matters if and only if you may have circular imports -- but, this is the order Python's own import normally uses, so it's better to mimic it (which has no specific downsides).
You can "make a new module object" in several ways (e.g., from functions in standard library modules such as new and imp), but "call the type to get an instance" is the normal Python way these days, and the normal place to obtain the type from (unless it has a built-in name or you otherwise have it already handy) is from the standard library module types, so that's what I recommend.
Now, finally:
>>> import ciao
>>> ciao.ciao()
'Ciao!'
>>>
...you can import the module and use its functions, classes, and so on. Other import (and from) statements will then find the module as sys.modules['ciao'], so you won't need to repeat this sequence of operations (indeed you don't need this last import statement here if all you want is to ensure the module is available for import from elsewhere -- I'm adding it only to show it works;-).
Edit: If you absolutely must import in this way packages and modules therefrom, rather than "plain modules" as I just showed, that's doable, too, but a bit more complicated. As this answer is already pretty long, and I hope you can simplify your life by sticking to plain modules for this purpose, I'm going to shirk that part of the answer;-).
Also note that this may or may not do what you want in cases of "loading the same module from memory multiple times" (this rebuilds the module each time; you might want to check sys.modules and just skip everything if the module's already there) and in particular when such repeated "load from memory" occurs from multiple threads (needing locks -- but, a better architecture is to have a single dedicated thread devoted to performing the task, with other modules communicating with it via a Queue).
Finally, there's no discussion of how to install this functionality as a transparent "import hook" which automagically gets involved in the mechanisms of the import statement internals themselves -- that's feasible, too, but not exactly what you're asking about, so here, too, I hope you can simplify your life by doing things the simple way instead, as this answer outlines.
Compiled Python file consist of
magic number (4 bytes) to determine type and version of Python,
timestamp (4 bytes) to check whether we have newer source,
marshaled code object.
To load module you have to create module object with imp.new_module(), execute unmashaled code in new module's namespace and put it in sys.modules. Below in sample implementation:
import sys, imp, marshal
def load_compiled_from_memory(name, filename, data, ispackage=False):
if data[:4]!=imp.get_magic():
raise ImportError('Bad magic number in %s' % filename)
# Ignore timestamp in data[4:8]
code = marshal.loads(data[8:])
imp.acquire_lock() # Required in threaded applications
try:
mod = imp.new_module(name)
sys.modules[name] = mod # To handle circular and submodule imports
# it should come before exec.
try:
mod.__file__ = filename # Is not so important.
# For package you have to set mod.__path__ here.
# Here I handle simple cases only.
if ispackage:
mod.__path__ = [name.replace('.', '/')]
exec code in mod.__dict__
except:
del sys.modules[name]
raise
finally:
imp.release_lock()
return mod
Update: the code is updated to handle packages properly.
Note that you have to install import hook to handle imports inside loaded modules. One way to do this is adding your finder into sys.meta_path. See PEP302 for more information.