I have two files 'mod1.py' and 'mod2.py'.
mod1 requires request module to function. But I have not imported them in the mod1 instead I have imported both request and mod1 module in mod2.
But
I get an error 'name 'requests' is not defined'. I know it works if i Import 'request' module in mod1 directly it works fine. But I have other modules I want to use that requires 'request' module. So how do I import the module once and make in accessible to all the other modules ?.
mod1.py
class getUrl():
def __init__(self, url):
self.url = url
def grab_html(self):
html = requests.get(self.url).text
return html
mod2.py
import requests
import mod1
module1 = mod1.getUrl('https://www.wikipedia.org/')
HTML = module1.grab_html()
Edit: Complete error
Traceback (most recent call last):
File "C:\Users\camel\Desktop\test\mod2.py", line 5, in <module>
HTML = module1.grab_html()
File "C:\Users\camel\Desktop\test\mod1.py", line 6, in grab_html
html = requests.get(self.url).text
NameError: name 'requests' is not defined
[Finished in 0.5s with exit code 1]
[shell_cmd: python -u "C:\Users\guru\Desktop\test\mod2.py"]
When you import something, it becomes a named thing in the module that imported it. Requests is not being used driectly by mod2.py, but is by mod1.py so that's where you should import it.
You can do this, for example.
mod1.py
import requests
class getUrl():
def __init__(self, url):
self.url = url
def grab_html(self):
html = requests.get(self.url).text
return html
mod2.py
import mod1
module1 = mod1.getUrl('https://www.wikipedia.org/')
HTML = module1.grab_html()
# And also access requests via mod1
indirectly = mod1.requests.get('https://www.wikipedia.org/').text
import requests, should be in mod1.py, because it is used in the methods of the class defined in mod1.py. you could import it in both places if it was needed in mod2.py as well.
As you are not using requests in mod2.py, you could just do import requests in mod1.py
If you are worried about the memory, it will take the same amount as you are going to use it in just one script. But if you are using if you are planning to use it in mod2.py as well, then you have to include there also.
You need to create an __init__.py file (it can be empty) file so that the folder containing mod1 is recognized as a module.
Then you can do from mod1 import *, or from path.to.mod1 import * and it will carry over all the imports over to mod2. Check out this relative answer. In my opinion this is a sensible way of doing things, as you can keep all your dependencies in a centralized location.
As you're concerned about memory utilization, take a look at another conversation on the same matter.
Related
I'm trying to make a library out of a Python project I don't own.
The project has the following directory layout:
.
├── MANIFEST.in
├── pyproject.toml
└── src
├── all.py
├── the.py
└── sources.py
In pyproject.toml I have:
[tool.setuptools]
packages = ["mypkg"]
[tool.setuptools.package-dir]
mypkg = "src"
The problem I'm facing is that when I build and install this package I can't use it because the author is importing stuff without mypkg prefix in the various source files.
F.ex. in all.py
from the import SomeThing
Since I don't own the package I can't go modify all the sources but I still want to be able to build a library from it by just adding MANIFEST.in and pyproject.toml.
Is it possible to somehow instruct setuptools to build a package that won't litter site-packages with all the sources while still allowing them to be imported without the mypkg prefix?
It isn't possible without adding a custom import hook with the package. The hook takes the form of a module that is shipped with the package, and it must be imported before usage from your module (e.g. in src/all.py)
src/mypkgimp.py
import sys
import importlib
class MyPkgLoader(importlib.abc.Loader):
def find_spec(self, name, path=None, target=None):
# update the list with modules that should be treated special
if name in ['sources', 'the']:
return importlib.util.spec_from_loader(name, self)
return None
def create_module(self, spec):
# Uncomment if "normal" imports should have precedence
# try:
# sys.meta_path = [x for x in sys.meta_path[:] if x is not self]
# return importlib.import_module(spec.name)
# except ImportError:
# pass
# finally:
# sys.meta_path = [self] + sys.meta_path
# Otherwise, this will unconditionally shadow normal imports
module = importlib.import_module('.' + spec.name, 'mypkg')
# Final step: inject the module to the "shortened" name
sys.modules[spec.name] = module
return module
def exec_module(self, module):
pass
if not hasattr(sys, 'frozen'):
sys.meta_path = [MyPkgLoader()] + sys.meta_path
Yes, the above uses different methods described by the thread I have linked previously, as importlib have deprecated those methods in Python 3.10, refer to documentation for details.
Anyway, for the demo, put some dummy classes in the modules:
src/the.py
class SomeThing: ...
src/sources.py
class Source: ...
Now, modify src/all.py to have the following:
import mypkg.mypkgimp
from the import SomeThing
Example usage:
>>> from sources import Source
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'sources'
>>> from mypkg import all
>>> all.SomeThing
<class 'mypkg.the.SomeThing'>
>>> from sources import Source
>>> Source
<class 'mypkg.sources.Source'>
>>> from sources import Error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Error' from 'mypkg.sources' (/tmp/mypkg/src/sources.py)
Note how the import initially didn't work, but after mypkg.all got imported, the sources import now works globally. Hence care may be needed to not shadow "real" imports and I have provided the example to import using the "default"[*] import mechanism.
If you want the module names to look different (i.e. without the mypkg. prefix), that will be a separate question, as code typically don't check for their own module name for functionality (and never mind that this actually shows how the namespace is implicitly used - changing the actual name is more akin to a module relocation, yes this can be done, but a bit more complicated and this answer is long enough as it is).
[*] "default" as in not including behaviors introduced by this custom import hook - other import hooks may do their own other weird shenanigans.
I have the following Python package with 2 moludes:
-pack1
|-__init__
|-mod1.py
|-mod2.py
-import_test.py
with the code:
# in mod1.py
a = 1
and
# in mod2.py
from mod1 import a
b = 2
and the __init__ code:
# in __init__.py
__all__ = ['mod1', 'mod2']
Next, I am trying to import the package:
# in import_test.py
from pack1 import *
But I get an error:
ModuleNotFoundError: No module named 'mod1'
If I remove the dependency "from mod1 import a" in mod2.py, the import goes correctly. But that dependency makes the import incorrect with that exception "ModuleNotFoundError".
???
The issue here is that from mod2 perspective the first level in which it will search for a module is in the path from which you are importing it (here I am assuming that pack1 is not in your PYTHONPATH and that you are importing it from the same directory where pack1 is contained).
This means that if pack1 is in the directory /dir/to/pack1 and you do:
from mod1 import a
Python will look for mod1 in the same directory as pack1, i.e., /dir/to/pack1.
To solve your issue it is enough to do either:
from pack1.mod1 import a
or in Python 3.5+
from .mod1 import a
As a side note, unless this is a must for you, I do not recommend designing your package to be used as from pack import *, even if __all__ exists to give you better control of your public API.
I'm trying to use a variable as a module to import from in Python.
Using ImportLib I have been successfully able to find the test...
sys.path.insert(0, sys.path[0] + '\\tests')
tool_name = selected_tool.split(".")[0]
selected_module = importlib.import_module("script1")
print(selected_module)
... and by printing the select_module I can see that it succesfully finds the script:
<module 'script1' from 'C:\\Users\\.....">
However, when I try to use this variable in the code to import a module from it:
from selected_module import run
run(1337)
The program quits with the following error:
ImportError: No module named 'selected_module'
I have tried to add a init.py file to the main directory and the /test directory where the scripts are, but to no avail. I'm sure it's just something stupidly small I'm missing - does anyone know?
Import statements are not sensitive to variables! Their content are treated as literals
An example:
urllib = "foo"
from urllib import parse # loads "urllib.parse", not "foo.parse"
print(parse)
Note that from my_module import my_func will simply bind my_module.my_func to the local name my_func. If you have already imported the module via importlib.import_module, you can just do this yourself:
# ... your code here
run = selected_module.run # bind module function to local name
I have created my own module with filename mymodule.py. The file contains:
def testmod():
print "test module success"
I have placed this file within /Library/Python/2.7/site-packages/mymodule/mymodule.py
I have also added a __init__.py file, these have compiled to generate
__init__.pyc and mymodule.pyc
Then in the python console I import it
import mymodule
which works fine
when I try to use mymodule.testmod() I get the following error:
AttributeError: 'module' object has no attribute 'testmod'
So yeah it seems like it has no functions?
You have a package mymodule, containing a module mymodule. The function is part of the module, not the package.
Import the module:
import mymodule.mymodule
and reference the function on that:
mymodule.mymodule.testmod()
You can use from ... import and import ... as to influence what gets imported exactly:
from mymodule import mymodule
mymodule.testmod()
or
from mymodule import mymodule as nestedmodule
nestedmodule.testmod
or
from mymodule.mymodule import testmod
testmod()
etc.
I have a module called spellnum. It can be used as a command-line utility (it has the if __name__ == '__main__': block) or it can be imported like a standard Python module.
The module defines a class named Speller which looks like this:
class Speller(object):
def __init__(self, lang="en"):
module = __import__("spelling_" + lang)
# use module's contents...
As you can see, the class constructor loads other modules at runtime. Those modules (spelling_en.py, spelling_es.py, etc.) are located in the same directory as the spellnum.py itself.
Besides spellnum.py, there are other files with utility functions and classes. I'd like to hide those files since I don't want to expose them to the user and since it's a bad idea to pollute the Python's lib directory with random files. The only way to achieve this that I know of is to create a package.
I've come up with this layout for the project (inspired by this great tutorial):
spellnum/ # project root
spellnum/ # package root
__init__.py
spellnum.py
spelling_en.py
spelling_es.py
squash.py
# ... some other private files
test/
test_spellnum.py
example.py
The file __init__.py contains a single line:
from spellnum import Speller
Given this new layout, the code for dynamic module loading had to be changed:
class Speller(object):
def __init__(self, lang="en"):
spelling_mod = "spelling_" + lang
package = __import__("spellnum", fromlist=[spelling_mod])
module = getattr(package, spelling_mod)
# use module as usual
So, with this project layout a can do the following:
Successfully import spellnum inside example.py and use it like a simple module:
# an excerpt from the example.py file
import spellnum
speller = spellnum.Speller(es)
# ...
import spellnum in the tests and run those tests from the project root like this:
$ PYTHONPATH="`pwd`:$PYTHONPATH" python test/test_spellnum.py
The problem
I cannot execute spellnum.py directly with the new layout. When I try to, it shows the following error:
Traceback (most recent call last):
...
File "spellnum/spellnum.py", line 23, in __init__
module = getattr(package, spelling_mod)
AttributeError: 'module' object has no attribute 'spelling_en'
The question
What's the best way to organize all of the files required by my module to work so that users are able to use the module both from command line and from their Python code?
Thanks!
How about keeping spellnum.py?
spellnum.py
spelling/
__init__.py
en.py
es.py
Your problem is, that the package is called the same as the python-file you want to execute, thus importing
from spellnum import spellnum_en
will try to import from the file instead of the package. You could fiddle around with relative imports, but I don't know how to make them work with __import__, so I'd suggest the following:
def __init__(self, lang="en"):
mod = "spellnum_" + lang
module = None
if __name__ == '__main__':
module = __import__(mod)
else:
package = getattr(__import__("spellnum", fromlist=[mod]), mod)