Summary:
The same import statement that works when __name__ == "__main__" stops working when imported from another module outside the immediate package. If I fix it so that it works when imported, it stops working when __name__ == "__main__". Is there a win-win solution for these conflicting scopes?
In more detail:
Say I have the following system layout.
package/
__init__.py
outer_module.py
subpackage/
__init__.py
inner_module_1.py
inner_module_2.py
And say that all modules are empty except for the following two.
# inner_module_1
import inner_module_2
&
# outer_module
import subpackage.inner_module_1
import subpackage.inner_module_2
If I run inner_module_1, there is no error.
If I run outer_module, I get the following ImportError from inner_module_1.
Traceback (most recent call last):
File "C:\package\outer_module.py", line 1, in <module>
import subpackage.inner_module_1
File "C:\package\subpackage\inner_module_1.py", line 1, in <module>
import inner_module_2
ImportError: No module named 'inner_module_2'
I can prevent that error by adding the package name to the import statement in inner_module_1 as follows.
# inner_module_1
import subpackage.inner_module_2
Although this change allows outer_module to run without an error, now running inner_module_1, which had before run without error, raises the following error.
Traceback (most recent call last):
File "C:\package\subpackage\inner_module_1.py", line 1, in <module>
import subpackage.inner_module_2
ImportError: No module named 'subpackage'
So here are my questions.
Is the import statement in inner_module_1 interpreted differently when accessed by outer_module than when run in inner_module_1, and if so, how so?
Can I write an import statement in inner_module_1 that works both when inner_module_1 is run and also when outer_module is run, and if so, what is that statement?
Thanks,
Victor
Related
I have a folder structure similar to this (my example has all the necessary bits):
web-scraper/
scraper.py
modules/
__init__.py
config.py
website_one_scraper.py
Where config.py just stores some global variables. It looks a bit like:
global var1
var1 = "This is a test!"
Within website_one_scraper.py it looks like this:
import config
def test_function():
# Do some web stuff...
return = len(config.var1)
if __name__ == "__main__":
print(test_function)
And scraper.py looks like this:
from module import website_one_scraper
print(website_one_scraper.test_function())
website_scraper_one.py works fine when run by itself, and thus the code under if __name__ == "__main__" is run. However, when I run scraper.py, I get the error:
ModuleNotFoundError: No module named 'config'
And this is the full error and traceback (albeit with different names, as I've changed some names for the example above):
Traceback (most recent call last):
File "c:\Users\User\Documents\Programming\Work\intrack-web-scraper\satellite_scraper.py", line 3, in
<module>
from modules import planet4589
File "c:\Users\User\Documents\Programming\Work\intrack-web-scraper\modules\planet4589.py", line 5, in
<module>
import config
ModuleNotFoundError: No module named 'config'
Also note that In scraper.py I've tried replacing from modules import website_one_scraper with import website_one_scraper, from .modules import website_one_scraper, and from . import website_one_scraper, but they all don't work.
What could the cause of my error be? Could it be something to do with how I'm importing everything?
(I'm using Python 3.9.1)
In your website_scraper_one.py, instead of import config.py try to use from . import config
Explanation:
. is the current package or the current folder
config is the module to import
I'm trying to import the requests module to get familiar with bs4, but the request module in the file I'm currently working in is grayed out so it isn't being recognized as a module. When I run the almost empty program, I get an error for an unrelated python file within my project.
Should I individually store each python file I make inside of a separate folder?
Both of these files are inside of the same project folder.
import requests
response = get('https://www.newegg.ca/p/N82E16868105274')
print(response.raise_for_status())
Error:
Traceback (most recent call last):
File "C:\Users\Denze\MyPythonScripts\Webscraping learning\beautifulsoup tests.py", line 1, in <module>
import requests
File "C:\Users\Denze\MyPythonScripts\requests.py", line 3, in <module>
res = requests.get('')
AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import)
Process finished with exit code 1
The other code in question that I think is causing my error:
import requests
res = requests.get('')
playFile = ('TestDownload.txt', 'wb')
for chunk in res.iter_content(100000):
playFile.write(chunk)
playFile.close()
You have a name collision. You're not importing the requests library, you're importing your script.
You wanted to do the following with your imports:
MyPythonScripts\beautifulsoup tests.py
→ requests.get() (the library)
What you're doing instead is:
MyPythonScripts\beautifulsoup tests.py
→ MyPythonScripts\requests.py
→ MyPythonScripts\requests.py .get() (the same file again)
That's the "circular import" that is mentioned in the traceback. The module imports itself and tries to use an attribute that isn't there before it finishes "executing", so the interpreter thinks it's due to the unfinished initialization
Raname MyPythonScripts\requests.py to something else and it should work.
I have this
folder_tree
and i want to import class from runner.py in crawlers.py
from scraper.runner import Runner
runner = Runner([{'name': 'my_name', 'urls': ["https://www.exaple.com/"]}])
runner.crawl()
but i got error
Traceback (most recent call last):
File "./scraper/crawlers/actor_crawler.py", line 3, in <module>
from scraper.runner import Runner
ModuleNotFoundError: No module named 'scraper'
Also, i tried relative import:
from ..runner import Runner
And got that:
ValueError: attempted relative import beyond top-level package
I've found the following link to be incredibly helpful in understanding how Python does imports.
The surefire way to ensure your module can be imported though is adding it to os.syspath. Try adding the following to the beginning of your script:
import os
import sys
sys.path.append(/path/to/scraper/scraper/runner.py)
from scraper.runner import Runner
runner = Runner([{'name': 'my_name', 'urls': ["https://www.exaple.com/"]}])
runner.crawl()
I have a following directory structure:
source
source_1.py
__init__.py
source1.py has class Source defined
source1.py
class Source(object):
pass
I am able to import using this
>>> from source.source1 import Source
>>> Source
<class 'source.source1.Source'>
However when trying to import using the below method it fails.
>>> from source import *
>>> source1.Source
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'source1' is not defined
Please let me know how can we use the 2nd import ?
For importing from a package (unlike importing from a module) you need to specify what * means. To do that, in __init__.py add a line like this:
__all__ = ["source1"]
See the Python documentation for Importing * From a Package.
I am pretty much new to Python and cannot quite understand the scenario below.
Firstly, there exists no module so the error below is understandable.
>>> import module1
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import module1
ImportError: No module named module1
Then I created the module and the sentences below worked perfectly.
module1.py
def func():
print 'this is a test module'
Python IDLE
>>> import module1
>>> module1.func()
this is a test module
On the execution of the first statement, module1.pyc file was created. Then I added another function to module1.py.
module1.py
def func():
print 'this is a test module'
def func1():
print 'this is func1'
and tried to import this new file and run the statements below. But error was thrown as follows.
Python IDLE
>>> import module1
>>> module1.func1()
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
module1.func1()
AttributeError: 'module' object has no attribute 'func1'
I then deleted module1.py from the folder and then re-saved module1.py (with no contents changed). Again I executed the above statements in the IDLE prompt and the same error was thrown. And this time when the import module1 statement was executed, module1.pyc file wasn't created unlike the previous time.
However, on restarting the IDLE everything works perfectly but why doesn't the IDE recompile the module1.pyc everytime import statement is executed(without restarting or opening another IDLE window) or could anyone explain what exactly happens in the memory when import statement is executed everytime.
Thanks!!
If you import a module that has already been imported, it will not "re-import" and you will just get a reference to the original module.
You should use reload to re-import the module if it has been already imported:
reload(module1)