I have a folder structure similar to this (my example has all the necessary bits):
web-scraper/
scraper.py
modules/
__init__.py
config.py
website_one_scraper.py
Where config.py just stores some global variables. It looks a bit like:
global var1
var1 = "This is a test!"
Within website_one_scraper.py it looks like this:
import config
def test_function():
# Do some web stuff...
return = len(config.var1)
if __name__ == "__main__":
print(test_function)
And scraper.py looks like this:
from module import website_one_scraper
print(website_one_scraper.test_function())
website_scraper_one.py works fine when run by itself, and thus the code under if __name__ == "__main__" is run. However, when I run scraper.py, I get the error:
ModuleNotFoundError: No module named 'config'
And this is the full error and traceback (albeit with different names, as I've changed some names for the example above):
Traceback (most recent call last):
File "c:\Users\User\Documents\Programming\Work\intrack-web-scraper\satellite_scraper.py", line 3, in
<module>
from modules import planet4589
File "c:\Users\User\Documents\Programming\Work\intrack-web-scraper\modules\planet4589.py", line 5, in
<module>
import config
ModuleNotFoundError: No module named 'config'
Also note that In scraper.py I've tried replacing from modules import website_one_scraper with import website_one_scraper, from .modules import website_one_scraper, and from . import website_one_scraper, but they all don't work.
What could the cause of my error be? Could it be something to do with how I'm importing everything?
(I'm using Python 3.9.1)
In your website_scraper_one.py, instead of import config.py try to use from . import config
Explanation:
. is the current package or the current folder
config is the module to import
Related
I have a script that have multiple files and a config file with all the variables store in one Python file.
Folder structure:
Config file:
If I try to run the main file which calls the head function imported, an error pops up saying that the config cannot be imported.
Imports:
Your Functions folder has a __init__.py file. If your app executes from Main.py (ie if Main.py satisfies __name__ == "__main__") therefore wherever you are in your app you could import the config like this:
from Functions.Config import *
Edit:
However,from module import * is not recommended with local import. You could give an alias to the import and call name.variable instead of calling name directly.
Example:
def head():
# import packages
import Function.Config as conf
print(conf.more_200)
head()
>>> 50
Your syntax are wrong.
It is supposed to be from Config import * and not import .Config import *
I have the following code structure:
Graph API/
│── main.py
├── helper_functions/
├── defines.py
├── insights.py
insights.py imports 2 functions from defines.py at the beginning:
from defines import getCreds, makeApiCall
It then uses "makeApiCall" for this function:
def getUserMedia( params ) :
// Some code about url endpoints etc.
return makeApiCall( url, endpointParams, params['debug'] ) # make the api call
I want to use the getUserMedia function in the main.py script, so I import it with:
from helper_functions.insights import *
But I get the error:
Traceback (most recent call last):
File "/Users/Graph_API/main.py", line 1, in <module>
import helper_functions.insights
File "/Users/Graph_API/helper_functions/insights.py", line 1, in <module>
from defines import getCreds, makeApiCall
ModuleNotFoundError: No module named 'defines'
What leads to this error? When I use the getUserMedia function within insights.py it works fine. I already tried importing defines.py to main.py as well, but I get the same error.
I am pretty new to programming, so I would really appreciate your help :)
You should replace
from defines import getCreds, makeApiCall
With
from helper_functions.defines import getCreds, makeApiCall
Or
from .defines import getCreds, makeApiCall
You must give a path either from project directory as in the first example, or from the relative path from the insights file by adding a dot.
You could also consider adding init.py to the helper_functions folder. Checkout here: What is __init__.py for?
I have this
folder_tree
and i want to import class from runner.py in crawlers.py
from scraper.runner import Runner
runner = Runner([{'name': 'my_name', 'urls': ["https://www.exaple.com/"]}])
runner.crawl()
but i got error
Traceback (most recent call last):
File "./scraper/crawlers/actor_crawler.py", line 3, in <module>
from scraper.runner import Runner
ModuleNotFoundError: No module named 'scraper'
Also, i tried relative import:
from ..runner import Runner
And got that:
ValueError: attempted relative import beyond top-level package
I've found the following link to be incredibly helpful in understanding how Python does imports.
The surefire way to ensure your module can be imported though is adding it to os.syspath. Try adding the following to the beginning of your script:
import os
import sys
sys.path.append(/path/to/scraper/scraper/runner.py)
from scraper.runner import Runner
runner = Runner([{'name': 'my_name', 'urls': ["https://www.exaple.com/"]}])
runner.crawl()
I'm new to python and I'm having this problem that I can't figure it out.
My file structure is:
enter image description here
On Criador.py I have several functions, for example:
def doSomething():
pass
def doSomethingElse():
pass
and Im trying to use one of this functions on the Controller.py file:
The first thing I did was, on the Controller.py:
import Controller.Criador
and then tried to use that function as:
Controller.Criador.doSomething()
After running Controller.py, I got this error:
ModuleNotFoundError: No module named 'Controller.Criador'; 'Controller' is not a package
I tried several other things, like:
from . import Criador
or
from Controller.Criador import doSomething
or
from Controller import Criador
and nothing helped, just changed the errors to:
ImportError: cannot import name 'Criador'
and
ModuleNotFoundError: No module named 'Controller.Criador'; 'Controller' is not a package
and
ImportError: cannot import name 'Criador'
Can someone give me a light about this? I'm using PyCharm and it does not give me any error when I declare the imports, only when I run the file
If Controller.py and Criador.py are in the same folder, you can do this inside Controller.py:
import Criador
Criador.doSomething()
Summary:
The same import statement that works when __name__ == "__main__" stops working when imported from another module outside the immediate package. If I fix it so that it works when imported, it stops working when __name__ == "__main__". Is there a win-win solution for these conflicting scopes?
In more detail:
Say I have the following system layout.
package/
__init__.py
outer_module.py
subpackage/
__init__.py
inner_module_1.py
inner_module_2.py
And say that all modules are empty except for the following two.
# inner_module_1
import inner_module_2
&
# outer_module
import subpackage.inner_module_1
import subpackage.inner_module_2
If I run inner_module_1, there is no error.
If I run outer_module, I get the following ImportError from inner_module_1.
Traceback (most recent call last):
File "C:\package\outer_module.py", line 1, in <module>
import subpackage.inner_module_1
File "C:\package\subpackage\inner_module_1.py", line 1, in <module>
import inner_module_2
ImportError: No module named 'inner_module_2'
I can prevent that error by adding the package name to the import statement in inner_module_1 as follows.
# inner_module_1
import subpackage.inner_module_2
Although this change allows outer_module to run without an error, now running inner_module_1, which had before run without error, raises the following error.
Traceback (most recent call last):
File "C:\package\subpackage\inner_module_1.py", line 1, in <module>
import subpackage.inner_module_2
ImportError: No module named 'subpackage'
So here are my questions.
Is the import statement in inner_module_1 interpreted differently when accessed by outer_module than when run in inner_module_1, and if so, how so?
Can I write an import statement in inner_module_1 that works both when inner_module_1 is run and also when outer_module is run, and if so, what is that statement?
Thanks,
Victor