We Keep Coding

pytest: how might I simulate non-availability of a pip-installed module? [duplicate]

I'm trying to use a third-party lib (docutils) on Google App Engine and have a problem with this code (in docutils):
try:
import pwd
do stuff
except ImportError:
do other stuff
I want the import to fail, as it will on the actual GAE server, but the problem is that it doesn't fail on my development box (ubuntu). How to make it fail, given that the import is not in my own code?

Even easier than messing with __import__ is just inserting None in the sys.modules dict:
>>> import sys
>>> sys.modules['pwd'] = None
>>> import pwd
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pwd

In your testing framework, before you cause docutils to be imported, you can perform this setup task:
import __builtin__
self.savimport = __builtin__.__import__
def myimport(name, *a):
if name=='pwd': raise ImportError
return self.savimport(name, *a)
__builtin__.__import__ = myimport
and of course in teardown put things back to normal:
__builtin__.__import__ = self.savimport
Explanation: all import operations go through __builtin__.__import__, and you can reassign that name to have such operations use your own code (alternatives such as import hooks are better for such purposes as performing import from non-filesystem sources, but for purposes such as yours, overriding __builtin__.__import__, as you see above, affords truly simple code).

Why python cant see root direcory of scrapy project

I have this
folder_tree
and i want to import class from runner.py in crawlers.py
from scraper.runner import Runner
runner = Runner([{'name': 'my_name', 'urls': ["https://www.exaple.com/"]}])
runner.crawl()
but i got error
Traceback (most recent call last):
File "./scraper/crawlers/actor_crawler.py", line 3, in <module>
from scraper.runner import Runner
ModuleNotFoundError: No module named 'scraper'
Also, i tried relative import:
from ..runner import Runner
And got that:
ValueError: attempted relative import beyond top-level package

I've found the following link to be incredibly helpful in understanding how Python does imports.
The surefire way to ensure your module can be imported though is adding it to os.syspath. Try adding the following to the beginning of your script:
import os
import sys
sys.path.append(/path/to/scraper/scraper/runner.py)
from scraper.runner import Runner
runner = Runner([{'name': 'my_name', 'urls': ["https://www.exaple.com/"]}])
runner.crawl()

Package import failure in Python 3.5

I have the following folder structure:
/main
main.py
/io
__init__.py
foo.py
In Python 2.7 I would write the following in main.py:
import io.foo
or
from io.foo import *
wheareas in Python 3.5 I get an import error:
Traceback (most recent call last):
File "./main.py", line 6, in <module>
import io.foo
ImportError: No module named 'io.foo'; 'io' is not a package
I couldn't find any help so far.

io is a built-in module. Don't name your local packages the same as a built-in module.

While #ErikCederstrand's answer is correct and probably sufficient for you, I was curious as to why it failed so I went digging through cpython's source. So for any future visitors, here's what I found.
The function where it's failing is here: https://github.com/python/cpython/blob/3.4/Lib/importlib/_bootstrap.py#L2207
On line 2209, it checks to see if the parent module has been loaded:
parent = name.rpartition('.')[0] # Value of 'io'
Since it has loaded the builtin io module, it continues on like normal. After the if returns false, it goes on to assign parent module which again is set to "io":
if name in sys.modules:
return sys.modules[name]
parent_module = sys.modules[parent]
The next lines are what cause the failure, and it's because builtin modules (io anyway) don't have a __path__ instance variable. The exception you see raised here are ultimately what you're seeing when you run it:
try:
path = parent_module.__path__
except AttributeError:
msg = (_ERR_MSG + '; {!r} is not a package').format(name, parent)
raise ImportError(msg, name=name)
If you change your module name like Erik says, then step through this whole process, you can see the call to get parent_module.__path__ works like it's supposed to and everything's happy.
So, tldr: you've tricked the import system into thinking it's already loaded your custom module, but when it goes to try and use it like a custom module it fails because it's actually the builtin io.
EDIT: It looks like __path__ is set here after it goes through a normal import process in init_module_attrs:
if _override or getattr(module, '__path__', None) is None:
if spec.submodule_search_locations is not None:
try:
module.__path__ = spec.submodule_search_locations
except AttributeError:
pass

For Python 3.4: How do I write an import module statement that works from any scope?

Summary:
The same import statement that works when __name__ == "__main__" stops working when imported from another module outside the immediate package. If I fix it so that it works when imported, it stops working when __name__ == "__main__". Is there a win-win solution for these conflicting scopes?
In more detail:
Say I have the following system layout.
package/
__init__.py
outer_module.py
subpackage/
__init__.py
inner_module_1.py
inner_module_2.py
And say that all modules are empty except for the following two.
# inner_module_1
import inner_module_2
&
# outer_module
import subpackage.inner_module_1
import subpackage.inner_module_2
If I run inner_module_1, there is no error.
If I run outer_module, I get the following ImportError from inner_module_1.
Traceback (most recent call last):
File "C:\package\outer_module.py", line 1, in <module>
import subpackage.inner_module_1
File "C:\package\subpackage\inner_module_1.py", line 1, in <module>
import inner_module_2
ImportError: No module named 'inner_module_2'
I can prevent that error by adding the package name to the import statement in inner_module_1 as follows.
# inner_module_1
import subpackage.inner_module_2
Although this change allows outer_module to run without an error, now running inner_module_1, which had before run without error, raises the following error.
Traceback (most recent call last):
File "C:\package\subpackage\inner_module_1.py", line 1, in <module>
import subpackage.inner_module_2
ImportError: No module named 'subpackage'
So here are my questions.
Is the import statement in inner_module_1 interpreted differently when accessed by outer_module than when run in inner_module_1, and if so, how so?
Can I write an import statement in inner_module_1 that works both when inner_module_1 is run and also when outer_module is run, and if so, what is that statement?
Thanks,
Victor

ImportError: cannot import name SlotMap

I am encountering an import error
Traceback (most recent call last):
File "C:\Users\bartis\Desktop\Python\TEC-KB\SlotMapper.pyw", line 9, in <module>
from SlotMapper import SlotMap
File "C:\Users\bartis\Desktop\Python\TEC-KB\SlotMapper.pyw", line 9, in <module>
from SlotMapper import SlotMap
ImportError: cannot import name 'SlotMap
This should be a straightforward issue, but I can’t seem to find the problem. If I place the SlotMapper.py file in the same directory as the GUI I am using the import of SlotMap occurs without error. If I move the file to a directory under the current working directory and add - sys.path.append(os.path.join(os.getcwd(), 'appLib')) I receive the error above. See import statements and modification of PYTHONPATH below. I know the PYTHONPATH has been modified after I checked it from the debugger. I also know since there are other files under appLib required for the GUI to operate. Finally, I have checked all of the imported files for a circular reference and find none… So stuck. Any suggestions welcome
import os
import sys
sys.path.append(os.path.join(os.getcwd(), 'appLib', 'KB-GUI'))
sys.path.append(os.path.join(os.getcwd(), 'appLib'))
from tkinter import *
from SlotMapper import SlotMap
from ShelfTypeSelection import ShelfTypeSelector
from PackTypeSelection import PackTypeSlotMappingSelector
from EntryWidgets import EntryBase, ShelfSlotEntry

The reason this is not working is because your file is named SlotMapper.pyw. The line
from SlotMapper import SlotMap
is trying to import SlotMap from your current file, hence the error. Try renaming your file to slotmapper_test.pyw or something like that, and everything should work as expected. You don't want your code files to have the same names as any modules you're trying to import, as the import mechanism will try to find the classes/functions there first, instead of searching your modules first.