We Keep Coding

Why can't I import without getting an error about another python file? ("partially initialized module has no attribute")

I'm trying to import the requests module to get familiar with bs4, but the request module in the file I'm currently working in is grayed out so it isn't being recognized as a module. When I run the almost empty program, I get an error for an unrelated python file within my project.
Should I individually store each python file I make inside of a separate folder?
Both of these files are inside of the same project folder.
import requests
response = get('https://www.newegg.ca/p/N82E16868105274')
print(response.raise_for_status())
Error:
Traceback (most recent call last):
File "C:\Users\Denze\MyPythonScripts\Webscraping learning\beautifulsoup tests.py", line 1, in <module>
import requests
File "C:\Users\Denze\MyPythonScripts\requests.py", line 3, in <module>
res = requests.get('')
AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import)
Process finished with exit code 1
The other code in question that I think is causing my error:
import requests
res = requests.get('')
playFile = ('TestDownload.txt', 'wb')
for chunk in res.iter_content(100000):
playFile.write(chunk)
playFile.close()

You have a name collision. You're not importing the requests library, you're importing your script.
You wanted to do the following with your imports:
MyPythonScripts\beautifulsoup tests.py
→ requests.get() (the library)
What you're doing instead is:
MyPythonScripts\beautifulsoup tests.py
→ MyPythonScripts\requests.py
→ MyPythonScripts\requests.py .get() (the same file again)
That's the "circular import" that is mentioned in the traceback. The module imports itself and tries to use an attribute that isn't there before it finishes "executing", so the interpreter thinks it's due to the unfinished initialization
Raname MyPythonScripts\requests.py to something else and it should work.

Best practice to locate "attempted relative import beyond top-level package"

I would like to narrow the common error "attempted relative import beyond top-level package" down by including the exception-handling in the imports already.
In my Django tests in DjangoProject/tests.py I wrote the imports relative to itself (tests.py).
The default unittests.py which executes each testfile is located in the root-folder which also includes the DjangoProject-folder.
As I was getting the error attempted relative import beyond top-level package the moment I added the
DjangoProject/tests.py to my Unit-Tests adnd it was clear it came from the imports. With a try/except I was able to write a slightly clearer error so I'm not in the need of guessing around why and which file caused the error.
Now my code looks like this:
try:
from django.utils.safestring import mark_safe
import json
from django.conf import settings
from viewsfunctions import *
from .. import PoM
import unittest
except Exception as importex:
print("Error in the tests.py-imports: "+str(importex))
And it threw first of all the message:
"No module named 'viewsfunctions'"
which makes sense and is helpful as viewsfunctions is a module inside DjangoProject. I replaced the line with: from DjangoProject.viewsfunctions import *
The next error is:
Error in the Imports: attempted relative import beyond top-level package
And this is exactly my problem. This could be anything. By just pdbing around I could track it down to from .. import PoM (as the module PoM is relative to tests.py in the top-folder, but relative to the executing unittests.py in the same folder)
In this case with just a few imports it's relatively fast to narrow it down, but is there any way to write a better exception message or deliver somehow more information in which of the import-lines the error actually happened?

If you would simply not catch the exception, you'd get a very useful traceback:
>>> from .. import foo
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Users/deceze/...", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
ImportError: attempted relative import with no known parent package
The problem is that you're catching the exception and are only printing the most minimal error message possible. If you do want to catch the exception for some reason yet still get a complete error message, the easiest way would be to use the logging module:
import logging
try:
...
except:
logging.exception('Error fooing the bar')
This will produce your custom error message followed by a complete stack trace.
To get even more manual, use the traceback module:
import traceback
try:
...
except:
print('Error fooing the bar')
traceback.print_exc()
With other functions in traceback you can get even more granular and print or retrieve messages from an exception as granular as you like.

Package import failure in Python 3.5

I have the following folder structure:
/main
main.py
/io
__init__.py
foo.py
In Python 2.7 I would write the following in main.py:
import io.foo
or
from io.foo import *
wheareas in Python 3.5 I get an import error:
Traceback (most recent call last):
File "./main.py", line 6, in <module>
import io.foo
ImportError: No module named 'io.foo'; 'io' is not a package
I couldn't find any help so far.

io is a built-in module. Don't name your local packages the same as a built-in module.

While #ErikCederstrand's answer is correct and probably sufficient for you, I was curious as to why it failed so I went digging through cpython's source. So for any future visitors, here's what I found.
The function where it's failing is here: https://github.com/python/cpython/blob/3.4/Lib/importlib/_bootstrap.py#L2207
On line 2209, it checks to see if the parent module has been loaded:
parent = name.rpartition('.')[0] # Value of 'io'
Since it has loaded the builtin io module, it continues on like normal. After the if returns false, it goes on to assign parent module which again is set to "io":
if name in sys.modules:
return sys.modules[name]
parent_module = sys.modules[parent]
The next lines are what cause the failure, and it's because builtin modules (io anyway) don't have a __path__ instance variable. The exception you see raised here are ultimately what you're seeing when you run it:
try:
path = parent_module.__path__
except AttributeError:
msg = (_ERR_MSG + '; {!r} is not a package').format(name, parent)
raise ImportError(msg, name=name)
If you change your module name like Erik says, then step through this whole process, you can see the call to get parent_module.__path__ works like it's supposed to and everything's happy.
So, tldr: you've tricked the import system into thinking it's already loaded your custom module, but when it goes to try and use it like a custom module it fails because it's actually the builtin io.
EDIT: It looks like __path__ is set here after it goes through a normal import process in init_module_attrs:
if _override or getattr(module, '__path__', None) is None:
if spec.submodule_search_locations is not None:
try:
module.__path__ = spec.submodule_search_locations
except AttributeError:
pass

python unittest assertRaises

I copied this verbatim from python.org unittest documentation:
import random
import unittest
class TestSequenceFunctions(unittest.TestCase):
def setUp(self):
self.seq = range(10)
def test_shuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
# should raise an exception for an immutable sequence
self.assertRaises(TypeError, random.shuffle, (1,2,3))
def test_choice(self):
element = random.choice(self.seq)
self.assertTrue(element in self.seq)
def test_sample(self):
with self.assertRaises(ValueError):
random.sample(self.seq, 20)
for element in random.sample(self.seq, 5):
self.assertTrue(element in self.seq)
if __name__ == '__main__':
unittest.main()
But I get this error message from python 2.7.2 [GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2:
.E.
======================================================================
ERROR: test_sample (__main__.TestSequenceFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tmp.py", line 23, in test_sample
with self.assertRaises(ValueError):
TypeError: failUnlessRaises() takes at least 3 arguments (2 given)
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (errors=1)
How can I get assertRaises() to work properly?

The ability to use unittest.TestCase.AssertRaises() as context manager was added in python 2.7.
http://docs.python.org/2/library/unittest.html#unittest.TestCase.assertRaises

Check that you are really using 2.7 python.
Tested using pythonbrew:
$ pythonbrew use 2.7.2
$ python test.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
$ pythonbrew use 2.6.5
$ python test.py
.E.
======================================================================
ERROR: test_sample (__main__.TestSequenceFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 23, in test_sample
with self.assertRaises(ValueError):
TypeError: failUnlessRaises() takes at least 3 arguments (2 given)
----------------------------------------------------------------------
Ran 3 tests in 0.000s
FAILED (errors=1)

If you're using 2.7 and still seeing this issue, it could be because you're not using python's unittest module. Some other modules like twisted provide assertRaises and though they try to maintain compatibility with python's unittest, your particular version of that module may be out of date.

Combining nosetests with contracts for Python

I'm using contracts for Python to specify preconditons/postconditions/invariants. I'm also using doctests for doing unit testing.
I'd like to have all of my doctest unit tests run with contracts enabled, and I'd like to run my tests using nose. Unfortunately, if I run the tests with nose, it does not execute the pre/post/invariant assertions. I put a setup function in each .py file to make sure that contract.checkmod gets called
def setup():
import contract
contract.checkmod(__name__)
I can confirm that this function is being executed by nose before it runs the tests, but the contracts still don't get executed.
On the other hand, if I run the doctest by calling doctest.testmod, the pre/post/inv do get called:
def _test():
import contract
contract.checkmod(__name__)
import doctest
doctest.testmod()
if __name__=='__main__':
_test()
Here's an example of a Python script whose test will succeed if called directly, but failed if called with nose:
import os
def setup():
import contract
contract.checkmod(__name__)
def delete_file(path):
"""Delete a file. File must be present.
>>> import minimock
>>> minimock.mock('os.remove')
>>> minimock.mock('os.path.exists', returns=True)
>>> delete_file('/tmp/myfile.txt')
Called os.path.exists('/tmp/myfile.txt')
Called os.remove('/tmp/myfile.txt')
>>> minimock.restore()
pre: os.path.exists(path)
"""
os.remove(path)
if __name__ == '__main__':
setup()
import doctest
doctest.testmod()
When I run the above file standalone, the tests pass:
$ python contracttest.py -v
Trying:
import minimock
Expecting nothing
ok
Trying:
minimock.mock('os.remove')
Expecting nothing
ok
Trying:
minimock.mock('os.path.exists', returns=True)
Expecting nothing
ok
Trying:
delete_file('/tmp/myfile.txt')
Expecting:
Called os.path.exists('/tmp/myfile.txt')
Called os.remove('/tmp/myfile.txt')
ok
Trying:
minimock.restore()
Expecting nothing
ok
2 items had no tests:
__main__
__main__.setup
1 items passed all tests:
5 tests in __main__.delete_file
5 tests in 3 items.
5 passed and 0 failed.
Test passed.
Here it is with nose:
$ nosetests --with-doctest contracttest.py
F
======================================================================
FAIL: Doctest: contracttest.delete_file
----------------------------------------------------------------------
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 2131, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for contracttest.delete_file
File "/Users/lorin/Desktop/contracttest.py", line 10, in delete_file
----------------------------------------------------------------------
File "/Users/lorin/Desktop/contracttest.py", line 17, in contracttest.delete_file
Failed example:
delete_file('/tmp/myfile.txt')
Expected:
Called os.path.exists('/tmp/myfile.txt')
Called os.remove('/tmp/myfile.txt')
Got:
Called os.remove('/tmp/myfile.txt')
----------------------------------------------------------------------
Ran 1 test in 0.055s