We Keep Coding

How to understand Python's module lookup

I created two new files, random.py and main.py, in the directory. The code is as follows:
# random.py
if __name__ == "__main__":
print("random")
# main.py
import random
if __name__ == "__main__":
print(random.choice([1, 2, 3]))
When I run the main.py file, the program reports an error.
Traceback (most recent call last):
File "main.py", line 8, in <module>
print(random.choice([1, 2, 3]))
AttributeError: module 'random' has no attribute 'choice'
Main.py imports my own defined random module.
However, if I create a new sys.py file and a main.py file in the same directory, the code is as follows:
# sys.py
if __name__ == "__main__":
print("sys")
# main.py
import sys
if __name__ == "__main__":
print(sys.path)
When I run the main.py file, successfully.
main.py imports the built-in modules sys.
Why is there such a clear difference?
The directory relationship of the script file is as follows:
C:.
main.py
random.py
sys.py
Thank you very much for your answer.
Forgive my poor english.

sys is a built-in module, meaning it's compiled directly into the Python executable itself. Built-in modules outprioritize external files when Python is looking for modules. The standard random module isn't built-in, so it doesn't get that treatment.
Quoting the docs:
When the named module is not found in sys.modules, Python next searches sys.meta_path, which contains a list of meta path finder objects. These finders are queried in order to see if they know how to handle the named module...
Python’s default sys.meta_path has three meta path finders, one that knows how to import built-in modules, one that knows how to import frozen modules, and one that knows how to import modules from an import path (i.e. the path based finder).
Since the finder for built-in modules comes before the finder that searches the import path, built-in modules will be found before anything on the import path.
You can see a tuple of the names of all modules your Python has built-in in sys.builtin_module_names.
That said, while any built-in module would outprioritize a module loaded from a file, sys has its own special handling. sys is one of the foundational building blocks of Python, and much of the sys module's setup needs to happen before the import system is functional enough for the normal import process to work. sys gets explicitly created during interpreter setup in a way that bypasses the normal import system, and then future imports for sys find it in sys.modules without hitting any meta path finders.
How and where sys is created is an implementation detail that varies from Python version to Python version (and is wildly different in different Python implementations), but in the CPython 3.7.4 code, you can see it beginning on line 755 in Python/pylifecycle.c.

tl;dr Caching
sys is somewhat of a special case among other python modules because it gets loaded at program start, unconditionally (presumably because a lot of the constants, functions, and data within - such as the streams stdout and stderr - are used by the python interpreter). As #user2357112 noted in the other answer, this is partly because it's built-in to the python executable, but also because it's necessary for running a substantial amount of python's core functionality (see below how it needs to be loaded for imports to work). random is part of the standard library, but it doesn't get loaded automatically when you execute, which is the primary relevant difference between it and sys, for our purposes
Looking at python's documentation on the subject clarifies how python resolves imports:
The first place checked during import search is sys.modules. This mapping serves as a cache of all modules that have been previously imported, including the intermediate paths.
...
During import, the module name is looked up in sys.modules and if present, the associated value is the module satisfying the import, and the process completes. However, if the value is None, then a ModuleNotFoundError is raised. If the module name is missing, Python will continue searching for the module.
As for where it looks for the module, you can see in your observed behavior that it looks in the local directory first. That is, it searches the local directory first and then the "usual places" afterwards.
The reason for the discrepancy between how sys is handled and how random is handled is caching - sys is cached (so python doesn't even check the path to import), whereas random is not cached (so python does check the path to import it, and imports locally).
There are a few ways you can change this behavior.
First, if you must have a local module called sys, you can use importlib to import it in relative or absolute terms, without running into the ambiguity with the sys that's already cached. I have no idea how this would affect other modules that independently try to import sys, and you really shouldn't be naming your files the same as standard library modules anyway.
Alternatively, if you want the code to check python's built-in modules before checking the local directory, then you should be able to do that by modifying sys.path, which shows the order in which paths are searched for input (the same as the $PATH environment variable, or any other similar language-specific one). The first element of sys.path is usually going to be an empty string '', that would result in searching the current working directory. So you can simply move that to the back of sys.path, to have it searched last instead of first:
sys.path.append(sys.path.pop(0))

Where should my Python 3 modules be?

I recently installed Python 3 on my Mac OSX 10.6.8 and I haven't had any problems with modules or imports until now. I'm writing a function that tests whether or not a triangle is right angled based on the length of the sides and the guide that the exercise was in has a bunch of equalities to test so I can see if it works:
testEqual(is_rightangled(1.5,2.0,2.5), True)
testEqual(is_rightangled(4.0,8.0,16.0), False)
testEqual(is_rightangled(4.1,8.2,9.1678787077), True)
testEqual(is_rightangled(4.1,8.2,9.16787), True)
testEqual(is_rightangled(4.1,8.2,9.168), False)
testEqual(is_rightangled(0.5,0.4,0.64031), True)
I should apparently import a function called testEqual(a,b,c) from a module called test, since the example programme in the guide starts with from test import testEqual, but when I typed that into my file I got this message:
from test import testEqual
ImportError: cannot import name testEqual
I suppose I should specify the path to the test module, but I can't find it my Python 3 library anywhere in my computer – just the 2.x ones that came installed with the computer, which are in /Library/Python. import turtle and import math worked, so it must be somewhere.

The test module in the Python stdlib doesn't contain a function called testEqual(). Its documentation starts with
Note: The test package is meant for internal use by Python only. It is
documented for the benefit of the core developers of Python. Any use
of this package outside of Python’s standard library is discouraged as
code mentioned here can change or be removed without notice between
releases of Python.
Are you sure that this guide you're following doesn't have its own test.py program that you're supposed to use instead?

When you write your testEqual() function make note of the directory you are working in. For instance on my mac I created a directory (folder) in documents so my path looks like: /Users/myName/Documents/python. Save your function (module) as testEqual.py and when you write you test.py script import testEqual after the shebang line. Once you have your scripts debugged your modules will be in a folder that python creates titled pycache don't remove that as it is compiled code. Now, as long as you are working in the same dir as your module you should not need to do anything other than use the import statement.

Python 2.7: "unresolved import: ConfigParser"

I recently wrote a Python 2.7 script (using PyDev on Eclipse) that took advantage of the built-in ConfigParser module, and the script works perfectly. But when I exported it and sent it to a colleague, he could not get it to work. He keeps getting an "unresolved import: ConfigParser" error even though we are using the exact same settings. This isn't supposed to happen as ConfigParser is built-in.
I've Googled everywhere but could not seem to find any working solution. Any help would be appreciated.

ConfigParser was renamed to configparser in python 3. Chances are he's using 3 and cannot find the old py2 name.
You can use:
try:
import configparser as ConfigParser
except ImportError:
import ConfigParser

To see what's happening it may be nice comparing on both computers which sys.path is being used (i.e.: put at the start of the module being run the code below and compare the output in each case):
import sys
print '\n'.join(sorted(sys.path))
Now, if the error is not when running the code (i.e.: it runs fine and you get no exceptions), and he gets the error only in PyDev, probably the interpreter configuration in his side is not correct and one of the paths printed through the command above is not being added to the PYTHONPATH (it could be that he's on a virtual env and didn't add the paths to the original /Lib or has added some path that shouldn't be there -- or even has some ConfigParser module somewhere else which is conflicting with the one from the Python standard library).

Reference objects in containing package?

I have a package, spam, that contains a contains the variable _eggs in __init__.py In the same package, in boiler.py, I have the class Boiler.
In Boiler, I want to refer to _eggs in the package’s __init__.py file. Is there a way that I can do this?

The most appropriate way to retrieve that value is via an explicit relative import:
from . import _eggs
However, one thing to keep in mind is that the following command line invocation will then fail to work:
python spam/boiler.py
The reason this won't work is the interpreter doesn't recognise any directly executed file as part of a package, so the relative import will fail.
However, with your current working directory set to the one containing the "spam" folder, you can instead execute the module as:
python -m spam.boiler
This gives the interpreter sufficient information to recognise where boiler.py sits in the module hierarchy and resolve the relative imports correctly.
This will only work with Python 2.6 or later - previous versions couldn't deal with explicit relative imports from main at all. (see PEP 366 for the gory details).
If you are simply doing import spam.boiler from another file, then that should work for any Python version that allows explicit relative imports (although it's possible Python 2.5 may need from __future__ import absolute_imports to correctly enable this feature)

Is there any way to get python omnicomplete to work with non-system modules in vim?

The only thing I can get python omnicomplete to work with are system modules. I get nothing for help with modules in my site-packages or modules that I'm currently working on.

Once I generated ctags for one of my site-packages, it started working for that package -- so I'm guessing that the omnicomplete function depends on ctags for non-sys modules.
EDIT: Not true at all.
Here's the problem -- poor testing on my part -- omnicomplete WAS working for parts of my project, just not most of it.
The issue was that I'm working on a django project, and in order to import django.db, you need to have an environment variable set. Since I couldn't import django.db, any class that inherited from django.db, or any module that imported a class that inherited from django.db wouldn't complete.

I get completion for my own modules in my PYTHONPATH or site-packages. I'm not sure what version of the pythoncomplete.vim script you're using, but you may want to make sure it's the latest.
EDIT: Here's some examples of what I'm seeing on my system...
This file (mymodule.py), I puth in a directory in PYTHONPATH, and then in site-packages. Both times I was able to get the screenshot below.
myvar = 'test'
def myfunction(foo='test'):
pass
class MyClass(object):
pass

Just ran across this on Python reddit tonight: PySmell. Looks like what you're looking for.
PySmell is a python IDE completion helper.
It tries to statically analyze Python source code, without executing it, and generates information about a project’s structure that IDE tools can use.

While it's important to note that you must properly set your PYTHONPATH environmental variable, per the the previous answer, there is a notable bug in Vim which prevents omnicompletion from working when an import fails. As of Vim 7.2.79, this bug hasn't been fixed.

Trouble-shooting tip: verify that the module you are trying to omni-complete can be imported by VIM. I had some syntactically correct Python that VIM didn't like:
:python import {module-name}
Traceback (most recent call last):
File "<string>", line 1, in ?
File "modulename/__init__.py", line 9
class empty_paranthesis():
^
SyntaxError: invalid syntax
Case-in-point, removing the parenthesis from my class definition allowed VIM to import the module, and subsequently OmniComplete on that module started to work.

I think your after the pydiction script. It lets you add your own stuff and site-packages to omni complete.
While your at it, add the following to your python.vim file...
set iskeyword+=.
This will let you auto-complete package functions e.g. if you enter...
os.path.
and then [CTRL][N], you'll get a list of the functions for os.path.