We Keep Coding

IPython reloading different module?

I made a test of shadowing Python's built-in string module with my own
module named 'string', to test module search path behavior. My custom
string script has only print('string' * 2) for contents. It is
located in the current directory as shown in code below.
Python 3.6.4 |Anaconda custom (64-bit)| (default, Jan 16 2018, 10:22:32) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: %pwd
Out[1]: 'C:\\Users\\stephen'
In [2]: import string
In [3]: string
Out[3]: <module 'string' from 'C:\\ProgramData\\Anaconda3\\lib\\string.py'>
In [4]: import imp
In [5]: imp.reload(string)
stringstring
Out[5]: <module 'string' from 'C:\\Users\\stephen\\string.py'>
Three questions about reload in IPython:
Why did reload think the module it actually imported (the local string.py) had already been imported? Is it just the sameness of file
name?
Why did reload import a different module than the one already imported? Did it skip the check of sys.modules dictionary, which was
checked first during original import?
Why did reload/IPython/Python not know it had pulled a fast one here, switching one module for another?
Edit: In going back and forth trying to take care of Stack Overflow complaining about my code block, I inadvertently lost some of what I meant to say. Here is what I can remember:
IPython is adding the built-in string module to sys.modules dictionary at startup (sys.modules is searched before sys.path), something the regular Python interactive prompt doesn't do. So I got a different behavior from the interactive prompt vs. IPython. With the interactive prompt, I always got the local string.py, since sys.path starts with current directory for module search. So, this much I understand about the difference between regular interactive (Anaconda) and IPython. The questions I listed are what remain confusing for me.

reload is supposed to repeat the process of locating the source code for the module it's reloading, and if it finds a different file from what the original import found, it's supposed to use the new file. After all, it needs to handle cases where a module was moved, or where a normal module was changed to a package or an extension module.
reload is not supposed to look at sys.modules and stop if it finds something. If it did that, it would perform no reloading!
The reason reload finds the local string.py file when the initial IPython-internal import didn't is because the import path has changed since the first import. You ran IPython in a way that doesn't cause Python itself to put the working directory on the module search path, and IPython imported the string module from the standard library under that configuration. Afterward, IPython placed the working directory on the module search path itself, mimicking regular interactive Python, so reload found the local string.py.

Tested on my Kernel
Looks you are using a local module name that shadows the name of a standard library or third party package or module that the application relies on. Python imports usually check from the the sys.path setting which might be set under Users\stephen. Hence when we reload it, it picks the first occurrence. Hope it helps.
For reference: https://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html

Installing Swampy for Python 3

I am currently running Python 3.2 on my computer and need to install Swampy for a book I am reading. Reading many pages and pages on a tutorial has left me further confused. I have downloaded 'swampy1.4'. I am trying to do this by following the set of instructions on this page- http://www.instructables.com/id/How-to-install-Python-packages-on-Windows-7/.
When trying to change directories to simplejson2.6.1 (I have an updated version of this software to the page) I am getting this error- 'The system could not find the specified path.'
Could anyone tell me where I am going wrong? It would be much appreciated.

To get swampy:
Doing a pip install won't install swampy for python 3.
This method should work:
Download the source code from here. Unzip the file to the directory you want. You're going to have to remember this directory.
Next, create a swampy.txt file in the following directory:
C:/Python32/Lib/site-packages
This assumes that you installed python in C:/Python32. You should modify this depending on where you installed python.
Remember the directory in which you unzipped the source code? Type the full path to the source code folder, not the directory you unzipped it into, into swampy.txt. After this, change the extension of the text file you just created from .txt to .pth. What this does is adds the source code to the search path of python.
You should be good to go now.

Quote from Swampy Installation Instructions:
You should see something like this:
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
>>>
If so, you have Python. If the version number starts with 2, you have
Python 2, which is the version of Python used in Think Python. If the
version number starts with 3, you have Python 3. You will not be able
to use Swampy with Python 3.
I suppose you want Swampy just to learn Python. In this case I would recommend you the official Python 3 tutorial.
If you want to continue reading the book don't be afraid of installing Python 2.7.3. Multiple versions of Python can coexist even on Windows assuming that you correctly setup the PATH variable.
There is also a newer version of Swampy (2.1.1).

You can now actually use swampy from source with Python 3.2. Please see http://www.greenteapress.com/thinkpython/swampy/install.html. It clearly says:
Swampy for Python 3 is not available as a package. But the source code is available in a zip file: Swampy source for Python 3: swampy-2.1.python3.zip
I tried to use it under Windows Vista following the instructions on the web page, and at least importing TurtleWorld worked just fine.

The link is correct but the explanation is vague even for an experienced Windows developer. It assumes too much knowledge of the Python installation process IMO.
e.g. "The simplest way to use this code is to unzip it in your home directory, cd into the unzipped directory and work there."
What is meant by 'home directory'? Then there is a reference to an 'unzipped directory', which I presume means the home directory. The change of name is confusing.
Nevertheless, say one unzips to C:\Python33\lib\swampy-2.1, and works from there. Whatever this means? I can only presume it means save your code in the swampy 'home directory'. It is not best practice to save your python code in a library directory. I use \dev\python\test\ but then
Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import swampy
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import swampy
ImportError: No module named 'swampy'
However, if the swampy directory is simply renamed to swampy (from swampy-2.1) then all is OK!
>>> import swampy
>>> from swampy.Gui import *
>>> g=Gui()
>>> g.title('Swampy.GUI')
>>> g.mainloop()
FYI this is my path (my dev drive is E: rather C:)
E:\Python33\Lib>path
PATH=E:\Python33\;E:\WINDOWS\system32;E:\WINDOWS;E:\WINDOWS\System32\Wbem;E:\Program Files\Microsoft SQL Server\100\Tool
s\Binn\;E:\Program Files\Microsoft SQL Server\100\DTS\Binn\;E:\WINDOWS\system32\WindowsPowerShell\v1.0;E:\Program Files\
Microsoft\Web Platform Installer\;E:\Program Files\Microchip\xc8\v1.21\bin;E:\Program Files\GtkSharp\2.12\bin
and I don't have a PYTHONPATH environment variable as suggested by other posts.

Installing Swampy for python 3 in Windows

I am a beginner python learner using The book 'Think Python' where I have to install module name Swampy. The link provided fro instruction and download has a tar.gz file. I found the python 3 version of the swampy with google search here. All setup tools for modules are under python 3. I am pretty lost, how do i install/use the module?
Thanks

You don't have to install Python modules. Just import them. (In fact, swampy is a package, which is basically a collection of modules.)
import swampy
Of course, Python has to know where to import them from. In this case, the simplest thing for you to do is to invoke Python from the directory containing the folder swampy, since the interpreter will first search for modules in the current directory. You can equivalently os.chdir to the directory after invoking Python from anywhere.
Don't worry about setuptools yet.

for python 2.7 I go to C:/Python27/Scripts dir and then try to "easy_install " or "pip install ".
can be a file.
if it does not help: try to unzip downloaded source files and execute "python setup.py install" from command promt

The link is correct but the explanation is vague even for an experienced Windows developer. It assumes too much knowledge of the Python installation process IMO.
e.g. "The simplest way to use this code is to unzip it in your home directory, cd into the unzipped directory and work there."
What is meant by 'home directory'? Then there is a reference to an 'unzipped directory', which I presume means the home directory. The change of name is confusing.
Nevertheless, say one unzips to C:\Python33\lib\swampy-2.1, and works from there. Whatever this means? I can only presume it means save your code in the swampy 'home directory'. It is not best practice to save your python code in a library directory. I use \dev\python\test\ but then
Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import swampy
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import swampy
ImportError: No module named 'swampy'
However, if the swampy directory is simply renamed to swampy (from swampy-2.1) then all is OK!
>>> import swampy
>>> from swampy.Gui import *
>>> g=Gui()
>>> g.title('Swampy.GUI')
>>> g.mainloop()
FYI this is my path (my dev drive is E: rather C:)
E:\Python33\Lib>path
PATH=E:\Python33\;E:\WINDOWS\system32;E:\WINDOWS;E:\WINDOWS\System32\Wbem;E:\Program Files\Microsoft SQL Server\100\Tool
s\Binn\;E:\Program Files\Microsoft SQL Server\100\DTS\Binn\;E:\WINDOWS\system32\WindowsPowerShell\v1.0;E:\Program Files\
Microsoft\Web Platform Installer\;E:\Program Files\Microchip\xc8\v1.21\bin;E:\Program Files\GtkSharp\2.12\bin
and I don't have a PYTHONPATH environment variable as suggested by other posts.

One easy method is by unpacking the contents into a directory of its own in the Python root directory. Then point Python to the location of the module in the file modulepaths.pth (You may need to create this file in the root directory of your python installation.)
You can either put the full path or a just a relative path. My modulepaths.pth looks like this with one relative and one full path:
swampy
c:\Python34\arduino

Using pyobjc imports in PyDev in Eclipse on Mac OS X

I've installed and configured PyDev version 1.6.5.2011020317 inside Eclipse, running on Mac OS X 10.6.6:
Version: Helios Service Release 1
Build id: 20100917-0705
I used 'Auto Config' to set up my Python interpreter: it correctly found /usr/bin/python (which is Python version 2.6.1) and added various system folders to the PYTHONPATH, including /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC. Now that path is the correct path to the Foundation module in OS X, as evinced by the command-line interpreter:
$ python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Foundation
>>> Foundation.__path__
['/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC/Foundation']
So why does PyDev complain about "Undefined variable from import: NSDictionary" on this class:
import Foundation
class MyClass(object):
def __init__(self, projectPath):
'''
Constructor
'''
self.projectDict = Foundation.NSDictionary.dictionaryWithContentsOfFile_(projectPath)
when I can use that class without any problem from the command-line interpreter?
Update: OK, I found out why it complains, which is that the Foundation module is using ScriptingBridge to dynamically generate the classes - presumably pydev isn't actually importing the module to see what classes are inside, it's just looking for .py[c] files. So let my question not be "why does this happen", but "what do I do to fix it"?

Why does this happen?: PyDev has no support for parsing the PyObjC scripting bridge metadata, and therefore has no way of introspecting / extracting symbols for many of the PyObjC classes.
What to do to fix it: In the PyDev source code there are several Python scripts which handle discovery of this metadata. The scripts are executed by Eclipse using the configured interpreter, and they return strings which are used to configure the interpreter, fill in completion lists, show usage tips, etc.
The scripts which seem relevant to your needs are:
interpreterInfo.py - called to obtain the list of directories and other default top-level imports for a given interpreter.
importsTipper.py - generates usage tips for a given symbol.
pycompletion.py - generates a list of completions for a given symbol.
Examples of calls to the above scripts using the system interpreter:
% /usr/bin/python interpreterInfo.py | grep PyObjC
|/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjCINS_PATH
Generate completions for a module:
>>> import pycompletion
>>> print pycompletion.GetImports('os')
##COMPLETIONS(/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/os.py,(EX_CANTCREAT, 3),(EX_CONFIG, 3),(EX_DATAERR, 3), ....
It seems possible to create a simple macobjc.py library with routines which detect and read the PyObjC.bridgesupport file(s). The PyDev scripts could be modified to call into this library in order to return the list of valid completions for those classes. You'll need to point Eclipse to a local copy of the PyDev source in order to develop and test your patches against these files. Once you're done you can send it upstream; I have to believe the PyDev folks would accept a well-written patch to support PyObjC completions.

What sets up sys.path with Python, and when?

When I run
import sys
print sys.path
on my Mac (Mac OS X 10.6.5, Python 2.6.1), I get the following results.
/Library/Python/2.6/site-packages/ply-3.3-py2.6.egg
...
/Library/Python/2.6/site-packages/ipython-0.10.1-py2.6.egg
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload
/Library/Python/2.6/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode
They are grouped into 5 categories.
/Library/Python/2.6/site-packages/*.egg
/Library/Python/2.6/site-packages
Frameworks/Python.framework/Versions/2.6/lib/python2.6
Frameworks/Python.framework/Versions/2.6/Extras/lib/python
PATH from PYTHONPATH environment variable.
And I can add more paths using the code
sys.path.insert(0, MORE_PATH)
What routines sets up those paths, and when?
Are some of the paths are built in python source code?
Is it possible that the paths inserted with 'sys.path.insert' are ignored? I'm curious about this, as with mod_wsgi, I found the paths are not found with 'sys.path.insert'. I asked another post for this question.
ADDED
Based on Michael's answer, I looked into site.py, and I got the following code.
def addsitepackages(known_paths):
"""Add site-packages (and possibly site-python) to sys.path"""
sitedirs = []
seen = []
for prefix in PREFIXES:
if not prefix or prefix in seen:
continue
seen.append(prefix)
if sys.platform in ('os2emx', 'riscos'):
sitedirs.append(os.path.join(prefix, "Lib", "site-packages"))
elif sys.platform == 'darwin' and prefix == sys.prefix:
sitedirs.append(os.path.join("/Library/Python", sys.version[:3], "site-packages"))
I also think that the directory name that has site.py (/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6 for my Mac) should be built into Python source code.

Most of the stuff is set up in Python's site.py which is automatically imported when starting the interpreter (unless you start it with the -S option). Few paths are set up in the interpreter itself during initialization (you can find out which by starting python with -S).
Additionally, some frameworks (like Django I think) modify sys.path upon startup to meet their requirements.
The site module has a pretty good documentation, a commented source code and prints out some information if you run it via python -m site.

From Learning Python:
sys.path is the module search path.
Python configures it at program
startup, automatically merging the
home directory of the top-level file
(or an empty string to designate the
current working directory), any
PYTHONPATH directories, the contents
of any .pth file paths you've
created, and the standard library
directories. The result is a list of
directory name strings that Python
searches on each import of a new file.

site.py is indeed the answers. I wanted to remove any dependencies on the old Python that is installed by default on my mac. This works pretty good, as 'site.py' is called each time the python interpreter is started.
For Mac, I manually added the following line at the end of main() in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/site.py:
sys.path = filter (lambda a: not a.startswith('/System'), sys.path)

Path has these parts:
OS paths that have your system libraries
current directory python started from
environmental variable $PYTHONPATH
you can add paths at runtime.
Paths are not ignored. But, they may not be found and that will not raise an error.
sys.path should only be added too, not subtracted from. Django would not remove paths.

Adding to the accepted answer, and addressing the comments that say a module shouldn't remove entries from sys.path:
This is broadly true but there are circumstances where you might want to modify sys.path by removing entries. For instance - and this is Mac-specific; *nix/Windows corollaries may exist - if you create a customised Python.framework for inclusion in your own project you may want to ignore the default sys.path entries that point at the system Python.framework.
You have a couple of options:
Hack the site.py, as #damirv indicates, or
Add your own sitecustomize module (or package) to the custom framework that achieves the same end result. As indicated in the site.py comments (for 2.7.6, anyway):
After these path manipulations, an attempt is made to import a module
named sitecustomize, which can perform arbitrary additional
site-specific customizations. If this import fails with an
ImportError exception, it is silently ignored.

Also note: if the PYTHONHOME env var is set, standard libraries will be loaded from this path instead of the default, as documented.
This is not a direct answer to the question, but something I just discovered that was causing the wrong standard libraries to be loaded, and my searches lead me here along the way.

You are using system python /usr/bin/python.
sys.path is set from system files at python startup.
Do not touch those files, in particular site.py, because this may perturb the system.
However, you can change sys.path within python, in particular, at startup :
in ~/.bashrc or ~/.zshrc:
export PYTHONSTARTUP=~/.pythonrc
in ~/.pythonrc:
write your changes to sys.path.
Those changes will be only for you in interactive shells.
For hacking at little risk for the system, install you own and more recent python version.