In order to import a project specific module somewhere located on your disk, one can easily append this directory to sys.path:
import sys
sys.path.append(some_module_path)
import some_module
However, the latter import now violates PEP E402 ("module level import not at top of file"). At least spyder tells me so. Is spyder here too picky?
In spyder there is the principal idea of a "project", where I assumed environments can be adjusted specific for this project. However, I have no clue, how to modify e.g. the sys.path depending on a spyder project.
How can I modify sys.path in a spyder project? Or is there a general python way of solving this issue?
You could put the sys.path extension in a separate module, e.g. _paths.py.
Contents of _paths.py:
import sys
sys.path.append(some_module_path)
sys.path.append(some_other_module_path)
# ...and so on...
And then in your main application:
import sys
import _paths
import some_module
some_module.some_func()
This solution puts your "project configuration" nicely in a single place (which makes it easy to maintain in the future), and complies with at least PEP8 (including E402) and pylint rules.
As alternative solution to the answer with a separate module if found this as working solution for me.
try:
sys.path.append(Path(__file__).parent.parent)
except IndexError:
pass
If I just use the sys.path.append(...), I get the warning, but using the try-catch block does not produce a warning.
I know this doesn't answer the question, but it may be helpful information.
You can import that module by directly specifying its path, without using sys.path.append
In Python 3 this is as simple as
import imp
some_module = imp.load_source('some_module', '/path/to/some_module.py')
More information here: How to import a module given the full path?
Related
According to the official documentation, os.path is a module. Thus, what is the preferred way of importing it?
# Should I always import it explicitly?
import os.path
Or...
# Is importing os enough?
import os
Please DON'T answer "importing os works for me". I know, it works for me too right now (as of Python 2.6). What I want to know is any official recommendation about this issue. So, if you answer this question, please post your references.
os.path works in a funny way. It looks like os should be a package with a submodule path, but in reality os is a normal module that does magic with sys.modules to inject os.path. Here's what happens:
When Python starts up, it loads a bunch of modules into sys.modules. They aren't bound to any names in your script, but you can access the already-created modules when you import them in some way.
sys.modules is a dict in which modules are cached. When you import a module, if it already has been imported somewhere, it gets the instance stored in sys.modules.
os is among the modules that are loaded when Python starts up. It assigns its path attribute to an os-specific path module.
It injects sys.modules['os.path'] = path so that you're able to do "import os.path" as though it was a submodule.
I tend to think of os.path as a module I want to use rather than a thing in the os module, so even though it's not really a submodule of a package called os, I import it sort of like it is one and I always do import os.path. This is consistent with how os.path is documented.
Incidentally, this sort of structure leads to a lot of Python programmers' early confusion about modules and packages and code organization, I think. This is really for two reasons
If you think of os as a package and know that you can do import os and have access to the submodule os.path, you may be surprised later when you can't do import twisted and automatically access twisted.spread without importing it.
It is confusing that os.name is a normal thing, a string, and os.path is a module. I always structure my packages with empty __init__.py files so that at the same level I always have one type of thing: a module/package or other stuff. Several big Python projects take this approach, which tends to make more structured code.
As per PEP-20 by Tim Peters, "Explicit is better than implicit" and "Readability counts". If all you need from the os module is under os.path, import os.path would be more explicit and let others know what you really care about.
Likewise, PEP-20 also says "Simple is better than complex", so if you also need stuff that resides under the more-general os umbrella, import os would be preferred.
Definitive answer: import os and use os.path. do not import os.path directly.
From the documentation of the module itself:
>>> import os
>>> help(os.path)
...
Instead of importing this module directly, import os and refer to
this module as os.path. The "os.path" name is an alias for this
module on Posix systems; on other systems (e.g. Mac, Windows),
os.path provides the same operations in a manner specific to that
platform, and is an alias to another module (e.g. macpath, ntpath).
...
Interestingly enough, importing os.path will import all of os. try the following in the interactive prompt:
import os.path
dir(os)
The result will be the same as if you just imported os. This is because os.path will refer to a different module based on which operating system you have, so python will import os to determine which module to load for path.
reference
With some modules, saying import foo will not expose foo.bar, so I guess it really depends the design of the specific module.
In general, just importing the explicit modules you need should be marginally faster. On my machine:
import os.path: 7.54285810068e-06 seconds
import os: 9.21904878972e-06 seconds
These times are close enough to be fairly negligible. Your program may need to use other modules from os either now or at a later time, so usually it makes sense just to sacrifice the two microseconds and use import os to avoid this error at a later time. I usually side with just importing os as a whole, but can see why some would prefer import os.path to technically be more efficient and convey to readers of the code that that is the only part of the os module that will need to be used. It essentially boils down to a style question in my mind.
Common sense works here: os is a module, and os.path is a module, too. So just import the module you want to use:
If you want to use functionalities in the os module, then import os.
If you want to use functionalities in the os.path module, then import os.path.
If you want to use functionalities in both modules, then import both modules:
import os
import os.path
For reference:
Lib/idlelib/rpc.py uses os and imports os.
Lib/idlelib/idle.py uses os.path and imports os.path.
Lib/ensurepip/init.py uses both and imports both.
Couldn't find any definitive reference, but I see that the example code for os.walk uses os.path but only imports os
I'm new to python and I don't know how to organize the project structure in the right way, so all auto imports would work in pycharm.
That's my current structure.
In PublisherSubscriberTest pycharm generated this import
from Rabbit.RabbitReceiver import RabbitReceiver
from Rabbit.RabbitSender import RabbitSender
But it's not working. That's the output.
ImportError: No module named Rabbit.RabbitReceiver
What have I done wrong?
I'm more familiar with java. And for example in java I would just create package with some classes and then I would be able to import them anywhere in my project. AFAIK it's not the same with python somehow.
Could someone explain this to me?
EDIT1:
Yes, I know about sys.path.append. I used to do it that way, but It seemed strange to me and i want to be able to do it without it.
import sys, os.path
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from Rabbit.RabbitReceiver import RabbitReceiver
from Rabbit.RabbitSender import RabbitSender
If you don't want to modify sys.path, the only way is to add -m flag when you run it
python -m messaging_system.tests.PublisherSubscriberTest
see How to fix "Attempted relative import in non-package" even with __init__.py
edit
OK, finally I found an ultimate answer: Relative imports for the billionth time
I suggest you read that post carefully, from which I learned a lot.
In short, if you want to do this, you have to add path-to-Rabbit to sys.path.
First of all let me tell you that I'm a new user and I'm just starting to learn Python in College so my apologies if this question is answered in other topic, but I searched and I can't seem to find it.
I received a file work.pyc from my teacher and he says I have to import it in my Wing IDE using the command from work import *, the question is I don't know where to put the file to import it.
It just says ImportError: No module named work.
Thank you
There are several options for this.
The most straightforward is to just place it in the same folder as the py file that wants to import it.
You may also want to have a look at this
if you're using the python interpreter (the one that lets you directly input python code into it and executes) you'll have to do this:
sys.path.append('newpath')
from work import *
where newpath is the path on your filesystem containing your work.pyc file
If you're working on a script called main.py in the folder project, one option is to place it at project/work.pyc
This will make the module importable because it's in the same working directory as your code.
The way Python resolves import statements works like this (simplified):
The Python interpreter you're using (/usr/bin/python2.6 for example, there can be several on your system) has a list of search paths where it looks for importable code. This list is in sys.path and you can look at it by firing up your interpreter and printing it out like this:
>>> import sys
>>> from pprint import pprint
>>> pprint(sys.path)
sys.path usually contains the path to modules from the standard library, additional installed packages (usually in site-packages) and possibly other 3rd party modules.
When you do something like import foo, Python will first look if there is a module called foo.py in the directory your script lives. If not, it will search sys.path and try to import it from there.
As I said, this explanation is a bit simplified. The details are explained in the section about the module search path.
Note 1:
The *.pyc you got handed is compiled Python bytecode. That means it's contents are binary, it contains instructions to be executed by a Python virtual machine as opposed to source code in *.py that you will normally deal with.
Note 2:
The advice your teacher gave you to do from work import * is rather bad advice. It might be ok to do this for testing purposes in the interactive interpreter, but your should never do that in actual code. Instead you should do something like from work import chop, hack
Main reasons:
Namespace pollution. You're likely to import things you don't need but still pollute your global namespace.
Readability. If you ever read someone elses code and wonder where foo came from, just scroll up and look at the imports, and you'll see exactly where it's being imported from. If that person used import *, you can't do that.
I'm using Python 2.7. I'm rather new to the python langauge. I have two python modules - "Trailcrest.py" and "Glyph.py", both in the same folder, but not in the Python27 folder.
I need to import "Trailcrest.py" into "Glyph.py", but I am getting the message that "no such module exists".
Additionally, whatever means I use to import the module needs to not be dependent on a solid-state path. This program is cross-platform, and the path can be changed depending on the user's preferences. However, these two modules will always be in the same folder together.
How do I do this?
If you have Trailcrest.py and Glyph.py in the same folder, importing one into the other is as simple as:
import Trailcrest
import Glyph
If this does not work, there seems to be something wrong with your Python setup. You might want to check what's in sys.path.
import sys
print sys.path
To elaborate a bit on Ferdinand Beyer's answer, sys.path is a list of file locations that the default module importer checks. Some, though not all installations of python will add the current directory or the directory of the __main__ module to the path. To make sure that the paths relative to a given module are importable in that module, do something like this:
import os.path, sys
sys.path.append(os.path.dirname(__file__))
But something like that shouldn't ever make it into a "production" product. Instead, use something like distutils to install the module's package into the python site-packages directory.
This can also be achieved using the environment variable PYTHONPATH which also influences Python's search path. This can be done in a shell script so that the Python files do not need to be altered. If you want it to import from the current working directory use the . notation in bash:
export PYTHONPATH=.
python python_prog.py
How do I set up module imports so that each module can access the objects of all the others?
I have a medium size Python application with modules files in various subdirectories. I have created modules that append these subdirectories to sys.path and imports a group of modules, using import thisModule as tm. Module objects are referred to with that qualification. I then import that module into the others with from moduleImports import *. The code is sloppy right now and has several of these things, which are often duplicative.
First, the application is failing because some module references aren't assigned. This same code does run when unit tested.
Second, I'm worried that I'm causing a problem with recursive module imports. Importing moduleImports imports thisModule, which imports moduleImports . . . .
What is the right way to do this?
"I have a medium size Python application with modules files in various subdirectories."
Good. Make absolutely sure that each directory include a __init__.py file, so that it's a package.
"I have created modules that append these subdirectories to sys.path"
Bad. Use PYTHONPATH or install the whole structure Lib/site-packages. Don't update sys.path dynamically. It's a bad thing. Hard to manage and maintain.
"imports a group of modules, using import thisModule as tm."
Doesn't make sense. Perhaps you have one import thisModule as tm for each module in your structure. This is typical, standard practice: import just the modules you need, no others.
"I then import that module into the others with from moduleImports import *"
Bad. Don't blanket import a bunch of random stuff.
Each module should have a longish list of the specific things it needs.
import this
import that
import package.module
Explicit list. No magic. No dynamic change to sys.path.
My current project has 100's of modules, a dozen or so packages. Each module imports just what it needs. No magic.
Few pointers
You may have already split
functionality in various module. If
correctly done most of the time you
will not fall into circular import
problems (e.g. if module a depends
on b and b on a you can make a third
module c to remove such circular
dependency). As last resort, in a
import b but in b import a at the
point where a is needed e.g. inside
function.
Once functionality is properly in
modules group them in packages under
a subdir and add a __init__.py file
to it so that you can import the
package. Keep such pakages in a
folder e.g. lib and then either add
to sys.path or set PYTHONPATH env
variable
from module import * may not
be good idea. Instead, import whatever
is needed. It may be fully qualified. It
doesn't hurt to be verbose. e.g.
from pakageA.moduleB import
CoolClass.
The way to do this is to avoid magic. In other words, if your module requires something from another module, it should import it explicitly. You shouldn't rely on things being imported automatically.
As the Zen of Python (import this) has it, explicit is better than implicit.
You won't get recursion on imports because Python caches each module and won't reload one it already has.