I did some websearch, but all I found was frustration.
I have a project in a directory (lets call it) "projectdir", in which I have "main.py".
In projectdir I have a subdirectory called "otherstuff", In which I have "foo.py".
How do I import foo.py, so I can use its contents in main.py, without doing much of the work that python designers/implementors should have, and without relying on boilerplate files?
Or is that impossible in python?
You need to put a __init__.py file in otherstuff subdirectory, to mark it as a package. After, you can import your module using:
import subdirectory.foo
or
from subdirectory import foo
The __init__.py file can be empty. There is no other "clean" way to achieve that in python.
you need to include __init__.py in your otherstuff directory. This is to tell python to search there for imports.
The python documentation explains how the module/package import works. And is def worth the time reading it, despite its kind of long length
Related
#Background and Problem#
I'm trying to build a web-scraper to back up my social media accounts (summer project, I know it's useless).
I'm trying to create a nice class structure, so I've come up with the following structure (I accept critique, I'm pretty new to programming):
\social-media-backup
\Files
__init__.py
File.py
Image.py
Video.py
\SocialMedia
SocialMediaFeed.py
SocialMediaPost.py
\Instagram
__init__.py
\MediaTypes
__init__.py
GraphImage.py
GraphVideo.py
\SearchTypes
__init__.py
InstagramUser.py
\Twitter
\VSCO
(Twitter and VSCO are, for now, empty. Anything without extension and starting with , is a folder. Every file has a class with the same name as the file inside)
#Questions#
Where can I learn Python's packaging system definitively? Any book or web-site recommendation?
How would I import Image into GraphImage? How would I import File into SocialMediaPost?
What do I need to write in __init__.py so as to import SOME_PACKAGE and have it import every module? (e.g.: import Files and have Files.Image and File.Video accessible).
(I know there are a lot of questions. They're written in order of importance)
#To accomplish importing File into SocialMediaPost I've tried:#
from Files.File import File
from ...Files.File import File
import File
from File import File
And almost any combination you can imagine.
I always get an Unable to import, No module named '__main__.Files' or Attempted relative import beyond top-level package.
#Expected behavior#
I'm used to Java's way of doing this, and I cannot figure out how to do this in Python. It seems so messed up. I really miss just adding a package and a folder tree from where the compiler would run.
knocks desk THERE MUST BE A BETTER WAY
##THANKS!!##
Lots of stuff is written about this... however most guides focus on how you do it, not what to do.
How I do it (for small to medium-sized projects):
Do not mess with sys.path.
Have one "project root" directory with your modules / packages underneath (as you already do).
Use absolute imports always, except for "sister" modules.
Always run as module, i.e. using python -m foo.bar
Concrete example using your structure. Assuming that your entry point might be in \SocialMedia\SocialMediaFeed.py, use import statements:
from . import SocialMediaPost (sister module)
import Instagram (child module)
from Files import Image (other module)
and run using: python -m SocialMedia.SocialMediaFeed
By running as module, you always have the project root (social-media-backup) added as "search path". This way absolute imports refering to its subfolders always work. By the way, you can print out the module search path using import sys; print(sys.path).
Some of this might seem overcomplicated, but I found that following the above points pays out greatly as soon as you try to package up stuff for installation (keyword setup.py).
Edit: to answer 3rd question: Have __init__.py contain:
from . import File
from . import Image
from . import Video
I would second the comments by Damian and user2357112 - try to avoid name collisions between folder, file and class/function when creating modules.
You probably won't be able to import anything outside of the current working directory without adding it to your PYTHONPATH. Adding a folder to your PYTHONPATH environment variable means that python will always check that folder when importing modules, so you'll be able to import it from anywhere.
There is a good thread on this already that will put you in the right direction:
Permanently add a directory to PYTHONPATH?
(It's a lot to cover in one post)
I have a project with the following file structure:
root/
run.py
bot/
__init__.py
my_discord_bot.py
dice/
__init__.py
dice.py
# dice files
help/
__init__.py
help.py
# help files
parser/
__init__.py
parser.py
# other parser files
The program is run from within the root directory by calling python run.py. run.py imports bot.my_discord_bot and then makes use of a class defined there.
The file bot/my_discord_bot.py has the following import statements:
import dice.dice as d
import help.help as h
import parser.parser as p
On Linux, all three import statements execute correctly. On Windows, the first two seem to execute fine, but then on the third I'm told:
ImportError: No module named 'parser.parser'; 'parser' is not a package
Why does it break on the third import statement, and why does it only break on Windows?
Edit: clarifies how the program is run
Make sure that your parser is not shadowing a built-in or third-party package/module/library.
I am not 100% sure about the specifics of how this name conflict would be resolved, but it seems like you can potentially a). have your module overridden by the existing module (which seems like it might be happening in your Windows case), or b). override the existing module, which could cause bugs down the road. It seems like b is what commonly trips people up.
If you think this might be happening with one of your modules (which seems fairly likely with a name like parser), try renaming your module.
See this very nice article for more details and more common Python "import traps".
Put run.py outside root folder, so you'll have run.py next to root folder, then create __init__.py inside root folder, and change imports to:
import root.parser.parser as p
Or just rename your parser module.
Anyway you should be careful with naming, because you can simply mess your own stuff someday.
Sorry for asking my own question 2nd time, but i am totally stuck in import file in python.
I have a directory structure below:
|--test/foo.py
|--library #This is my PYTHONPATH
|--|--script1.py
|--|--library_1
|--|--|--script2.py
|--|--library_2
|--|--library_3
I am accessing library/library_1/script2.py from test/foo.py.
Here i am confused about what is the better approach. Generally all library folders or utility functions should be added to pythonpath.
This is a folder structure i am maintaining to differentiate utility functions and test scripts.
I tried putting __init__.py in library and library1 & then imported like from library1 import script2, but getting error as No module named script.
I have tried appending that path to system path as well.
Working: if i add another pythonpath like path/to/library/libray_1/. So should i do this for all folders which are inside library folder to make it work ?
Here's what you need to do:
|--test/foo.py
|--library #This is my PYTHONPATH
|--__init__.py
|--|--script1.py
|--|--library_1
|--|--|--__init__.py
|--|--|--script2.py
|--|--library_2
|--|--|--__init__.py
|--|--library_3
|--|--|--__init__.py
And inside the first __init__.py below library you need to do:
import library1
import library2
import script
Then, if library is your python path, you can do this within test/foo.py with no errors:
import library
library.library1.bar()
library.script.foo()
I've been writing in python for a couple of months now and I've never found a through explanation of how import works. I downloaded this folder with subfolders with python files in them. I'm trying to use one of these files and I'm loosing my mind. How do you properlly import a folder with all the files in it?
Any help would be greatly appreciated.
As written in the python documentation on modules:
If you have a folder sound looking like that :
sound/ Top-level package
__init__.py Initialize the sound package
effects/ Subpackage for sound effects
__init__.py
echo.py
surround.py
reverse.py
To import all files of effects folder :
from sound.effects import *
Note that to be able to import module, they have to contain an __init__.py file.
First, check to see if the subdirectories have a file named __init__.py file in them. Python will not recognize directories that do not contain these files.
Then, you will have to manually change the PYTHONPATH, which you can find in sys.path. You can find a great example here.
Edit: I'm not 100% sure this is what you were asking for. If you want to import ALL the python files in a directory, you will have to import them one by one. For example, given a directory like so:
parent/
__init__.py
runner.py
example.py
language.py
you would have to type
from parent import runner, example, language
or
from parent import * # this will also import __init__
You have to create a __init__.py file in the directory to make it a package. In this file you import all the symbols from the underlying files.
See http://docs.python.org/tutorial/modules.html (especially part 6.4 Packages) for further notes on that.
The parent folder must be either in PYTHONPATH or the folder path indicated in a file with extension .pth situated in a location in your path, usually in site-packages.
Then your package and all folders inside it from which you have to import need to have a file named __init__. This file can be used for program initialization but as a starting point it can be an empty file.
For example my program folder, situated in C:\python26 has the estructure:
programas\
.....package1\
.........__init__.py
.........module1.py
.........subpackage1\
.............__init__.py
.............module2.py
.....package2\
.........__init__.py
.........module3
.....__init__.py
.....lonelyscript1.py
.....lonelyscript2.py
file site-packages\site.pth contains:
C:\Python26\programas
I have several project directories and want to have libraries/modules that are specific to them. For instance, I might have a directory structure like such:
myproject/
mymodules/
__init__.py
myfunctions.py
myreports/
mycode.py
Assuming there is a function called add in myfunctions.py, I can call it from mycode.py with the most naive routine:
execfile('../mymodules/myfunctions.py')
add(1,2)
But to be more sophisticated about it, I can also do
import sys
sys.path.append('../mymodules')
import myfunctions
myfunctions.add(1,2)
Is this the most idiomatic way to do this? There is also some mention about modifying the PYTHONPATH (os.environ['PYTHONPATH']?), but is this or other things I should look into?
Also, I have seen import statements contained within class statements, and in other instances, defined at the top of a Python file which contains the class definition. Is there a right/preferred way to do this?
Don't mess around with execfile or sys.path.append unless there is some very good reason for it. Rather, just arrange your code into proper python packages and do your importing as you would any other library.
If your mymodules is in fact a part of one large project, then set your package up like so:
myproject/
__init__.py
mymodules/
__init__.py
myfunctions.py
myreports/
__init__.py
myreportscode.py
And then you can import mymodules from anywhere in your code like this:
from myproject.mymodules import myfunctions
myfunctions.add(1, 2)
If your mymodules code is used by a number of separate and distinct projects, then just make it into a package in its own right and install it into whatever environment it needs to be used in.