We Keep Coding

Yet another Python Relative import mishap

#Background and Problem#
I'm trying to build a web-scraper to back up my social media accounts (summer project, I know it's useless).
I'm trying to create a nice class structure, so I've come up with the following structure (I accept critique, I'm pretty new to programming):
\social-media-backup
\Files
__init__.py
File.py
Image.py
Video.py
\SocialMedia
SocialMediaFeed.py
SocialMediaPost.py
\Instagram
__init__.py
\MediaTypes
__init__.py
GraphImage.py
GraphVideo.py
\SearchTypes
__init__.py
InstagramUser.py
\Twitter
\VSCO
(Twitter and VSCO are, for now, empty. Anything without extension and starting with , is a folder. Every file has a class with the same name as the file inside)
#Questions#
Where can I learn Python's packaging system definitively? Any book or web-site recommendation?
How would I import Image into GraphImage? How would I import File into SocialMediaPost?
What do I need to write in __init__.py so as to import SOME_PACKAGE and have it import every module? (e.g.: import Files and have Files.Image and File.Video accessible).
(I know there are a lot of questions. They're written in order of importance)
#To accomplish importing File into SocialMediaPost I've tried:#
from Files.File import File
from ...Files.File import File
import File
from File import File
And almost any combination you can imagine.
I always get an Unable to import, No module named '__main__.Files' or Attempted relative import beyond top-level package.
#Expected behavior#
I'm used to Java's way of doing this, and I cannot figure out how to do this in Python. It seems so messed up. I really miss just adding a package and a folder tree from where the compiler would run.
knocks desk THERE MUST BE A BETTER WAY
##THANKS!!##

Lots of stuff is written about this... however most guides focus on how you do it, not what to do.
How I do it (for small to medium-sized projects):
Do not mess with sys.path.
Have one "project root" directory with your modules / packages underneath (as you already do).
Use absolute imports always, except for "sister" modules.
Always run as module, i.e. using python -m foo.bar
Concrete example using your structure. Assuming that your entry point might be in \SocialMedia\SocialMediaFeed.py, use import statements:
from . import SocialMediaPost (sister module)
import Instagram (child module)
from Files import Image (other module)
and run using: python -m SocialMedia.SocialMediaFeed
By running as module, you always have the project root (social-media-backup) added as "search path". This way absolute imports refering to its subfolders always work. By the way, you can print out the module search path using import sys; print(sys.path).
Some of this might seem overcomplicated, but I found that following the above points pays out greatly as soon as you try to package up stuff for installation (keyword setup.py).
Edit: to answer 3rd question: Have __init__.py contain:
from . import File
from . import Image
from . import Video

I would second the comments by Damian and user2357112 - try to avoid name collisions between folder, file and class/function when creating modules.
You probably won't be able to import anything outside of the current working directory without adding it to your PYTHONPATH. Adding a folder to your PYTHONPATH environment variable means that python will always check that folder when importing modules, so you'll be able to import it from anywhere.
There is a good thread on this already that will put you in the right direction:
Permanently add a directory to PYTHONPATH?
(It's a lot to cover in one post)

Create and import a custom python package - import doesn't work in the root

I'm totally new to Python, and I want to create my first Python library for peronal uses.
I'm using Python 2.7.5, and running the IDLE interface.
So far, what I understood from the documentation and related questions is that:
Python goes through a list of directories listed in sys.path to find scripts and libraries
The package directory must contain a __init__.py file, which can be empty
The module I want to create should be a modulename.py file with the code inside the package directory
(Sources: http://www.johnny-lin.com/cdat_tips/tips_pylang/path.html --- https://docs.python.org/2/tutorial/modules.html)
And here is what I tried that fails:
Created a personal package directory C:\....\pythonlibs
Created a subpackage dir C:\....\pythonlibs\package
Created the __init__.py file inside both folders
Created the mymodule.py file in the packacge dir
And then in the IDLE used this code:
import sys
sys.path.append(r'C:\....\pythonlibs')
First issue:
Currently I have to do this append every time I enter the IDLE. How can I keep the directory in sys.path permanently just as there are a lot of other directories there?
Then I tried importing my package:
import pythonlibs #fails!! why?
import pythonlibs.package #fails!! why?
import package #works
The error is: ImportError: No module named pythonlibs
Second issue?
This seems to be against the documentation, why can't I import from the root pythonlibs folder?

With line
sys.path.append(r'C:\....\pythonlibs')
you are instructing interpreter to start looking for modules (libraries) in this directory. Since this directory does not contain pythonlibs folder (the parent does), it can't import it.
Similarly - because it contains the module package, it can import it.

Better approach to use script inside nested directory PYTHONPATH

Sorry for asking my own question 2nd time, but i am totally stuck in import file in python.
I have a directory structure below:
|--test/foo.py
|--library #This is my PYTHONPATH
|--|--script1.py
|--|--library_1
|--|--|--script2.py
|--|--library_2
|--|--library_3
I am accessing library/library_1/script2.py from test/foo.py.
Here i am confused about what is the better approach. Generally all library folders or utility functions should be added to pythonpath.
This is a folder structure i am maintaining to differentiate utility functions and test scripts.
I tried putting __init__.py in library and library1 & then imported like from library1 import script2, but getting error as No module named script.
I have tried appending that path to system path as well.
Working: if i add another pythonpath like path/to/library/libray_1/. So should i do this for all folders which are inside library folder to make it work ?

Here's what you need to do:
|--test/foo.py
|--library #This is my PYTHONPATH
|--__init__.py
|--|--script1.py
|--|--library_1
|--|--|--__init__.py
|--|--|--script2.py
|--|--library_2
|--|--|--__init__.py
|--|--library_3
|--|--|--__init__.py
And inside the first __init__.py below library you need to do:
import library1
import library2
import script
Then, if library is your python path, you can do this within test/foo.py with no errors:
import library
library.library1.bar()
library.script.foo()

Python: Importing from module that is executed

I am creating module (with submodules). Lets call it lib. I am trying to make it work as following:
I am able to run it (there is lib.__main__). It uses lib.utils inside.
When executed part of its job is to load other file/module passed by user. Currently it does it by importlib.import_module( name ).
This loaded module also needs to use lib.utils.
I am having following choice:
In loaded module use import utils instead of import lib.utils. I find it somehow misleading and would like to aviod this.
Run module in any external way, even using file with only import lib.__main__ inside.
Only other thing I have tought of was doing sys.path.append(os.getcwd()). Not only it seems very dirty, but also makes log.utils module to load twice.
Is there anything that would allow me to execute lib.__main__, but require using import lib.utils in loaded module?

From the docs
If the script name refers to a directory or zipfile, the script name is added to the start of sys.path and the main.py file in that location is executed as the main module.
In your case, if you run python lib mymodule, lib is added to sys.path and __main__.py is executed. lib is not a package, its simply a directory in sys.path. __main__.py is not in a package and so package-relative imports don't work.
Since lib is in sys.path, its top level .py files can be imported directly and any subdirectories with __init__.py are importable packages. So, both __main__.py and mymodule could do import utils and get the same thing.
Now it gets confusing. Because you are sitting in lib's parent directory and because there is a lib.__init__.py, lib.utils is also valid. Its only that way because of your current directory (or maybe you added the directory to PYTHONPATH or something). So, you've got two different modules as far as python is concerned because you got there on two different paths.
The solution is to delete lib/__init__.py. lib shouldn't be package. Then you have the question of what to do with the modules like lib/utils.py. Normally, one would create a package directory and move the scripts there so that you get namespace encapsulation. Supposing you call that directory mypackages, then __main__.py and mymodule.py could both import mypackages.utils.

Import Python Issues

I've been writing in python for a couple of months now and I've never found a through explanation of how import works. I downloaded this folder with subfolders with python files in them. I'm trying to use one of these files and I'm loosing my mind. How do you properlly import a folder with all the files in it?
Any help would be greatly appreciated.

As written in the python documentation on modules:
If you have a folder sound looking like that :
sound/ Top-level package
__init__.py Initialize the sound package
effects/ Subpackage for sound effects
__init__.py
echo.py
surround.py
reverse.py
To import all files of effects folder :
from sound.effects import *
Note that to be able to import module, they have to contain an __init__.py file.

First, check to see if the subdirectories have a file named __init__.py file in them. Python will not recognize directories that do not contain these files.
Then, you will have to manually change the PYTHONPATH, which you can find in sys.path. You can find a great example here.
Edit: I'm not 100% sure this is what you were asking for. If you want to import ALL the python files in a directory, you will have to import them one by one. For example, given a directory like so:
parent/
__init__.py
runner.py
example.py
language.py
you would have to type
from parent import runner, example, language
or
from parent import * # this will also import __init__

You have to create a __init__.py file in the directory to make it a package. In this file you import all the symbols from the underlying files.
See http://docs.python.org/tutorial/modules.html (especially part 6.4 Packages) for further notes on that.

The parent folder must be either in PYTHONPATH or the folder path indicated in a file with extension .pth situated in a location in your path, usually in site-packages.
Then your package and all folders inside it from which you have to import need to have a file named __init__. This file can be used for program initialization but as a starting point it can be an empty file.
For example my program folder, situated in C:\python26 has the estructure:
programas\
.....package1\
.........__init__.py
.........module1.py
.........subpackage1\
.............__init__.py
.............module2.py
.....package2\
.........__init__.py
.........module3
.....__init__.py
.....lonelyscript1.py
.....lonelyscript2.py
file site-packages\site.pth contains:
C:\Python26\programas