We Keep Coding

New module in Python

I've tried to create a new module in Python. It's git link : https://github.com/Sanmitha-Sadhishkumar/strman
After uploading and installing that using pip, I found that I could access that module as
import strman.strman as s
s.func_name
What are the changes to be made to access that as
import strman
strman.func_name

In your __init__.py file, you want to use a relative import.
from .strman import *
You have a strman package (the outer directory) and within it a strman module (the strman.py file). That's a perfectly common pattern. But without the relative import your __init__.py wasn't importing from deep enough in the hierarchy.
More generally, whenever you import from a sibling module within a project, you almost always should use a relative import, because it's explicit and avoids various complications, such as the example in your case.

You have to move the files into the main directory. Your directory will look like this
\myProject
- init.py
- strman.py
- main.py # <-- this would be the file your programming is
Read more about modules in the docs

Yet another Python Relative import mishap

#Background and Problem#
I'm trying to build a web-scraper to back up my social media accounts (summer project, I know it's useless).
I'm trying to create a nice class structure, so I've come up with the following structure (I accept critique, I'm pretty new to programming):
\social-media-backup
\Files
__init__.py
File.py
Image.py
Video.py
\SocialMedia
SocialMediaFeed.py
SocialMediaPost.py
\Instagram
__init__.py
\MediaTypes
__init__.py
GraphImage.py
GraphVideo.py
\SearchTypes
__init__.py
InstagramUser.py
\Twitter
\VSCO
(Twitter and VSCO are, for now, empty. Anything without extension and starting with , is a folder. Every file has a class with the same name as the file inside)
#Questions#
Where can I learn Python's packaging system definitively? Any book or web-site recommendation?
How would I import Image into GraphImage? How would I import File into SocialMediaPost?
What do I need to write in __init__.py so as to import SOME_PACKAGE and have it import every module? (e.g.: import Files and have Files.Image and File.Video accessible).
(I know there are a lot of questions. They're written in order of importance)
#To accomplish importing File into SocialMediaPost I've tried:#
from Files.File import File
from ...Files.File import File
import File
from File import File
And almost any combination you can imagine.
I always get an Unable to import, No module named '__main__.Files' or Attempted relative import beyond top-level package.
#Expected behavior#
I'm used to Java's way of doing this, and I cannot figure out how to do this in Python. It seems so messed up. I really miss just adding a package and a folder tree from where the compiler would run.
knocks desk THERE MUST BE A BETTER WAY
##THANKS!!##

Lots of stuff is written about this... however most guides focus on how you do it, not what to do.
How I do it (for small to medium-sized projects):
Do not mess with sys.path.
Have one "project root" directory with your modules / packages underneath (as you already do).
Use absolute imports always, except for "sister" modules.
Always run as module, i.e. using python -m foo.bar
Concrete example using your structure. Assuming that your entry point might be in \SocialMedia\SocialMediaFeed.py, use import statements:
from . import SocialMediaPost (sister module)
import Instagram (child module)
from Files import Image (other module)
and run using: python -m SocialMedia.SocialMediaFeed
By running as module, you always have the project root (social-media-backup) added as "search path". This way absolute imports refering to its subfolders always work. By the way, you can print out the module search path using import sys; print(sys.path).
Some of this might seem overcomplicated, but I found that following the above points pays out greatly as soon as you try to package up stuff for installation (keyword setup.py).
Edit: to answer 3rd question: Have __init__.py contain:
from . import File
from . import Image
from . import Video

I would second the comments by Damian and user2357112 - try to avoid name collisions between folder, file and class/function when creating modules.
You probably won't be able to import anything outside of the current working directory without adding it to your PYTHONPATH. Adding a folder to your PYTHONPATH environment variable means that python will always check that folder when importing modules, so you'll be able to import it from anywhere.
There is a good thread on this already that will put you in the right direction:
Permanently add a directory to PYTHONPATH?
(It's a lot to cover in one post)

Use __init__.py to modify sys path is a good idea?

I want to ask you something that came to my mind doing some stuff.
I have the following structure:
src
- __init__.py
- class1.py
+ folder2
- __init__.py
- class2.py
I class2.py I want to import class1 to use it. Obviously, I cannot use
from src.class1 import Class1
cause it will produce an error. A workaround that works to me is to define the following in the __init__.py inside folder2:
import sys
sys.path.append('src')
My question is if this option is valid and a good idea to use or maybe there are better solutions.
Another question. Imagine that the project structure is:
src
- __init__.py
- class1.py
+ folder2
- __init__.py
- class2.py
+ errorsFolder
- __init__.py
- errors.py
In class1:
from errorsFolder.errors import Errors
this works fine. But if I try to do in class2 which is at the same level than errorsFolder:
from src.errorsFolder.errors import Errors
It fails (ImportError: No module named src.errorsFolder.errors)
Thank you in advance!

Despite the fact that it's slightly shocking to have to import a "parent" module in your package, your workaround depends on the current directory you're running your application, which is bad.
import sys
sys.path.append('src')
should be
import sys,os
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)),os.pardir))
to add the parent directory of the directory of your current module, regardless of the current directory you're running your application from (your module may be imported by several applications, which don't all require to be run in the same directory)

One correct way to solve this is to set the environment variable PYTHONPATH to the path which contains src. Then import src.class1 will always work, regardless of which directory you start in.

from ..class1 import Class1 should work (at least it does here in a similar layout, using python 2.7.x).
As a general rule: messing with sys.path is usually a very bad idea, specially if this allows a same module to be imported from two different paths (which would be the case with your files layout).
Also, you may want to think twice about your current layout. Python is not Java and doesn't require (nor even encourage) a "one class per module" approach. If both classes need to work together, they might be better in a same module, or at least in modules at the same level in the package tree (note that you can use the top-level package's __init__ as a facade to provide direct access to objects defined in submodules / subpackages). NB : I'm not saying that your current layout is necessarily wrong, just that it might not be the simplest one.

There is also a solution that does not involve the use of the environment variable PYTHONPATH.
In src/ create setup.py with these contents:
from setuptools import setup
setup()
Now you can do pip install -e /path/to/src and import to your hearts content.
-e, --editable <path/url> Install a project in editable mode (i.e. setuptools "develop mode") from a local project path or a VCS url.

No, Its not good. Python takes modules in two ways:
Python looks for its modules and packages in $PYTHONPATH.
Refer: https://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH
To find out what is included in $PYTHONPATH, run the following code in python (3):
import sys
print(sys.path)
all folder containing init.py are marked as python package.(if they are subdirectories under PYTHONPATH)
By help of these two ways you can full-fill the need to create a python project.

Implementing Home Grown Plugins for a Python Service

I am writing a simple scheduling service. I don't want to hard-code all of the tasks it can schedule and instead would like to support plugins that can be dropped in a folder and loaded dynamically at runtime.
My plan is to have a JSON file (or any configuration file) that maps a task name to the location of a Python file (a module) which will have a class called Plugin. Pretty simple I thought. When someone schedules a task to run, they pass the task name and the time to run it. When the time elapses, the plugin is loaded (or reloaded) and is ran with any additional arguments passed to the scheduler.
I have been looking at the imp module to see how to load modules at runtime. I am not sure whether I want to list plugins using their physical location (file system path) or to use their module names like you'd see in a import statement. It seems imp wants to use physical location.
I got two different versions of this code "working". Here is one that uses importlib:
pluginName = self.__pluginLookup[pluginName]
module = import_module(pluginName)
module = reload(module) # force reload
plugin = module.Plugin()
return plugin
This is one I wrote using imp:
path = self.__pluginLookup[pluginName]
path, moduleName = split(path)
moduleName, extension = splitext(moduleName)
file, path, description = find_module(moduleName, [path])
with file:
module = load_module(moduleName, file, path, description)
plugin = module.Plugin()
return plugin
The problem I am running into is handling dependencies. If I have a plugin.py file that depends on a dependency.py file in the same folder, saying import dependency doesn't seem to work. Instead, it looks up the dependency from the PYTHONPATH.
How can I make the imports relative to the plugins themselves?

You could append path to sys.path:
import sys
sys.path.append(path)
where path is the directory containing the dependency.py.

If you have a plugins directory with an __init__.py, you can add that directory to sys.path. Then modules inside there can do from . import dependency to import another plugin. Or, if the plugin is itself a subpackage (i.e., a directory with its own __init__.py) then it can do from . import dep to import a dependency within the same plugin, or from .. import dep to import a dependency from the global plugins directory. With this setup you don't even need to use imp or the like; you can use the __import__ function, which works by module name.
One possible wrinkle, though, is you say the plugins directory will be "under the current working directory". What does that mean exactly? You mean you want people to be able to add plugins anywhere on the drive and still use them? It would be better to have one central plugins directory for your app, and add that to sys.path, and tell people to put their plugins there.

Managing resources in a Python project

I have a Python project in which I am using many non-code files. Currently these are all images, but I might use other kinds of files in the future. What would be a good scheme for storing and referencing these files?
I considered just making a folder "resources" in the main directory, but there is a problem; Some images are used from within sub-packages of my project. Storing these images that way would lead to coupling, which is a disadvantage.
Also, I need a way to access these files which is independent on what my current directory is.

You may want to use pkg_resources library that comes with setuptools.
For example, I've made up a quick little package "proj" to illustrate the resource organization scheme I'd use:
proj/setup.py
proj/proj/__init__.py
proj/proj/code.py
proj/proj/resources/__init__.py
proj/proj/resources/images/__init__.py
proj/proj/resources/images/pic1.png
proj/proj/resources/images/pic2.png
Notice how I keep all resources in a separate subpackage.
"code.py" shows how pkg_resources is used to refer to the resource objects:
from pkg_resources import resource_string, resource_listdir
# Itemize data files under proj/resources/images:
print resource_listdir('proj.resources.images', '')
# Get the data file bytes:
print resource_string('proj.resources.images', 'pic2.png').encode('base64')
If you run it, you get:
['__init__.py', '__init__.pyc', 'pic1.png', 'pic2.png']
iVBORw0KGgoAAAANSUhE ...
If you need to treat a resource as a fileobject, use resource_stream().
The code accessing the resources may be anywhere within the subpackage structure of your project, it just needs to refer to subpackage containing the images by full name: proj.resources.images, in this case.
Here's "setup.py":
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='proj',
packages=find_packages(),
package_data={'': ['*.png']})
Caveat:
To test things "locally", that is w/o installing the package first, you'll have to invoke your test scripts from directory that has setup.py. If you're in the same directory as code.py, Python won't know about proj package. So things like proj.resources won't resolve.

The new way of doing this is with importlib. For Python versions older than 3.7 you can add a dependency to importlib_resources and do something like
from importlib_resources import files
def get_resource(module: str, name: str) -> str:
"""Load a textual resource file."""
return files(module).joinpath(name).read_text(encoding="utf-8")
If your resources live inside the foo/resources sub-module, you would then use get_resource like so
resource_text = get_resource('foo.resources', 'myresource')

You can always have a separate "resources" folder in each subpackage which needs it, and use os.path functions to get to these from the __file__ values of your subpackages. To illustrate what I mean, I created the following __init__.py file in three locations:
c:\temp\topp (top-level package)
c:\temp\topp\sub1 (subpackage 1)
c:\temp\topp\sub2 (subpackage 2)
Here's the __init__.py file:
import os.path
resource_path = os.path.join(os.path.split(__file__)[0], "resources")
print resource_path
In c:\temp\work, I create an app, topapp.py, as follows:
import topp
import topp.sub1
import topp.sub2
This respresents the application using the topp package and subpackages. Then I run it:
C:\temp\work>topapp
Traceback (most recent call last):
File "C:\temp\work\topapp.py", line 1, in
import topp
ImportError: No module named topp
That's as expected. We set the PYTHONPATH to simulate having our package on the path:
C:\temp\work>set PYTHONPATH=c:\temp
C:\temp\work>topapp
c:\temp\topp\resources
c:\temp\topp\sub1\resources
c:\temp\topp\sub2\resources
As you can see, the resource paths resolved correctly to the location of the actual (sub)packages on the path.
Update: Here's the relevant py2exe documentation.

# pycon2009, there was a presentation on distutils and setuptools. You can find all of the videos here
Eggs and Buildout Deployment in Python - Part 1
Eggs and Buildout Deployment in Python - Part 2
Eggs and Buildout Deployment in Python - Part 3
In these videos, they describe how to include static resources in your package. I believe its in part 2.
With setuptools, you can define dependancies, this would allow you to have 2 packages that use resources from 3rd package.
Setuptools also gives you a standard way of accessing these resources and allows you to use relative paths inside of your packages, which eliminates the need to worry about where your packages are installed.