I'm using Python 3.95, and have two files:
shapes.py, which defines class Circle, class Square, and class Triangle
textures.py, which defines class Smooth and class Rough
I'd like to be able to do something like:
from .shapes import *
but I get error ImportError: attempted relative import with no known parent package.
All files are stored in the same folder, and I'm trying to execute Python from that folder, on Python files stored in that folder.
There are numerous posts on SO describing this error, and I've tried their solutions, but to no avail. It seems like some of this depends on Python 2 vs 3, some depends on which folder you're executing from, some depends on packages and modules (which I do not entirely understand).
Some things I've tried which have not worked:
Creating an __init__.py file
Using from shapes import * instead of from .shapes import *
Using from .shapes import Circle
Adding these lines:
import sys
import os
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))
This is likely a basic mistake: I'm not trying to build a module or package for distribution, I simply want to put different classes into different files, instead of all my classes in one giant file. How can I do so?
The current directory is only prepended to sys.path automatically if the script is the python file (not if the script is the first argument to python like python app.py).
I think that you can either:
add #! /usr/bin/env python at the start of the main file and make it executable with chmod +x main.py
append the right directoy to sys.path, as it seems to me that you are trying to append the parent dir. SCRIPT_DIR in your last attempt is already the current directory and you don't need to call dirname on it again.
There's no need for a relative import if your main script needs to use a module in the same folder. If you have a main.py and a shapes.py (containing for example a function fun1()), just use any of these:
import shapes
shapes.fun1()
from shapes import fun1
fun1()
from shapes import *
fun1()
Note that the latter is not preferable, since it's not at all clear what you're importing and if shapes.py is later changed to include something that shadows something from an earlier import, it may break your code in surprising ways. Surprise is bad.
If you're writing a package called shapes, create a folder called shapes, put an __init__.py in it, and a Python file which could be called shapes.py or anything you want, as you'll be importing what's relevant from the __init__.py.
Then shapes/__init__.py could be something like:
from .shapes import *
Here, using the * makes a bit more sense, although I'd still favour from .shapes import fun1. The . is not needed, but here it makes sense, because it is relative to another file that's also in the package.
And shapes/shapes.py might be something like:
from .other_mod import fun2
def fun1():
fun2()
Here, the . makes sense if shapes.py and other_mod.py are always going to be sitting next to each other in the package, but perhaps move around within the internal structure. Or perhaps you want to use the explicit . to avoid conflicts with other names.
If other_mod.py is something like:
def fun2():
print('hello')
You could write a main.py in the same folder where the shapes folder sits and the code with shapes imports above would work.
Can you try create new folder and add [shapes.py,textures.py,init.py] files into shapes folder
backend/
|
|------folder_shapes/shapes.py
|------folder_shapes/textures.py
|------folder_shapes/__init__.py
|------your_script.py
|
Then try import from your_script.py
Related
Over the years I have written a few hundred functions in matlab for space engineering purposes which I use on a daily basis. They are all nicely put in one folder ordered in subfolders, and in matlab I just have an addpath() command for the root folder in the startup.m file, and then I can use any of the functions right away after starting up matlab.
I am now trying to do the same in python.
As far as I understand in python I shouldn't have 1 file for every function like in matlab, but rather bundle all the functions together in 1 py file. Is this correct? I am trying to avoid this, since I have a strong preference for short scripts rather than 1 huge one, due to it being way more intuitive for me that way.
And then, once I have all my python scripts, can I place them anywhere in order to use them? Because I read that python works differently than matlab in this aspect, and scripts need to be in the working directory in order to import them. However, I want to load the scripts and be able to use them regardless of my active working directory. So I am guessing I have to do something with paths. I have found that I can append to pythonpath using sys.path.insert or append, however this feels like a workaround to me, or is it the way to go?
So considering I have put all my rewritten matlab fuctions in a single python file (lets call it agfunctions.py) saved in a directory (lets call it PythonFunctions). The core of my startup.py would then be something like (I have added the startup file to PYTHONSTARTUP path):
# startup.py
import os, sys
import numpy as np
import spiceypy as spice
sys.path.append('C:\Users\AG5\Documents\PythonFunctions')
import agfunctions as ag
Does any of this make sense? Is this the way to go, or is there a better way in python?
Well, the python package is probably the best way to solve your problem. You can read more here. Python packages have no need to be built and not always are created for sharing, so do not worry.
Assume you had this file structure:
Documents/
startup.py
PythonFunctions/
FirstFunc.py
SecondFunc.py
Then you can add file __init__.py in your PythonFunctions directory with next content:
__all__ = ['FirstFunc', 'SecondFunc']
Init file must be updated if you change filenames, so maybe it isn't best solution for you. Now, the directory looks like:
Documents/
startup.py
PythonFunctions/
__init__.py
FirstFunc.py
SecondFunc.py
And it's all - PythonFunctions now is a package. You can import all files by one import statement and use them:
from PythonFunctions import *
result_one = FirstFunc.function_name(some_data)
result_two = SecondFunc.function_name(some_other_data)
And if your startup.py is somewhere else, you can update path before importing as next:
sys.path.append('C:\Users\AG5\Documents')
More detailed explanations can be found here
There is no need to write all your matlab functions within one file. You can also keep (kind of) your desired folder structure of your library. However, you should write all functions in the deepest subfolder into one *.py file.
Suppose your MATLAB library is in the folder space_engineering and is set up like this:
space_engineering\
subfolder1\
functiongroup1\
printfoo.m
printbar.m
subfolder2\
...
subfolder3\
...
...
After doing addpath(genpath('\your_path\space_engineering')) all functions in subfolders subfolder* and functiongroup* are available in the global namespace like this
>> printfoo % function printfoo does only do fprintf('foo')
foo
I understand this is an behaviour your want to preserve in your migrated python library. And there is a way to do so. Your new python library would be structured like this:
space_engineering\
__init__.py
subfolder1\
__init__.py
functiongroup1.py
functiongroup2.py
subfolder2\
__init__.py
...
subfolder3\
__init__.py
...
...
As you can see the deepest subfolder layers functiongroup* of the MATLAB structure is replaced now by functiongroup*.py files, so called modules. The only compromise you have to allow for is that functions printfoo() and printbar() are now defined in this .py modules instead of having individual .py files.
# content of functiongroup1.py
def printfoo():
print(foo)
def printbar():
print(bar)
To allow for doing the same function calling as in MATLAB you have to make the function names printfoo and printbar available in the global namespace by adjusting the __init__.py files of each subfolder
# content of space_enginieering\subfolder1\__init__.py
from .functiongroup1 import *
from .functiongroup2 import *
as well as the __init__.py of the main folder
# content of space_engineering\__init__.py
from .subfolder1 import *
from .subfolder2 import *
from .subfolder3 import *
The from .functiongroup1 import * statement loads all names from module functiongroup1 into the namespace of subfolder1. Successively from .subfolder1 import * will forward them to the global namespace.
Like this you can do in an python console (or in any script) e.g.:
>>> sys.path.append('\your_path\space_engineering')
>>> from space_engineering import *
>>> printfoo()
foo
This way you can use your new python library in the same way as you former used the MATLAB library.
HOWEVER: The usage of from xyz import * statement is not recommend in python (see here why, it is similar to why not using eval in MATLAB) and some Pythonistas may complain recommending this. But for your special case, where you insist on creating a python library with MATLAB like comfort, it is a proper solution.
I have a Django app that is up and running, mostly in an easy-to-understand way. But I have one quirky problem.
My code sits one level below where urls.py and views.py sit. I have a number of Python files, some of which are used as imports across the actual code in the other impoPython files.
I have found that, as part of a Django app, I need to precede import filenames with a '.' for them to import. So, if I have a file of code called foo.py, and an import file called vars.py, this won't import right:
# foo.py
from vars import *
but this works:
# foo.py
from .vars import *
The problem comes if I try to be more Pythonic (and safe) and name the import, so if I import everything it gets a prefix. But neither of the following work at all:
# foo.py
import vars as v
import .vars as v
my presumption is Django (using virtualenv or through other means) defines certain paths to look up import files, and the '.' becomes necessary to point the Python interpreter to the same directory in which the importing file sits to locate the import files it needs to find. But, somehow, this 'breaks' the ability to give the imported file contents its own prefix (or namespace, I guess).
Any thoughts?
i need to call a function from from one python class to another which are at different directories.
I'm using Eclipse and PyDev for developing scripts.
sample.py
class Employee:
def meth1(self,arg):
self.arg=arg
print(arg)
ob=Employee()
ob.meth1("world")
main.py
class main:
def meth(self,arg):
self.arg=arg
print(arg)
obj1=main()
obj1.meth("hello")
I need to access meth1 in main.py
updated code
main.py
from samp.sample import Employee
class main:
def meth(self,arg):
self.arg=arg
print(arg)
obj1=main()
obj1.meth("hello")
After executing main.py it is printing "world" automatically without calling it.
my requirement is i need to call meth1 from main.py explicitly
please find my folder below
import is the concept you need here. main.py will need to import sample and then access symbols defined in that module such as sample.Employee
To ensure that sample.py can be found at import time, the path to its parent directory can be appended to sys.path (to get access to that, of course, you will first need to import sys). To manipulate paths (for example, to turn a relative path like '../samp' into an absolute path, you might want to import os as well and take a look at the standard library functions the sub-module os.path has to offer.
You will have to import the sample.py file as a module to your main.py file. Since the files are in different directories, you will need to use the __init__.py
From the documentation:
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.
Here is a related stack overflow question which explains the solution in more detail.
It's generally not a good idea to alter the sys.path variable. This can make scripts non-portable. Better is to just place your extra modules in the user site directory. You can find out what that is with the following script.
import os
import sys
import site
print(os.path.join(site.USER_BASE, "lib", "python{}.{}".format(*sys.version_info[0:2]), "site-packages"))
Place your modules there. Then you can just import as any other module.
import sample
emp = sample.Employee()
Later you can package it using distutils or setuptools and the imports will still work the same.
In the below hierachy, is there a convenient and universal way to reference to the top_package using a generic term in all .py file below? I would like to have a consistent way to import other modules, so that even when the "top_package" changes name nothing breaks.
I am not in favour of using the relative import like "..level_one_a" as relative path will be different to each python file below. I am looking for a way that:
Each python file can have the same import statement for the same module in the package.
A decoupling reference to "top_package" in any .py file inside the package, so whatever name "top_package" changes to, nothing breaks.
top_package/
__init__.py
level_one_a/
__init__.py
my_lib.py
level_two/
__init__.py
hello_world.py
level_one_b/
__init__.py
my_lib.py
main.py
This should do the job:
top_package = __import__(__name__.split('.')[0])
The trick here is that for every module the __name__ variable contains the full path to the module separated by dots such as, for example, top_package.level_one_a.my_lib. Hence, if you want to get the top package name, you just need to get the first component of the path and import it using __import__.
Despite the variable name used to access the package is still called top_package, you can rename the package and if will still work.
Put your package and the main script into an outer container directory, like this:
container/
main.py
top_package/
__init__.py
level_one_a/
__init__.py
my_lib.py
level_two/
__init__.py
hello_world.py
level_one_b/
__init__.py
my_lib.py
When main.py is run, its parent directory (container) will be automatically added to the start of sys.path. And since top_package is now in the same directory, it can be imported from anywhere within the package tree.
So hello_world.py could import level_one_b/my_lib.py like this:
from top_package.level_one_b import my_lib
No matter what the name of the container directory is, or where it is located, the imports will always work with this arrangement.
But note that, in your original example, top_package it could easily function as the container directory itself. All you would have to do is remove top_package/__init__.py, and you would be left with efectively the same arrangement.
The previous import statement would then change to:
from level_one_b import my_lib
and you would be free to rename top_package however you wished.
You could use a combination of the __import__() function and the __path__ attribute of a package.
For example, suppose you wish to import <whatever>.level_one_a.level_two.hello_world from somewhere else in the package. You could do something like this:
import os
_temp = __import__(__path__[0].split(os.sep)[0] + ".level_one_a.level_two.hello_world")
my_hello_world = _temp.level_one_a.level_two.hello_world
This code is independent of the name of the top level package and can be used anywhere in the package. It's also pretty ugly.
This works from within a library module:
import __main__ as main_package
TOP_PACKAGE = main_package.__package__.split('.')[0]
I believe #2 is impossible without using relative imports or the named package. You have to specify what module to import either by explicitly calling its name or using a relative import. otherwise how would the interpreter know what you want?
If you make your application launcher one level above top_level/ and have it import top_level you can then reference top_level.* from anywhere inside the top_level package.
(I can show you an example from software I'm working on: http://github.com/toddself/beerlog/)
I'm taking a look at how the model system in django works and I noticed something that I don't understand.
I know that you create an empty __init__.py file to specify that the current directory is a package. And that you can set some variable in __init__.py so that import * works properly.
But django adds a bunch of from ... import ... statements and defines a bunch of classes in __init__.py. Why? Doesn't this just make things look messy? Is there a reason that requires this code in __init__.py?
All imports in __init__.py are made available when you import the package (directory) that contains it.
Example:
./dir/__init__.py:
import something
./test.py:
import dir
# can now use dir.something
EDIT: forgot to mention, the code in __init__.py runs the first time you import any module from that directory. So it's normally a good place to put any package-level initialisation code.
EDIT2: dgrant pointed out to a possible confusion in my example. In __init__.py import something can import any module, not necessary from the package. For example, we can replace it with import datetime, then in our top level test.py both of these snippets will work:
import dir
print dir.datetime.datetime.now()
and
import dir.some_module_in_dir
print dir.datetime.datetime.now()
The bottom line is: all names assigned in __init__.py, be it imported modules, functions or classes, are automatically available in the package namespace whenever you import the package or a module in the package.
It's just personal preference really, and has to do with the layout of your python modules.
Let's say you have a module called erikutils. There are two ways that it can be a module, either you have a file called erikutils.py on your sys.path or you have a directory called erikutils on your sys.path with an empty __init__.py file inside it. Then let's say you have a bunch of modules called fileutils, procutils, parseutils and you want those to be sub-modules under erikutils. So you make some .py files called fileutils.py, procutils.py, and parseutils.py:
erikutils
__init__.py
fileutils.py
procutils.py
parseutils.py
Maybe you have a few functions that just don't belong in the fileutils, procutils, or parseutils modules. And let's say you don't feel like creating a new module called miscutils. AND, you'd like to be able to call the function like so:
erikutils.foo()
erikutils.bar()
rather than doing
erikutils.miscutils.foo()
erikutils.miscutils.bar()
So because the erikutils module is a directory, not a file, we have to define it's functions inside the __init__.py file.
In django, the best example I can think of is django.db.models.fields. ALL the django *Field classes are defined in the __init__.py file in the django/db/models/fields directory. I guess they did this because they didn't want to cram everything into a hypothetical django/db/models/fields.py model, so they split it out into a few submodules (related.py, files.py, for example) and they stuck the made *Field definitions in the fields module itself (hence, __init__.py).
Using the __init__.py file allows you to make the internal package structure invisible from the outside. If the internal structure changes (e.g. because you split one fat module into two) you only have to adjust the __init__.py file, but not the code that depends on the package. You can also make parts of your package invisible, e.g. if they are not ready for general usage.
Note that you can use the del command, so a typical __init__.py may look like this:
from somemodule import some_function1, some_function2, SomeObject
del somemodule
Now if you decide to split somemodule the new __init__.py might be:
from somemodule1 import some_function1, some_function2
from somemodule2 import SomeObject
del somemodule1
del somemodule2
From the outside the package still looks exactly as before.
"We recommend not putting much code in an __init__.py file, though. Programmers do not expect actual logic to happen in this file, and much like with from x import *, it can trip them up if they are looking for the declaration of a particular piece of code and can't find it until they check __init__.py. "
-- Python Object-Oriented Programming Fourth Edition Steven F. Lott Dusty Phillips