I am working on a package with the following structure.
Package
|- __init__.py
|- dir
|- subdir
|- moduleB.py
|- __init__.py
|- __init__.py
|- moduleA.py
main.py
main.py tries to import moduleA, which in turn imports moduleB. However, it runs in to an error when it tries to import moduleA, citing an error at a line of code that has since been changed.
I figured this would is a caching issue, so I deleted all of the pycache files in the package but it still fails.
What can I do to fix this, and what can I do to ensure that this does not remain a problem?
The actual code is
import tensorflow as tf
from UROP.data_structure.default_dictionary import DefaultDictionary
def default_distribution(shape, variation, name=''):
return tf.truncated_normal(
shape=shape,
stddev=variation,
name=name
)
#tdelaney was correct, and stepping through with a debugger revealed that the kernel I was using redirected me to its own private cache. I was using Hydrogen in Atom, and restarting the computer cleared the cache and solved the problem.
However, I was unable to find a long-term solution to the cached dependencies that would not require re-starting my computer.
Related
I am trying to make my own package so that I can use the files in a different folder. This package contains a few different modules and then the main module that imports all the others inside it. For example:
Folder
|- main.py
|- other.py
|- something.py
|- __init__.py
Inside the main.py I have the imports:
import other
import something
and it works just fine when running the file itself; however, I added the __init__.py file and tried to import it into a different folder. The package is recognized, but the main.py gives me the following error:
Exception has occurred: ModuleNotFoundError No module named
'univariate'
File "C:...\stats.py", line 8, in
import univariate
File "F:...\testing.py", line 7, in
from stats import stats
For clarification, the actual main file is called stats.py. This is my first experience trying to make a package so I might be missing something. Thank you.
You need to change your imports into relative imports
import .other
import .something
or to change it to absolute imports rooted to your project folder
import x.y.other
import x.y.something
you can read here about the imports
When you have a module that you're trying to import you don't need the ".py" part.
Having a folder with a init.py file (even a blank one) means that a project that contains that folder can import from it.
/myproject
| - /mymodule
| - |- stats.py
| - |- other.py
| - |- something.py
| - |- __init__.py
| - main.py
then in main.py all you need to do is import mymodule or from mymodule import stats
I always hate to FTFM someone, but here's a link to how to build packages from the official documentation. But, where this really starts to shine is when you need to package your module so that someone else can run it Digital Ocean has a pretty good tutorial here.
Project structure
I have the following folder structure
|
|- src
| |- mypackage
| | |- __init__.py
| | |- mymodule.py
| |- utils.egg
|- main.py
in mymodule.py file I can import the egg adding it to the sys.path as
import sys
sys.path.append('src/utils.egg')
import utils
When calling main.py everything works fine (python -m main).
Problem
The problem comes from pylint. First, it shows the following message in mymodule.py file
Unable to import 'utils' pylint(import-error)
if I ask for suggestions (CRTL + Space) when importing I got
utils.build
.dist
.utils
.setup
# |- suggestions
And from utils.utils I can acces the actual classes / functions in utils module. Of course if I import utils.utils, when executing the main script, an importing error pops up.
How can I configure my vscode setting in order fix pylint?
should I install the egg instead of copy it to the working folder?
Is my project's folder-structure ok, or it goes against recommended practices?
Extra info
In case you wonder the EGG-INFO/SOURCE.txt file looks like
setup.py
utils/__init__.py
utils/functions.py
utils.egg-info/PKG-INFO
utils.egg-info/SOURCES.txt
utils.egg-info/dependency_links.txt
utils.egg-info/top_level.txt
utils/internals/__init__.py
utils/internals/somemodule.py
utils/internals/someothermodule.py
Also, there aren't build nor dist folder in the egg.
This is an issue with Pylint itself and not the Python extension, so it will come down to however you need to configure Pylint.
As for whether you should copy an egg around or install it, you should be installing it into your virtual environment, or at least copying over the appropriate .pth file to make the egg directory work appropriately.
I have started learning python very recently. I do have a typescript and C# background. I was trying to understand the packages and imports statements. Following is the very basic folder structure I have created and I have few questions on this
app/
|-> pkg1/
|-> fib.py
|-> __init__.py
|- >pkg2/
|-> mul.py
|-> __init__.py
|-> app.py
ext/
|-> ext.py
|-> __init__.py
fib.py
------
from pkg2.mul import mul
// from ..pkg2.mul import mul -> Error: attempted relative import beyond top-level package
// but this should work as per docs https://docs.python.org/3/tutorial/modules.html#intra-package-references
// can someone tell why do I get this error?
def fib(n):
return mul(n) * 2
mul.py
------
def mul(n):
return n * 10
app.py
------
from pkg1.fib import fib
print(fib(1))
I have read Python treats any file as entry file.
Does python even know app.py is the entry point? No I guess.
Why does it work when I have given pkg2.mul in fib.py
Why does it work when I have given pkg1.fib in app.py
What if I want to include ext.py inside mul.py and app.py
As per folder structure, if app.py imports pkg1.fib (relative to app.py) is valid then fib.py must import mul in different way correct? why it is working for pkg2.mul? Is it actually Valid? pkg2/ is not relative to fib.py.
Am I missing something? In typescript we use relative imports and C# we use namespaces and include files but here I am pretty confused. Any help is greatly appreciated.
Your relative import attempt did not work because pkg1 and pkg2 are separate packages. Relative imports only work within one package.
Does python even know app.py is the entry point?
No. It's just some .py file sitting around in a directory.
Why does it work when I have given pkg2.mul in fib.py
Coincidence. You are in the app/ directory, and the first element of sys.path is the working directory. It likely won't work if you cd to some other directory.
Why does it work when I have given pkg1.fib in app.py
Same as above.
What if I want to include ext.py inside mul.py and app.py
You have to package ext and include it as a dependency of app.
As per folder structure, if app.py imports pkg1.fib (relative to app.py) is valid then fib.py must import mul in different way correct? Why it is working for pkg2.mul? Is it actually valid? pkg2/ is not relative to fib.py.
It's not importing relative to app.py. They are both importing from cwd (check the first element of sys.path displayed by using python -m site)
The basic issue is that your app is not package (because it is missing an __init__.py file) which means when you go two levels up to the app directory you aren't in a package any more, and thus the error stating that you are attempting to import beyond the top-level package.
See this section on intra-package references for more info on how you could restructure your package to allow what you are attempting to do. You also check out this section explaining the module search path for a little more context.
I’m looking for solution for designing my program.
My program consists of 3 blocks:
Classes
Functions
Other utilities
I want to structure my program this way:
program_folder/
main.py
classes_folder/
class_1.py
class_2.py
functions_folder/
set_of_func_1.py
set_of_func_1.py
utilities_folder/
set_of_utilities_1.py
set_of_utilities_1.py
I want to:
any scripts in «classes_folder» were able to import any of scripts in
«functions_folder».
any scripts in «functions_folder» were able
to import any of scripts in «utilities_folder».
all scripts were
normally used by main.py.
all scripts in «classes_folder»,
«functions_folder» and «utilities_folder» could be tested when worked
as «main» (if __name__ == “__main__”: some tests)
«program_folder»
could be in any place in my computer (there shouldn’t be dependency
on exact path to «program_folder»).
From all the above I thought I have to:
Change import search path for all scripts in «classes_folder»,
«functions_folder» and «utilities_folder».
Set current working
directory to «program_folder» for all scripts?
Is there a way I can do it?
Does my idea look good or have I put there some unexpected problems?
You can create a skeleton project like the following:
/path/to/project/
setup.py
my_project/
__init__.py
a/
__init__.py
b/
__init__.py
==> ./my_project/__init__.py <==
print('my_project/__init__.py')
==> ./my_project/a/__init__.py <==
import my_project
print('my_project/a/__init__.py')
==> ./my_project/b/__init__.py <==
import my_project.a
print('my_project/b/__init__.py')
==> ./setup.py <==
from distutils.core import setup
setup(name='my_project',
version='1.0',
description='my_project',
author='author',
packages=['my_project'])
Then you can install the project locally using pip install -e /path/to/project/ (the project folder is not copied, just gets registered; there's a dependency on the exact path, but this dependency is not hard-coded in project files themselves).
As the result, import my_project, import my_project.a etc. do that they mean:
$ python my_project/b/__init__.py
my_project/__init__.py
my_project/a/__init__.py
my_project/b/__init__.py
A common Python project structure could look like this:
project_name/
setup.py
requirements.txt
project_name/
__main__.py
classes/
__init__.py
class1.py
class2.py
functions/
__init__.py
functions.py
utils/
__init__.py
utils.py
Then, you could modify your imports from absolute to relative and run your package using something like:
$ /path/to/project_name> python -m project_name
Note that setup.py is only required if you want to install your package under some of your interpreters.
Note: see comments below also
I'm working on a Python application consisting of a core and multiple independent modules using the core. I'm having difficulty setting up relative imports of packages.
app
|- __init__.py
|- core
|- __init__.py
|- corefile.py
|- module1
|- __init__.py
|- main.py
The __init__.py files are empty. I'm running Python 2.7.1.
main.py
from .core import *
Running python main.py results in ValueError: Attempted relative import in non-package.
Similar questions: Ultimate answer to relative python imports, How to do relative imports in Python?, Relative imports in Python
Thanks for the help.
In short, you can only use relative imports from packages that are, themselves, imported.
For example, if you had:
$ cat run.py
from app.module1 import main
main.main()
$ python run.py
Then you could use a relative import in app/module1/main.py (although it would need to be from ..core import foo, because core/ is one level above main.py).
import sys
abs_filepath = '/home/n/Documents/IMPORTANT/deep_learning/drori_2018/ final_proj/Ryans_branch/StackGAN/'
# insert your absolute filepath above as abs_filepath = '/path/to/targ/dir'
sys.path.append(abs_filepath)
Please correct it if there are problems with doing the import this way
Other Answers:
Also please see here for a thorough answer about what's going on.