This is my project structure:
- config
- data
- src
- resources
- db
- test
N.B.: I am using Python 3.9 and every folder that contains a .py file also has a __init__.py file
All the scripts I want to run are located in the /src folder and they used code from other scripts placed in the /src/resources folder (which is basically acting like a library).
Some of these scripts also read YAML files from the /config folder
Here is the problem, I cannot find a way to properly run these scripts from the command line, I am always getting errors like:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/runpy.py", line 185, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/local/lib/python3.8/runpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "/home/pi/crypto/src/ethMessage.py", line 4, in <module>
import update_db
File "/home/pi/crypto/src/update_db.py", line 1, in <module>
from db.mysql_main import insertValueAndFee
File "/home/pi/crypto/src/db/mysql_main.py", line 6, in <module>
from src.resources.parser import read_yaml
ModuleNotFoundError: No module named 'src'
I tried both with relative and absolute import, right now absolute import is what I am using (e.g. from src.resources.parser import read_yaml)
Which is the proper way to run scripts from the command line?
EDIT:
As you suggested, I added
sys.path.append( os.path.abspath(os.path.dirname(__file__)+'/..') )
to all the main scripts, and I am still getting a similar error:
Traceback (most recent call last):
File "src/ethMessage.py", line 6, in <module>
import update_db
File "/home/pi/crypto/src/update_db.py", line 1, in <module>
from db.mysql_main import insertValueAndFee
File "/home/pi/crypto/src/db/mysql_main.py", line 6, in <module>
from src.resources.parser import read_yaml
ModuleNotFoundError: No module named 'src'
To clarify, I am running my script from the global folder, which in my case is named "crypto".
I am also open to change the project structure with one that doesn't create problems.
If you want to refer to all of those packages by their root name, then all you have to do is add that folder to the Python path. So, for main program scripts in src, just add something like this:
import os
import sys
sys.path.append( os.path.abspath(os.path.dirname(__file__)+'/..') )
Now, the parent directory of your script will be on the path, no matter where you run it from. Now you can say
from src.resources.parser import read_yaml
If someone is still looking for a solution, I highly recommend not to bother with Python's imports: they are probably the worst part of the whole language.
Instead, if you want to use some files as a library, you should use setuptools to create a package from those files.
Then, you can install it locally or publish it on PyPi.
This way, you can import your library in a script just like another third-party module, (e.g. requests, selenium, ...), and things will work, instead of giving you a headache because a file is in a directory instead of another.
Related
Only for a .py file that is saved on my Desktop, importing some modules (like pandas) fail due to Module not found from an import that happens within the module.
This behaviour doesn't happen when the file is saved to a different location.
Working on a Mac and i made a test.py file that only holds: import pandas as pd
when this test.py is saved on my desktop it generates this error:
Desktop % python3 test.py
Traceback (most recent call last):
File "/Users/XXX/Desktop/test.py", line 2, in <module>
import pandas as pd
File "/Users/XXX/Desktop/pandas/__init__.py", line 22, in <module>
from pandas.compat import (
File "/Users/XXX/Desktop/pandas/compat/__init__.py", line 15, in <module>
from pandas.compat.numpy import (
File "/Users/XXX/Desktop/pandas/compat/numpy/__init__.py", line 7, in <module>
from pandas.util.version import Version
File "/Users/XXX/Desktop/pandas/util/__init__.py", line 1, in <module>
from pandas.util._decorators import ( # noqa
File "/Users/XXX/Desktop/pandas/util/_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly # noqa
File "/Users/XXX/Desktop/pandas/_libs/__init__.py", line 13, in <module>
from pandas._libs.interval import Interval
ModuleNotFoundError: No module named 'pandas._libs.interval'
the weird thing is that if i save the test.py file to any other location on my HD it imports pandas perfectly.
Same thing happens for some other modules. The module im trying to import seems to go oke but it fails on an import that happens from within the module.
running which python3 in console from either the desktop folder or any other folder results in:
/Users/XXXX/.pyenv/shims/python
python3 --version results in Python 3.10.9 for all locations.
You have a directory named pandas on your desktop.
Python trying to import from this directory instead of the global package named pandas.
You can also see that in the exception, look at the trace, from /Users/XXX/Desktop/test.py the code moves to /Users/XXX/Desktop/pandas/__init__.py and so on.
Just rename the name of the directory on your desktop.
For your own safety, you should not name your local directories with the same names as global packages.
The issue may be related to Desktop folder permission, check this
https://support.apple.com/en-gb/guide/mac-help/mchld5a35146/mac
From the article
Choose Apple menu > System Settings, then click Privacy & Security in the sidebar. (You may need to scroll down.)
Click Files and Folders.
For each app in the list, turn the ability to access files and folders in specific locations on or off.
You should be able to select permissions for Desktop folder as in this example
In my src/ directory of my project as of this post I have a config.py I want to be used in importing most of my python files in this PyObj-C and py2app project. As of the current sent link, everything works fine when ran with python3. However, once I build it with my Makefile using make build which runs python setup.py py2pp -A I get this error when running the build:
Traceback (most recent call last):
File "/Users/leif/PycharmProjects/shoutout/src/dist/Shoutout!.app/Contents/Resources/__boot__.py", line 149, in <module>
_run()
File "/Users/leif/PycharmProjects/shoutout/src/dist/Shoutout!.app/Contents/Resources/__boot__.py", line 143, in _run
exec(compile(source, script, "exec"), globals(), globals())
File "/Users/leif/PycharmProjects/shoutout/src/main.py", line 2, in <module>
import AppDelegate
File "/Users/leif/PycharmProjects/shoutout/src/AppDelegate.py", line 11, in <module>
from WindowController import mainWindow
File "/Users/leif/PycharmProjects/shoutout/src/WindowController.py", line 14, in <module>
from sutils.tasks import backendTasks as tasks
File "/Users/leif/PycharmProjects/shoutout/src/sutils/tasks.py", line 17, in <module>
from config import ymlDir, scheduleDir, configDir, resourcesURL
ModuleNotFoundError: No module named 'config'
2022-11-11 15:45:00.625 Shoutout![60555:5326242] Launch error
2022-11-11 15:45:00.625 Shoutout![60555:5326242] Launch error
See the py2app website for debugging launch issues
It seems my tasks.py, which has some classmethods in it for background tasks, in my utilities folder can't import it once it's compiled. It can't seem to access config.py once it's built for some reason or another
I have my setup.py here, which includes the config.py and the rest of the utilities folder's files and beyond that directory included in the data_files variable within the setup method in a list. I have also tried not using sutils/config.py, 'sutils/tasks.py' and so on and just did 'config.py', 'tasks.py', but this didn't work either.
I am stumped here. Is there a way to get around this, or a clear reason I am missing right under my nose somehow?
Thanks for any input or comments!
Again the specific commit of my src/ of this post I want to solve:
https://github.com/leifadev/shoutout/tree/398846a045bbb0c17bbd6905a578a2907349d9fa/src
I have a cloud instance of a Linux machine (openSuSE) with multiple users.
I have created a virtual environment and installed all my required libraries (including Klein).
I have two users "a" and "b".
While logged in as "a" and inside virtualenv, when I open python shell at home directory and type
import klein
it imports normally.
Now when I change directory to
/home/b/
and run the same (open python shell, import klein) while being in the same virtualenv, it gives me an error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/a/.local/lib/python3.6/site-packages/klein/__init__.py", line 3, in <module>
from klein._plating import Plating
File "/home/a/.local/lib/python3.6/site-packages/klein/_plating.py", line 16, in <module>
from .app import _call
File "/home/a/.local/lib/python3.6/site-packages/klein/app.py", line 19, in <module>
from twisted.internet import endpoints, reactor
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/endpoints.py", line 58, in <module>
from twisted.protocols.tls import TLSMemoryBIOFactory
File "/home/a/.local/lib/python3.6/site-packages/twisted/protocols/tls.py", line 63, in <module>
from twisted.internet._sslverify import _setAcceptableProtocols
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/_sslverify.py", line 158, in <module>
verifyHostname, VerificationError = _selectVerifyImplementation()
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/_sslverify.py", line 141, in _selectVerifyImplementation
from service_identity import VerificationError
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/__init__.py", line 7, in <module>
from . import cryptography, pyopenssl
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/cryptography.py", line 16, in <module>
from .exceptions import SubjectAltNameWarning
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/exceptions.py", line 21, in <module>
#attr.s
AttributeError: module 'attr' has no attribute 's'
Command "which python" gives same address at both location which is my virtualenv python address and that should be expected.
But what causes this weird python shell behavior.
Thank you
I solved it and a very shameful reason caused the error.
One of the modules Twisted uses is "attr" module. I had named one of my files attr.py and that is what was causing all the error.
I myself am not deleting this question if moderation has no problem, maybe somebody like me might be stuck at the same situation. It may help them.
Never name your python files same as that of any standard module unless overriding.
Also if your issue persists, then Jean's answer will definitely resolve it.
There can be multiple different Python packages that provide the same Python module. For example, there are at least two packages that provide the attr module:
https://pypi.org/project/attr/
https://pypi.org/project/attrs/
It's possible you've installed the wrong package based on the requirements. You can check what you have installed with pip freeze.
I have a tkinter app that I am compiling to an .exe via py2exe.
In the setup file, I have set it to include lxml, urllib, lxml.html, ast, and math.
When I run python setup.py py2exe in a CMD console, it compiles fine. I then go to the dist folder It has created, and run the .exe file.
When I run the .exe I get this popup window.
(source: gyazo.com)
I then procede to open the Trader.exe.log file, and the the contents say the following;
Traceback (most recent call last):
File "Trader.py", line 1, in <module>
File "lxml\html\__init__.pyc", line 42, in <module>
File "lxml\etree.pyc", line 12, in <module>
File "lxml\etree.pyc", line 10, in __load
File "lxml.etree.pyx", line 84, in init lxml.etree (src\lxml\lxml.etree.c:190292)
ImportError: cannot import name _elementpath
Included here is a copy of my setup.py file.
Please help me find the problem here. Thanks in advance.
Looks like py2exe doesn't realize it should include the lxml._elementpath module, which is conditionally imported by lxml.etree. You can tell it to include that module explicitly with the includes keyword argument in your setup.py.
setup(
options={'py2exe': {"includes": ["lxml._elementpath"]}}
)
Py2exe has made documentation of this error on this page: http://www.py2exe.org/index.cgi/WorkingWithVariousPackagesAndModules
They also offer a working solution.
I have a directory with two .py files in my C:\Python27\word_data called main.py and gethtml.py.
I want to import gethtml.py in my main.py, print def from that file, and I tried to do like this:
import gethtml
print gethtml.getHtmlText()
When I run this in a Python shell I get an error:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
execfile("word_rank/main.py")
File "word_rank/main.py", line 3, in <module>
import gethtml
ImportError: No module named gethtml
What am I missing?
You could check that the working directory for your python session is the directory containing your two python files. You can get Python to report the location of the current working directory as follows:
import os
print(os.getcwd())
Python will look in the current working directory (and in some directories in PATH) for the file you are trying to import. Not being able to find the file would give the error above.