I am in need of some help with the below. Using Python 3 code on Windows 10. Written in Spyder over Anaconda install and custom environment.
I have some code that runs in Spyder but not Anaconda Prompt. It gets a ModuleNotFoundError. The directory/file structure is like this:
my_python
smb_bau
etl
init.py
etl_globals.py
init.py
my_config.conf
my_connections.py
my_definitions.py
init.py
(underscores round init not displaying for some reason)
Some notes before the problematic code:
It's just test code for now that will be replaced with the real hive code later.
The code works when all the files are in one foler.
Th my_ prefixes are just to avoid any word that might cause issues while testing.
In my_connections I have this code:
class hive_connection:
def __init__(self):
self.my_name = None
self.my_town = None
import configparser
from smb_bau.my_definitions import CONFIG_PATH
config = configparser.ConfigParser()
config.read(CONFIG_PATH)
my_details = config["my_details"]
self.my_name = my_details["my_name"]
self.my_town = my_details["my_town"]
And this is what's in my_definitions:
import os
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
CONFIG_PATH = os.path.join(ROOT_DIR, 'my_config.conf')
(I gratefully stole this from another thread to solve the issue of getting config info from a different folder).
The file I run is etl_globals which contains this code:
from smb_bau.my_connections import hive_connection
hive = hive_connection()
print(hive.my_name, hive.my_town)
When I run etl_globals in Spyder it works perfectly. But it fails in Anaconda Prompt.
I open up Anaconda Prompt, activate my environment, navigate to the etl folder, and enter python etl_globals.py
The error message I get is:
Traceback (most recent call last):
File "etl_globals.py", line 1, in
from smb_bau.my_connections import hive_connection
ModuleNotFoundError: No module named 'smb_bau'
I don't understand why the import of modules works in Spyder but not Anaconda Prompt.
The overarching goal is to be able to not have all the files in one folder.
Any help would be gratefully appreciated.
Many thanks.
Edit:
Just to be sure what the issue is I simplified it to create a file called my_functions (in the smb_bau folder) with this in:
def fn_test():
print(5)
And a file in the etl folder called my_test with this in:
from smb_bau.my_functions import fn_test
fn_test()
And got the same error:
Traceback (most recent call last):
File "my_test.py", line 1, in
from smb_bau.my_functions import fn_test
ModuleNotFoundError: No module named 'smb_bau'
Why does from smb_bau.my_functions import xxx work in Spyder but not Anaconda prompt?
Also, this may be relevant: the only reason there is a folder called my_python at the top is that I was getting an error saying no module named smb_bau. So I moved it all down a level and then it would accept smb_bau as a module but only in Spyder).
I also just checked PYTHONPATH and the root folder is there.
I have only been using Python for a week so I'm sure there are other bit of my code that aren't great -- but it does work in the spyder client.
Thanks and sorry if any of this is unclear.
Related
I think I'm missing something obvious here. I cloned this repo, and now have this directory structure on my computer:
When I try to run python baby_cry_detection/pc_main/train_set.py, I get a ModuleNotFoundError.
Traceback (most recent call last):
File "baby_cry_detection/pc_main/train_set.py", line 10, in <module>
from baby_cry_detection.pc_methods import Reader
ModuleNotFoundError: No module named 'baby_cry_detection'
However, if I type python and enter the interactive shell and then type the command
from baby_cry_detection.pc_methods import Reader
it imports just fine with no error. I'm completely baffled. I'm using a virtualenv and both instances are using the same python installation, and I haven't changed directories at all.
I think sys.path could be the reason that the module is not found when python command is executed. Here is how we can check if that is indeed the case:
In the train_set.py file, add import sys; print(sys.path). Looking at the error, the path may contain /path/to/baby_cry_detection/baby_cry_detection/pc_main. If that is the case, then we have found the issue which is that baby_cry_detection.pc_methods will not be found in the directory that sys.path is looking into. We'll need to append the parent baby_cry_detection directory to sys.path or use relative imports. See this answer.
The reason that python prompt successfully imports the module could be because the prompt is started in the correct parent directory. Try changing the directory to baby_cry_detection/pc_main/ and try importing the module.
I'm trying to learn the ropes of starting a Python script from Excel VBA using xlwings 0.9.2. According to the docs, I need to change the PYTHONPATH to the path for my py file. I've seen several versions of this question, and various answers, but none have addressed my specific scenario. I have a basic test module "module1.py" and an Excel file "Book2.xlsm" located on the desktop which has a button to run this macro:
Sub macro1()
RunPython ("import module1.py; module1.run_all()")
End Sub
module1 goes like this:
import xlwings as xw
def run_all():
wb.Book.caller()
xw.sheets("Sheet1").range("A1").value = "Done!"
return
I then imported the "xlwings.bas" file and edited the VBA code to read PYTHONPATH = "C:\Users\bwamp\Desktop\module1\module1, which refers to the subfolder that holds "module1.py" (full path: "C:\Users\bwamp\Desktop\module1\module1\module1.py", for clarity). Press the button to run macro1 and I get the following error:
Error
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named 'module1.py'; 'module1' is not a package
Press Ctrl+C to copy this message to the clipboard.
OK
Any ideas what I'm doing wrong?
You have two errors: You need to import your module without the .py ending:
Sub macro1()
RunPython ("import module1; module1.run_all()")
End Sub
And it's xw.Book.caller() instead of wb.Book.caller().
I had the same error. Solved by downgrade to xlwings version 0.7.2
My project looks like this:
running-pycharm-project-at-cmd
- main.py
- c
- run_project.py
- z
- __init__.py
- the_module.py
- y
- __init__.py
- template.md
- the_module_module.py
- the_support_function.py
The contents of the .py files are shown below:
main.py
from c.run_project import run
print('running main.py...')
run()
c/run_project.py
from c.z.the_module import the_module_function, the_module_write_function
def run():
print('Running run_project.py!')
the_module_function()
# write a file:
the_module_write_function(read_file='./z/y/template.md', write_file='../README.md')
if __name__ == '__main__':
run()
c/z/the_module.py
from c.z.y.the_module_module import the_module_module_function
def the_module_function():
print('the_module_function is running!')
the_module_module_function()
pass
def the_module_write_function(read_file, write_file):
with open(read_file, 'r') as fid:
with open(write_file, 'w') as fid_out:
contents = fid.read()
contents.replace('{}', 'THE-FINAL-PRODUCT!')
fid_out.write(contents)
c/z/y/the_module_module.py
from .the_support_function import this_support_data
def the_module_module_function():
print('The module module function is running!')
print("Here is support data: {}".format(this_support_data))
pass
c/z/y/the_support_function.py
this_support_data = "I am the support data!"
GitHub repo to make replication easier: running-pycharm-project-at-cmd
The problem: in Pycharm load up the project with running-pycharm-project-at-cmd as the root. Then I right click and run the run_project.py file. Everything works fine. I cannot figure out how to run run_project.py from the command line in the same way. Creating main.py was a workaround suggested elsewhere, but it fails due to the relative references to the template file (in actual project, the y folder is a git submodule and therefore cannot have absolute references in it).
If you copy your main.py file into the c folder and rename it __init__.py, then you could run:
$ python -m c
The -m argument tells python to run a module or package (in this case c). Python will look in the c's folder for an __init__.py file and run that. You just need to ensure that the folder c is either in your PYTHONPATH or is a subfolder of the current working directory.
I circled back to this and was able to get it working. Note that I am on Windows 7, and some of this might be platform dependent. In order to get this working without changing any code, the key is setting PYTHONPATH environment variable before running. PyCharm does this automatically (by default, but it is configurable in the Run Configuration options).
Steps:
Recreating the issue:
Clone the github repo to G:\TEST.
Navigate to G:\TEST\running-pycharm-project-at-cmd-master\c
python run_project.py
Traceback (most recent call last):
File "run_project.py", line 1, in <module>
from c.z.the_module import the_module_function, the_module_write_function
ImportError: No module named 'c'
Simplest Solution (if able to run Python from the script location):
set the PYTHONPATH environment variable before running. e.g.
SET PYTHONPATH=G:\TEST\running-pycharm-project-at-cmd-master
Now python run_project.py
Running run_project.py!
the_module_function is running!
The module module function is running!
Here is support data: I am the support data!
To run from a remote location (this is definitely Windows specific):
Create a .bat file that navigates to correct working directory before running Python. i.e.:
START cmd /K "cd G:\TEST\running-pycharm-project-at-cmd-master\c & python run_project.py"
G:\TEST\myotherlocation
run_it.bat
Running run_project.py!
the_module_function is running!
The module module function is running!
Here is support data: I am the support data!
Using windows 7 and python 2.7. I have a package named Regetron in c:\Python27\Lib\site-packages\regetron which contains __init__.py and engine.py. When I try to run this library from the command prompt by typing regetron I get the following error:
Traceback (most recent call last):
File "C:\Python27\Scripts\regetron.py", line 6, in <module>
from regetron.engine import Regetron
File "C:\Python27\Scripts\regetron.py", line 6, in <module>
from regetron.engine import Regetron
ImportError: No module named engine
I added c:\Python27\Lib\site-packages\regetron to %PYTHONPATH% and can successfully import this module from other scripts located in other folders as well as the interactive prompt, but for some reason it refuses to run from the command prompt. What is going on?
You actually have two problems here. Fixing either one of them would actually eliminate your immediate error, but you need to fix both of them.
When I try to run this library from the command prompt by typing regetron
You shouldn't have a script named regetron and also have a module or package named regetron. Fix it by renaming your script. But if you want to understand why it's causing a problem:
The current working directory is always part of sys.path. So, you're in the directory with regetron.py in, and you run it with regetron. That means that regetron.py is on the path. So when you import regetron, it finds your script, not the package. Or, when you from regetron import engine, it finds your script, and tries to import a variable/function/class/whatever named engine from it, rather than finding the package and trying to import a module underneath it.
I added c:\Python27\Lib\site-packages\regetron to %PYTHONPATH%
Never add a package's directory to sys.path.
Since site-packages is already on your sys.path, the code in regetron/engine.py is already available as regetron.engine. You don't want it to also be available as engine. This will cause all kinds of problems.
So, rename you script to something else, remove regetron from %PYTHONPATH%, and everything will be fine.
But you may want to (re-)read the section on Packages in the tutorial.
UPDATE 10 Secs later
Fixed properly now, and thanks to JF and Gauden.
UPDATE
I have found a temporary fix by saving the IDLE file in the directory the other working file is in. (I would still like to know how to fix it entirely if I can.)
That's not a permanant fix, so if you want to try and help make it work wherever the file is saved, feel free.
This is the start of a python file:
#!/usr/bin/python
# -*- coding: utf-8 -*-
import wikipedia
import pagegenerators
import sys
import re
import pywikibot
from pywikibot import *
(You may have noticed it's a pywikipedia script, but I think that's irrelevent)
This file runs fine from the command line.
However, when I try and use IDLE to develop the script, or just use the IDLE interpreter, I get an error:
>>> import wikipedia
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
import wikipedia
ImportError: No module named wikipedia
I don't really have a clue why it isn't working.
I have also tried this:
>>> imp.find_module("wikipedia.py","f:/ake/pa/th/")
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
imp.find_module("wikipedia.py","f:/ake/pa/th/")
ImportError: No frozen submodule named f:/ake/pa/th/.wikipedia.py
The path given in the error log is the correct path to the wikipedia.py file, there's just that . before wikipedia.py.
I then tried adding the path to sys.path, but that didn't work either:
>>> import sys
>>> sys.path.append("c/users/adam/py")
#the same error...
Path to the module: `c:\users\joe_bloggs\py\wikipedia.pyc
Python executable: Command line:C:\Python27\python.exe, IDLE: C:\Python27\pythonw.exe
PYTHONPATH throws, in both:
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
print os.environ['PYTHONPATH'].split(os.pathsep)
File "C:\Python27\lib\os.py", line 423, in __getitem__
return self.data[key.upper()]
KeyError: 'PYTHONPATH'
OS: Windows 7
Python version: 2.7.2
A new PATH: IDLE, and Command Line:
C:\Program Files\Common Files\Microsoft Shared\Windows Live
C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live
C://Python27
C:\Program Files (x86)\IVI Foundation\VISA\WinNT\Bin
C:\Program Files (x86)\QuickTime\QTSystem\
C:\Program Files (x86)\Windows Live\Shared
EDIT The answer to the above question proved to be fairly simple, but I am editing this answer as a possible troubleshooting checklist for future reference, and as a checklist for others who may need to prepare questions of this nature in the future.
CLUE 1: What is the path to the module you are importing?
>>> import wikipedia
>>> print wikipedia.__file__
This will give you the path to the compiled module, and is one clue.
CLUE 2: What is the path to the Python executable?
(See also this question).
>>> import sys
>>> print sys.executable
Try this in the shell and in an IDLE script. If the two results are different, then you are using two Python interpreters and only one of them has a path that points to the wikipedia module.
CLUE 3: What is the sys.path?
Also repeat this in both shell and as a script in IDLE.
>>> print '\n'.join( sys.path )
(You may be able to use sys.path.append("d:/irectory/folder/is/in") to add that location to the sys.path. This should add that directory to the list of places Python looks for modules.)
CLUE 4: What is the PYTHONPATH and does it differ in the two environments?
(See also this answer).
Finally repeat this in both shell and as a script in IDLE.
>>> import os
>>> print '\n'.join( os.environ['PATH'].split(os.pathsep) )
Again note the two results (from shell and from IDLE) and see if there is difference in the PYTHONPATH in the two environments.
If all these tests prove inconclusive, I would add as much of this information as you can to your question as it would help give you specific further leads. Also add what OS you are using and any tracebacks that you get.
I had the same problem when trying to import a newly installed library on my Raspberry Pi. I followed all the instructions to install the library (Adafruit RHT Sensor) and it worked fine from the terminal. However, I couldn't get it to work from within IDLE.
It turned out that the problem was that the Raspberry Pi has both Python 2 and 3 installed. The install I'd done (using the 'python' command) only applied to Python 2. I had to perform another install using the 'python3' command to install it for Python 3. After that, I restarted IDLE and all worked fine.
The suggestion above to print the sys executable path helped point out the discrepancy:
import sys
print sys.executable