Trying to figure this out, because there is an inconsistency between when I run the code from Pycharm and from terminal.
Pycharm add automatically the current working directory; so if I add a module that is contained in my CWD, that is not in Pythonpath, it works just fine.
But when running from terminal, Python does complain, because my import statements refer to modules that are not reachable, because the CWD is not added to PYTHONPATH (I did verify this printing out the content of the variable, while running from Pycharm and from Terminal).
So at this point I am assuming that in my startup code, I need to add to Pythonpath the current directory, or this is not correct? I have no way to tell where the final user may put my code in; I just assume that the whole directory containing all my different modules, is located in a specific place.
To be more specific, this is where I am at:
my CWD when I run from Pycharm is /apps/myapp/logic/, I run the script after cd in that directory, and I call the script with ./myscript.py
The script has the #!/usr/bin/python3 line as first line, instead of running it with python3 -m myscript.py
The error I get, is when loading a module that is either in the same directory of my script (/apps/myapp/logic/) or one level above (/apps/myapp/); sadly the module load happen before my __main__ is running; so I can't add to sys.path the current directory from which the script run.
All these issues are not happening if I just run the script from Pycharm
After various trial and error, and thanks to the info that I did get from the comments; I did find 2 ways to solve the issue.
1) Create a shell script or another python script, which is adding the current directory (where all the files lives), and have no import in this file. Then the script call the script that has the main function.
2) On the top of the module, right after import sys, add the statement to add the path, in this way the current directory will be added to the PATH and it will be accessible, when the import try to access the module.
Neither look very nice, but this is all that I was able to find, to solve the issue. Pretty sure there is a more elegant way to do so
Related
I'm currently trying to import one of my modules from a different folder.
Like this...
from Assets.resources.libs.pout import Printer, ForeColor, BackColor
This import method works completely fine in PyCharm, however, when i try to launch the file in cmd or IDLE, i get this error.
ModuleNotFoundError: No module named 'Assets'
This is my file structure from main.py to pout.py:
- Assets
- main.py
- resources
- libs
- pout.py
Any clue about how i could fix this ?
Any help is appreciated !
Edit: The original answer was based on the assumption that the script you're running is within the folder structure given, which a re-read tells me may not be true. The general solution is to do
sys.path.append('path_to_Assets')
but read below for more detail.
Original answer
The paths that modules can be loaded from will differ between the two methods of running the script.
If you add
import sys
print(sys.path)
to the top of your script before the imports you should be able to see the difference.
When you run it yourself the first entry will be the location of the script itself, followed by various system/environment paths. When you run it in PyCharm you will see the same first entry, followed by an entry for the top level of the project. This is how it finds the modules when run from PyCharm. This behaviour is controlled by the "Add content roots to PYTHONPATH" option in the run configuration.
Adding this path programmatically in a way that will work in all situations isn't trivial because, unlike PyCharm, your script doesn't have a concept of where the top level should be. BUT if you know you'll be running the script from the top level, i.e. your working directory will be the folder containing Assets and you're running something like python Assets/main.py then you can do
sys.path.append(os.path.abspath('.'))
and that will add the correct folder to the path.
Appending sys path didn't work for me on windows, hence here is the solution that worked for me:
Add an empty __init__.py file to each directory
i.e. in Assets, resources, libs.
Then try importing with only the base package names.
Worked for me!
I im using Git bash to open jupyter lab and a notebook file. I want to import a file such as test.py, with a function such as test_func(x). The test.py is in another folder then the working directory. using pwd in the notebook i get something like "C:\Users\Documents\Code_folder\". I have added the path of the test.py using sys.path.insert(1, "C:\Users\Code\), where the test.py is located.
I then have no issues with importing the module, but if i add another module, test_func2(y), and i say run test.test_func2??, i cant find the function, and when running test.test_func??, i see that the output on line: File: "c:\users\code\". I belive is the lower case of the File that gets me the missing module.
Why does this happen, and can i change it in a simple say without changing all my codes?
Edit: test_func2 is another function in test.py
This may simply be an issue with how you're importing. I'm not sure of the internal mechanics of Jupyter, but in a terminal window if you change the module it has to be reloaded (reimported.) In Python3 the reload was moved to the imp module.
See stackoverflow:How do I unload (reload) a module?
For Jupyter, I assume you have the import test.py in a previous window. If you add a function to a .py file, just go back to that window and rerun the import...although I'm not sure that will guarantee a reload (since just re-running the command import test.py in the terminal Python would not work.)
I have a Python script which imports the datetime module. It works well until someday I can run it in a directory which has a Python script named datetime.py. Of course, there are a few ways to resolve the issue. First, I run the script in a directory that does not contain the script datetime.py. Second, I can rename the Python script datetime.py. However, neither of the 2 approaches are perfect ways. Suppose one ship a Python script, he never knows where users will run the Python script. Another possible fix is to prevent Python from search the current working directory for modules. I tried to remove the empty path ('') from sys.path but it works in an interactive Python shell but not in a Python script. The invoked Python script still searches the current path for modules. I wonder whether there is any way to disable Python from searching the current path for modules?
if __name__ == '__main__':
if '' in sys.path:
sys.path.remove('')
...
Notice that it deosn't work even if I put the following code to the beginning of the script.
import sys
if '' in sys.path:
sys.path.remove('')
Below are some related questions on StackOverflow.
Removing path from Python search module path
pandas ImportError C extension when io.py in same directory
initialization of multiarray raised unreported exception python
Are you sure that Python is searching for that module in the current directory, and not on the script directory? I don't think Python adds the current directory to the sys.path, except in one case. Doing so could even be a security risk (akin to having . on the UNIX PATH).
According to the documentation:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first
So, '' as a representation of the current directory happens only if run from interpreter (that's why your interactive shell test worked) or if the script is read from the standard input (something like cat modquest/qwerty.py | python). Neither is a rather 'normal' way of running Python scripts, generally.
I'm guessing that your datetime.py stands side by side with your actual script (on the script directory), and that it just happens that you're running the script from that directory (that is script directory == current directory).
If that's the actual scenario, and your script is standalone (meaning just one file, no local imports), you could do this:
sys.path.remove(os.path.abspath(os.path.dirname(sys.argv[0])))
But keep in mind that this will bite you in the future, once the script gets bigger and you split it into multiple files, only to spend several hours trying to figure out why it is not importing the local files...
Another option is to use -I, but that may be overkill:
-I
Run Python in isolated mode. This also implies -E and -s. In isolated mode sys.path contains neither the script’s directory nor the user’s site-packages directory. All PYTHON* environment variables are ignored, too. Further restrictions may be imposed to prevent the user from injecting malicious code.
I'm trying to run a python project from github I follow some suggestions about the problem like change from Python 3 to Python 2, or check if the init.py is in each folder..., but I can't solve it.
This is the error I got
Traceback (most recent call last):
File "C:/Users/Pulse/Desktop/foil-python-master/src/trimlogic/test/FamilyTreeTestCase.py", line 3, in
from trimlogic.test.helper import FoilTestCase
ImportError: No module named trimlogic.test.helper
It doesn't matter if I run python2 or python 3, neither from console (CMD) or PyCharm
This is the structure of the project:
I don't know specifically what your problem is, but can maybe help you find it.
Python does module imports by path. There is a magic environment variable called PYTHONPATH that tells it where to look for modules. So when you do import trimlogic.test.helper in a python file what happens is...
It looks at PYTHONPATH and gets a list of the directories in it.
It goes to the first one and looks to see if there is a trimlogic.py or a folder called trimlogic that includes the file __init__.py. If it finds a file trimlogic.py it will read it and look for a variable called test. If it is a folder, it will go into that folder and look for test.py or a folder called test with __init__.py in it. And so on. If it fails, it will check the rest of the directories in the PYTHONPATH variable the same way. Your command is failing because the src directory that contains the trimlogic folder is not on your PYTHONPATH.
There is a magic trick here. The current directory when you launch the python.exe is automatically added to the path. In a console, if you did cd C:/Users/Pulse/Desktop/foil-python-master/src and then tried running python.exe trimlogic/test/FamilyTreeTestCase.py your command would work as expected.
Further, Pycharm has some magic to help with this. First, if you right click the src directory and mark it as a sources root it will automatically run your python commands with that directory on the PYTHONPATH. You can also edit the run commands to change the working directory or the environment variables to be launched before executing commands.
I am experiencing an odd problem that I'm not quite sure how to tackle. I have a Python-selenium script that uses relative paths to log results to a text-file.
Here is the part of the script which sets up the log-file:
log_file = './demo-logfiles/log_file_template.txt'
sys.stdout = open('log_file_template.txt', 'a',)
As you can see, it uses a relative path to a folder. If I run this script as:
python demo.py firefox MAC, it runs flawlessly and the logfile gets sent to the proper folder.
If I run this exact Python script from within a larger shell-script, it returns an error that the './demo-logfiles/log_file_template.txt' doesn't exist.
I have found that if I change the script to '../demo-logfiles/log_file_template.txt' it works in the larger shell script, but stops working if I run it normally.
It either works in one, or the other. What is the reason for the relative directories being interpreted in different ways? I would not like to have two separate scripts for running in Python/shell.
The original python script is in the directory /blah/blah/DEMO/demo.py, and the the shell script that runs it is in /blah/blah/DEMO/demo-autotest/autotest_logger.sh
I have confirmed that this problem occurs for any script I try to run. I shouldn't have to change the original Python code to make it work with the shell script. I already accounted for it in the shell script, and it successfully runs the file.
Thanks.
You should never use a "." (or any relative path) in a directory path in a script unless you really mean you want to refer to the directory that the user is running the script from. If you want to refer to a location relative to the script that's running, you can do the following:
import os
import sys
directory = os.path.dirname(os.path.abspath(__file__))
sys.stdout = open(os.path.join(directory, "demo-logfiles", "log_file_template.txt"), "a")
Best practices side note: you should probably use the logging module rather than reassigning sys.stdout.
The term "relative directories" means a path is relative to something. You probably assume it's relative to the script which contains the path but that's not correct.
Relative paths are relative to the current directory of the process which interprets the script, i.e. the folder in which you started python. If you're in the shell, you can see the current directory with echo $PWD
If you start python in /blah/blah, then that becomes the current directory and all relative paths are relative to /blah/blah.
See the answer of David Hollman for how to get the path of the current script and then how to build paths relative to that.