I have been using PyCharm since I began learning Python because of its amazing UI that helped me learn a great deal about the language. As I progress into more advanced projects, I am beginning to prefer using a text editor / command line combo so that I can build my own venv's and have better access to source control. My question is, how does PyCharm manage custom local packages that I created so that I can import them wherever I want in the directory? For instance a project that I have built exclusively in PyCharm that runs will raise numerous import errors when trying to run that same project in VS code, or even a command line shell (yes I did have the PyCharm created venv activated before running on both attempts). For further examples, here is the project structure I am confused about:
RootDirectory
package_1_folder
__init__.py
pckg_1_class.py
program_using_pckg_1_folder
class_using_pckg1class.py
venv
The above structure has no issues being imported and used in PyCharm, however VS code / Sublime when used with command prompt / gitbash will raise either an ImportError or a ModuleNotFound error. I have even gone as far as adding the desired packages to my laptops windows PATH, using sys.path.append (I know this is not good practice I was only trying to get it to work), and even modified the .pylintrc file with the project path with no success. Any help explaining why these errors are happening would be greatly appreciated :)
NOTE:
I have been able to use the packages in VS code as long as the program importing the module is located at the root directory level, but not in its own folder in the root directory. Again, this statement WILL work in PyCharm, I just want to know how PyCharm is able to achieve this.
After numerous attempts to locate how the system was keeping tack of module within the IDE, I found that my answer was not visible from the IDE. I found a .idea folder in my root directory that contains a few .xml documents that manage the directory including where to read modules from.
I'm trying to configure my IntelliJ Ultimate 2017 IDE to run Python unit tests. It's not working.
The project is a Gradle project, and most modules are Java/Scala. There are several Python modules though.
The Python plugin is installed and the Python facet added to the appropriate modules.
The Python Interpreter is set up fine. The IDE can actually move around in the code with a ctrl-click action.
Under 'Project Structure' -> '<mymodule>' -> dependencies both the interpreter and the <Module source> are included.
Each Python module has a 'content root'. Immediately under this is a src folder, under which there is a test folder. The contents of the test folder mirror the src folder, except that files are prepended with test_ to avoid Python name collisions.
The test configuration has both 'Add source/content roots to PYTHONPATH' options checked. The interpreter is specified.
If test is marked as a tests folder within the module settings, I can right click and 'run tests' for the tests in that folder. However, ImportErrors occur, because it can't find modules within the src directory.
There are __init__.py files in all the appropriate places.
IntelliJ gives you what it is running, and if I run that on the command, it works, something like:
python -m unittest discover -s /Users/nathanielford/myrepo/mymodule/src/tests -p test_util.py
It appears as though the root path that is being used is set to wherever the unittest file I'm running is: it doesn't use src as the path, and so it cannot find things to import. There is no obvious place to specify this behavior. It didn't seem to work like this in the past.
This started happening recently, after an IntelliJ update. It's possible some configuration became screwy and needs to be fixed, but I don't know what that is. It's also possible that some bug was introduced into IntelliJ, but I don't know how to identify that, either. What am I missing?
Had the same issue and resolved it by no marking the test folder as excluded but just moving it out of the source.
So if the structure is:
src
--->package
--->resources
test
mark in IntelliJ src folder as Sources and src/resources as Resources.
You tests will run from within IDEA.
As I am new to python, I am not sure whether this will affect the packaging. Doing a build though doesn't export the test folder.
Hope that helps.
I have similar problems: cannot import manually installed module.
I try to add classpath in Mac environment parameters and intellij SDKs, but it doesn't work.
My final solution is to add the classpath of the module to Run/Debug Configurations:
Open Run/Debug Configurations and select your unittest class:
Run --> Edit Configurations...
Add your module's classpath to Environment variables:
I've create a pydev project in eclipse.
At the top level of my workspace I can see these two files:
.project
.pydevproject
I can also see these in each of my subfolders that contain my actual projects.
At the top of my workspace there is also a
.metadata. folder.
What should I commit to source control?
Ie what can I delete and still be able to open the project with minimal effort (hopefully entirely automated regeneration of files)? If this was Visual Studios C++ project the answer would be to keep just the ".sln", "vcxproj" and "vcxproj.filters" because the "vs" folder and "suo" files will autogenerate on openning. I've tried to delete the ".metadata" folder, but after that nothing appears to load in my workspace.
Also, I am working with someone not using an IDE. What eclipse files do we need to update to keep in sync?
Disclaimer: I am not familiar with PyDev, just with Eclipse in general. You definitely should not check in the .metadata folder. That one is for your Eclipse workspace as a whole and contains your personal configuration. (That's why your workspace appeared empty after you deleted that folder.) In fact, you should not check in your workspace folder at all, but just the several project folders within it.
Whether to check in the .project files is sort of a matter of taste. Those contain project specific information and settings and with those its easier to import the project into Eclipse, but you can import the project without those, too, it's just a bit more work. If other developers are not using Eclipse, those are useless for them. In the worst case, your co-developers will delete those files from source control and when you update your project later, they are deleted on your end, too, messing up your project.
About deleting the files: Note that there is a difference between not checking files into version control and deleting them locally. So in short: Do not commit those files into version control, but don't delete them locally, either. Depending on what sort of version control you are using, you can set it to ignore those files.
What is the significance of this error message:
I just updated PyCharm and I don't know the significance of this error.
I understand that .xml is a markdown file (?), but without the .Rmd extension I'm not sure I can open it in RStudio to see when / by who it was created. It could have been a file created in the context of a Coursera course. A bit of research tells me that the .idea directory is a folder that python creates with every project (?).
Did you change anything in your project folder recently? Does the folder contain a .git subfolder?
Try going to the project folder root and issue this command:
git init
I have written a module (a file my_mod.py file residing in the folder my_module).
Currently, I am working in the file cool_script.py that resides in the folder cur_proj. I have opened the folder in PyCharm using File -- open (and I assume, hence, it is a PyCharm project).
In ProjectView (CMD-7), I can see my project cur_proj (in red) and under "External Libraries" I do see my_module. In cool_script.py, I can write
from my_module import my_mod as mm
and PyCharm even makes suggestion for my_mod. So far so good.
However, when I try to run cool_script.py, PyCharm tells me
"No module named my_module"
This seems strange to me, because
A) in the terminal (OS 10.10.2), in python, I can import the module no problem -- there is a corresponding entry in the PYTHONPATH in .bashrc
B) in PyCharm -- Settings -- Project cur_proj -- Project Interpreter -- CogWheel next to python interpreter -- more -- show paths for selected interpreter icon, the paths from PYTHONPATH do appear (as I think they should)
Hence, why do I get the error when I try to run cool_script.py? -- What am I missing?
Notes:
I am not declaring a different / special python version at the top of cool_script.py
I made sure that the path to my_module is correct
I put __init__.py files (empty files) both in my_module and in cur_proj
I am not using virtualenv
Addendum 2015-Feb-25
When I go in PyCharm to Run -- Edit Configurations, for my current project, there are two options that are selected with a check mark: "Add content roots to PYTHONPATH" and "Add source roots to PYTHONPATH". When I have both unchecked, I can load my module.
So it works now -- but why?
Further questions emerged:
What are "content roots" and what are "source roots"? And why does adding something to the PYTHONPATH make it somehow break?
should I uncheck both of those options all the time (so also in the defaults, not only the project specific configurations (left panel of the Run/Debug Configurations dialog)?
If your own module is in the same path, you need mark the path as Sources Root. In the project explorer, right-click on the directory that you want import. Then select Mark Directory As and select Sources Root.
So if you go to
-> Setting -> Project:My_project -> Project Structure,
Just the directory in which the source code is available and mark it as "Sources" (You can see it on the same window). The directory with source code should turn blue. Now u can import in modules residing in same directory.
PyCharm Community/Professional 2018.2.1
I was having this problem just now and I was able to solve it in sort of a similar way that #Beatriz Fonseca and #Julie pointed out.
If you go to File -> Settings -> Project: YourProjectName -> Project Structure, you'll have a directory layout of the project you're currently working in. You'll have to go through your directories and label them as being either the Source directory for all your Source files, or as a Resource folder for files that are strictly for importing.
You'll also want to make sure that you place __init__.py files within your resource directories, or really anywhere that you want to import from, and it'll work perfectly fine.
What I tried is to source the location where my files are.
e.g. E:\git_projects\My_project\__init__.py is my location.
I went to File -> Setting -> Project:My_project -> Project Structure and added the content root to about mention place E:\git_projects\My_project
it worked for me.
Always mark as source root the directory ABOVE the import!
So if the structure is
parent_folder/src/module.py
you must put something like:
from src.module import function_inside_module
and have parent_folder marked as "source folder" in PyCharm
I was getting the error with "Add source roots to PYTHONPATH" as well. My problem was that I had two folders with the same name, like project/subproject1/thing/src and project/subproject2/thing/src and I had both of them marked as source root. When I renamed one of the "thing" folders to "thing1" (any unique name), it worked.
Maybe if PyCharm automatically adds selected source roots, it doesn't use the full path and hence mixes up folders with the same name.
my_module is a folder not a module and you can't import a folder, try moving my_mod.py to the same folder as the cool_script.py and then doimport my_mod as mm. This is because python only looks in the current directory and sys.path, and so wont find my_mod.py unless it's in the same directory
Or you can look here for an answer telling you how to import from other directories.
As to your other questions, I do not know as I do not use PyCharm.
The key confusing step that must be done is to recreate the run configuration for the source file that you're trying to execute, so that the IDE picks up the new paths.
The way that actually worked for me was to go to Run/Edit Configurations..., select the configuration for the file that you're trying to run on the left side, uncheck the "Add source roots to PYTHONPATH" box, save, and then go back and check the box and save. THEN it would work.
This can be caused when Python interpreter can't find your code. You have to mention explicitly to Python to find your code in this location.
To do so:
Go to your python console
Add sys.path.extend(['your module location']) to Python console.
In your case:
Go to your python console,
On the start, write the following code:
import sys
sys.path.extend([my module URI location])
Once you have written this statement you can run following command:
from mymodule import functions
The solution for this problem without having to Mark Directory as Source Root is to Edit Run Configurations and in Execution select the option "Redirect input from" and choose script you want to run. This works because it is then treated as if the script was run interactively in this directory. However Python will still mark the module name with an error "no module named x":
When the interpreter executes the import statement, it searches for x.py in a list of directories assembled from the following sources:
The directory from which the input script was run or the current directory if the interpreter is being run interactively
The list of directories contained in the PYTHONPATH environment variable, if it is set.
An installation-dependent list of directories configured at the time Python is installed, in my case usr/lib/python3.6 on Ubuntu.
Content roots are folders holding your project code while source roots are defined as same too. The only difference i came to understand was that the code in source roots is built before the code in the content root.
Unchecking them wouldn't affect the runtime till the point you're not making separate modules in your package which are manually connected to Django. That means if any of your files do not hold the 'from django import...' or any of the function isn't called via django, unchecking these 2 options will result in a malfunction.
Update - the problem only arises when using Virtual Environmanet, and only when controlling the project via the provided terminal. Cause the terminal still works via the default system pyhtonpath and not the virtual env. while the python django control panel works fine.
PyCharm 2021.2.4 -- March 4th, 2022
Solved it by marking the source directory as "Source". Accomplished it by
right clicking on the source directory in the Project structure on the left of the IDE. I had a code source named src/.
navigate to "Mark Directory as".
select "Source" as directory type.
In short,
src -> Mark Directory as -> Source
This can occur if you are running a python file with the same name as one of its parent directories and try to import another
e.g.
Say you have files
my_project/processing/method1/processing.py
my_project/processing/algorithms/predict.py
And in processing.py you do something like:
from my_project.processing.algorithms.predict import Predict
it will throw the error
ln -s . someProject
If you have someDirectory/someProjectDir and two files, file1.py and file2.py, and file1.py tries to import with this line
from someProjectDir import file2
It won't work, even if you have designated the someProjectDir as a source directory, and even if it shows in preferences, project, project structure menu as a content root. The only way it will work is by linking the project as show above (unix command, works in mac, not sure of use or syntax for Windows).
There seems some mechanism where Pycharm does this automatically either in checkout from version control or adding as context root, since the soft link was created by Pycharm in a dependent project. Hence, just copying the same, although the weird replication of directory is annoying and necessity is perplexing. Also in the dependency where auto created, it doesn't show as new directory under version control. Perhaps comparison of .idea files will reveal more.
try installing the missing modules using the "terminal" on pycharm
python -m pip install your-module
Pycharm 2017.1.1
Click on View->ToolBar & View->Tool Buttons
On the left pane Project would be visible, right click on it and
press Autoscroll to source
and then run your code.
This worked for me.
The answer that worked for me was indeed what OP mentions in his 2015 update: uncheck these two boxes in your Python run config:
"Add content roots to PYTHONPATH"
"Add source roots to PYTHONPATH"
I already had the run config set to use the proper venv, so PyCharm doing additional work to add things to the path was not necessary. Instead it was causing errors.