I am new to Pycharm and need some help. I am working on a project that makes use of a large library of modules (specifically, Schrodinger; which allows for a lot of cool chemistry programs). Schrodinger requires the use of Python 2.7, if that makes any difference.
There are too many modules to install to the project directory. When I move the project directory to the location of the modules, my script becomes stuck on 'initializing'. I have attempted to import it as a package to no avail.
I have also tried to use the sys.path command, however a lot of the modules make use of other modules as well. So I that has become a pain very quickly.
How can I use these modules within Pycharm? And if there is no easy way, do you have a recommendation for an IDE that does have this feature?
Thanks
Pycharm doesn't identifies user defined modules which are not imported to Pycharm.
I usually mask the module as a Sources Root see the picture for more details. if the modules are in same project.
Alternative way: In your case import the external modules using File -> Open modules with open -> open in current window -> add to currently opened project this looks like two different projects. Now you can mark Sources Root for the complete module (i.e. learning) which you have imported.
import stackoverflow
Now pycharm can identifies the user defined modules.
Related
Basically, I can only reference my other files as modules when they are in a very specific location:
C:\Users\Dave\Desktop\Programming\Python.
If I want to create a new folder for a large project with multiple modules, say
C:\Users\Dave\Desktop\Programming\Python\Project1,
I can no longer import any modules and keep getting a ModuleNotFoundError. I've looked into it and it seems I need to add that folder to the Python Path somehow, but I couldn't find any answers on how to do it. My computer runs on Windows 10 if that matters.
I think the immediate solution to your problem/the answer to your question would be to use sys.path.append() at the top of your script.
import sys
sys.path.append("<ABSOLUTE/PATH/TO/YOUR/CUSTOM/MODULES/FOLDER>")
import custom_module
This isn't an ideal solution for any kind of prod use, however. Based on what you wrote in your question, this may not be what you're looking for. More info might help to craft a more stable solution:
Are you using virtual environments when running python? Do you just run python script.py or is there a specific version of python you're using that's installed elsewhere than the common C:\Program Files\Python?
When you say that you work on a project with multiple modules, does that mean there are custom modules that you/someone wrote or is it just that that project uses non-standard library modules, i.e. you had to pip install them?
Can we get an example of the code you're running and the folder structure of your project/where the modules are that you need?
I would like to use PyCharm Community Edition 2020.1.2 x64 on Windows 10 as IDE for writing QGIS plugins. I need to base on old version - QGIS 2.18, so I need python 2.7, qt4, and also core modules from QGIS 2.18. I downloaded QGIS 2.18 from official site. It provides a ready to use python environment as .bat script, that sets all needed python paths. So I used path D:\Program Files\QGIS 2.18\bin\python-qgis-ltr.bat to set main python interpreter for my project.
After that I was able to run python console in PyCharm, import all QGIS modules, and any classes from QGIS standard collection like QgsFeature, QgsVectorLayer etc. Everything was fine untill I tried to import this classes from PyCharm editor in .py file as from qgis.core import QgsFeature. In editor PyCharm sees the package qgis.core and I'm able to import it, but PyCharm's code completion inside the module sees only classes that are saved in D:\Program Files\QGIS 2.18\apps\qgis-ltr\python\qgis\core\__init__.py e.g. QgsEditError class, but all standard QGIS class underlines on red.
Python console ran on exactly the same interpreter see the same sources as are visible in "External Libraries" tab in PyCharm, so command:
qgis.core.__path__ returns ['D:\\Program Files\\QGIS 2.18\\apps\\qgis-ltr\\python\\qgis\\core'],
qgis.core.__file__ returns 'D:\\Program Files\\QGIS 2.18\\apps\\qgis-ltr\\python\\qgis\\core\\__init__.pyc,
dir(qgis.core) returns full list of QGIS standard core classes.
If I run the .py script, in which PyCharm's code validator is showing incorrect import object by red underline, the script is executed without any error and I can easly print the imported QgsFeature object. Why my PyCharm's editor see different content of qgis.core module then console ran also from IDE on exactly the same interpreter?
Allow me to qualify myself. I lost interest for adventures and trail blazing in computing some time ago. I won some big battles and lost some too. And I unnecessary wasted a lot of time that could have been better spent. Nowadays, I just wait a reasonable period of time before upgrading anything, and wait a spell before downloading the newest anything, and pretty much avoid trying to do things outside a conventional path.
So your post got me curious and I looked around some. I found very little information about PyCharm being used building QGIS plugins. That pretty much tells me all I want to know: that it's not real common for PyCharm to be used in building QGIS plugins.
But there was quite abit of information about building QGIS plugins with python. There's even a QGIS Tutorial & Tips on the subject. QGIS suggests using the Qt Creator and further goes on to say that QGIS itself is written using the QT framework. "For plugin development, we use an application called Qt Creator."
If you overcome your current predicament I'd like to hear how you accomplished it.
For what it's worth, here's the link to the QGIS Tutorials & Tips.
https://www.qgistutorials.com/en/docs/building_a_python_plugin.html
Finally, the problem solved itself. It was something wrong with building a binary skeletons of C++ libraries, what PyCharm does every time the application is launched. My PyCharm didn't want to do it and I found many similar problems with other libraries (e.g. Qt) that people were struggling with. After two days my PyCharm just fixed itself and created all needed binary skeletons for my QGIS libraries.
This issue has been driving me insane for the past few days.
So basically, I'm trying to port over a Pure Python project to a proper PyCharm project. This is to basically improve code quality and project structure.
I wish it was as simple as basically creating a virtualenv to house everything, but it isn't. This project will eventually be developed simultaneously by multiple developers with Git as source control, and the default libraries will be modified. I presume this means that the libraries should ideally be tracked by Git in the end. Virtualenv shouldn't help here as far as I know because it's not portable between systems (or at least that's still being tested).
This project will also be, in the future, deployed to a Centos server.
So the only plan I can think of to successfully pull off this would be to simply bring in all of the libraries (which was done using pip install -t Libraries <ExampleLibrary>) into a single folder, with a __init__.py inside, and use them from other python files as a package within the Pycharm project.
Is this possible / recommended? I tried various methods to reference these libraries, but they all don't work during runtime. Somehow when the files in the library import something else from their own package, an ImportError is raised saying that there's no such module.
Will accept any other suggestions too.
Using Pycharm Community Edition.
EDIT: After having a good night's rest I think the crux of the issue is really just project organization. Before I ported it over to Pycharm the project worked as expected, but this had all of the python files in the root directory, and the libraries in a subfolder of the root, with every project file having the same boilerplate code:
import os, sys
absFilePath = os.path.dirname(os.path.abspath(__file__));
sys.path.insert(1, absFilePath + "/lib")
I was hoping that by using Pycharm to help me flesh out the packages, I could avoid having repeated boilerplate code.
Note: Not full solution.
The addition of the template code below forces the file containing the code to be in the same directory as the libs folder.
For Pycharm, all I had to do was mark the libs folder as a source folder. Even with the addition of the template code to the file, the modified libraries still work as expected.
For the Python Shell, this template code is still needed:
import os
import sys
absFilePath = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(1, absFilePath + "/lib")
I'm working toward adopting Python as part of my team's development tool suite. With the other languages/tools we use, we develop many reusable functions and classes that are specific to the work we do. This standardizes the way we do things and saves a lot of wheel re-inventing.
I can't seem to find any examples of how this is usually handled with Python. Right now I have a development folder on a local drive, with multiple project folders below that, and an additional "common" folder containing packages and modules with re-usable classes and functions. These "common" modules are imported by modules within multiple projects.
Development/
Common/
Package_a/
Package_b/
Project1/
Package1_1/
Package1_2/
Project2/
Package2_1/
Package2_2/
In trying to learn how to distribute a Python application, it seems that there is an assumption that all referenced packages are below the top-level project folder, not collateral to it. The thought also occurred to me that perhaps the correct approach is to develop common/framework modules in a separate project, and once tested, deploy those to each developer's environment by installing to the site-packages folder. However, that also raises questions re distribution.
Can anyone shed light on this, or point me to a resource that discusses this issue?
If you have common code that you want to share across multiple projects, it may be worth thinking about storing this code in a physically separate project, which is then imported as a dependency into your other projects. This is easily achieved if you host your common code project in github or bitbucket, where you can use pip to install it in any other project. This approach not only helps you to easily share common code across multiple projects, but it also helps protect you from inadvertently creating bad dependencies (i.e. those directed from your common code to your non common code).
The link below provides a good introduction to using pip and virtualenv to manage dependencies, definitely worth a read if you and your team are fairly new to working with python as this is a very common toolchain used for just this kind of problem:
http://dabapps.com/blog/introduction-to-pip-and-virtualenv-python/
And the link below shows you how to pull in dependencies from github using pip:
How to use Python Pip install software, to pull packages from Github?
The must-read-first on this kind of stuff is here:
What is the best project structure for a Python application?
in case you haven't seen it (and follow the link in the second answer).
The key is that each major package be importable as if "." was the top level directory, which means that it will also work correctly when installed in a site-packages. What this implies is that major packages should all be flat within the top directory, as in:
myproject-0.1/
myproject/
framework/
packageA/
sub_package_in_A/
module.py
packageB/
...
Then both you (within your other packages) and your users can import as:
import myproject
import packageA.sub_package_in_A.module
etc
Which means you should think hard about #MattAnderson's comment, but if you want it to appear as a separately-distributable package, it needs to be in the top directory.
Note this doesn't stop you (or your users) from doing an:
import packageA.sub_package_in_A as sub_package_in_A
but it does stop you from allowing:
import sub_package_in_A
directly.
...it seems that there is an assumption that all referenced packages
are below the top-level project folder, not collateral to it.
That's mainly because the current working directory is the first entry in sys.path by default, which makes it very convenient to import modules and packages below that directory.
If you remove it, you can't even import stuff from the current working directory...
$ touch foo.py
$ python
>>> import sys
>>> del sys.path[0]
>>> import foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named foo
The thought also occurred to me that perhaps the correct approach is
to develop common/framework modules in a separate project, and once
tested, deploy those to each developer's environment by installing to
the site-packages folder.
It's not really a major issue for development. If you're using version control, and all developers check out the source tree in the same structure, you can easily employ relative path hacks to ensure the code works correctly without having to mess around with environment variables or symbolic links.
However, that also raises questions re distribution.
This is where things can get a bit more complicated, but only if you're planning to release libraries independently of the projects which use them, and/or having multiple project installers share the same libraries. It that's the case, take a look at distutils.
If not, you can simply employ the same relative path hacks used in development to ensure you project works "out of the box".
I think that this is the best reference for creating a distributable python package:
link removed as it leads to a hacked site.
also, don't feel that you need to nest everything under a single directory. You can do things like
platform/
core/
coremodule
api/
apimodule
and then do things like from platform.core import coremodule, etc.
Is there a standard file in python which lists all the modules comprising the project, and other metadata?
Is this simply the 'package'? Or, do different IDEs use their own specific files?
There really isn't a single file in any package that consistently lists every module the entire package imports. Some people make entries to the __init__.py and some don't. Usually most python supported IDE's will make available to you whatever is on your pythonpath. Eclipse pydev, for instance, will add the specific project to the pythonpath of that project space.
If your project is on the pythonpath, then it should resolve.
Application builders like py2app/py2exe will scan the entire project and create an import graph to discover every module needed for that project
There's no real equivalent in Python by itself. Python packages are the way to encapsulate a set of modules and include metadata, but it isn't exactly equivalent to the notion of a "project."
Otherwise, are some projects which use project files in order to give you some of the features which IDEs provide. In particular, you should check out the rope library and the PyCharm IDE for some systems which implement a project file.