In order to develop complex gem5 python configuration scripts with more convenient IDE the gem5 lib has to be added
to the project. However, for those who are not experts in python and Pycharm (Such as myself)
there are some difficulties.
For those who are unfamiliar with gem5, this is
a short explanation:
gem5 is an open source simulator that inspects hardware architecture.
It can be downloaded from github: link to github.
The installation process is described at the following link.
<gem5_installation_dir>/gem5/configs/learning_gem5/part1
Inside the above path, there is a basic python script file: simple.py
This file contains some imports.
Editing this file with Pycharm requires some dependencies that are located in the gem5 installation directory.
The questions:
1) How to add those dependencies of gem5 into Pycharm?
2) How to config the Pycharm with gem5 execution command?
Any help would be appreciated.
A few pointers from what I see at gem5 d9cb548d83fa81858599807f54b52e5be35a6b03 (May 2020) under gem5/configs/learning_gem5/part1/two_level.py:
from common: common is at configs/common which gets found because of the above m5.util.addToPath('../../') call, so add configs/ to the PYTHONPATH as shown at: PyCharm and PYTHONPATH
import m5 comes from src/python/m5 so add src/python to the PYTHONPATH as above
from caches import * comes from the sibling learning_gem5/part1/caches.py, so likely this will be found automatically by PyCharm. Otherwise add that directory to the PYTHONPATH.
from m5.objects import *: this is likely the one you are really interested in as it contains all the interesting objects, but unfortunately PyCharm simply cannot handle it I believe since the SimObjects are added dynamically to that namespace at startup in a very convoluted way via PyBind11 native modules + code generation.
A description of how this works in more detail can be found here, but basically every SimObject class goes through some heavy code autogeneration to make this work, say e.g. src/cpu/simple/AtomicSimpleCPU.py due to SimObject('AtomicSimpleCPU.py') in src/cpu/simple/SConscript.
As of 2017, PyCharm said they did not have plans for a proper native C/C++ extension setup: https://intellij-support.jetbrains.com/hc/en-us/community/posts/206018984-Developing-Python-extension-in-C-using-PyCharm
With that said, I've found that it is not worth to use an IDE for the Python part of gem5. This is because the Python tends to be very simple to understand with IPDB (or impossible to setup an IDE for), and if you just grep stuff you tend to quickly guess what is going on. For C++ though I do recommend setting up Eclipse: How to setup Eclipse IDE for gem5 development?
Related: Add custom modules to PyCharm Linter
Related
I would like to use PyCharm Community Edition 2020.1.2 x64 on Windows 10 as IDE for writing QGIS plugins. I need to base on old version - QGIS 2.18, so I need python 2.7, qt4, and also core modules from QGIS 2.18. I downloaded QGIS 2.18 from official site. It provides a ready to use python environment as .bat script, that sets all needed python paths. So I used path D:\Program Files\QGIS 2.18\bin\python-qgis-ltr.bat to set main python interpreter for my project.
After that I was able to run python console in PyCharm, import all QGIS modules, and any classes from QGIS standard collection like QgsFeature, QgsVectorLayer etc. Everything was fine untill I tried to import this classes from PyCharm editor in .py file as from qgis.core import QgsFeature. In editor PyCharm sees the package qgis.core and I'm able to import it, but PyCharm's code completion inside the module sees only classes that are saved in D:\Program Files\QGIS 2.18\apps\qgis-ltr\python\qgis\core\__init__.py e.g. QgsEditError class, but all standard QGIS class underlines on red.
Python console ran on exactly the same interpreter see the same sources as are visible in "External Libraries" tab in PyCharm, so command:
qgis.core.__path__ returns ['D:\\Program Files\\QGIS 2.18\\apps\\qgis-ltr\\python\\qgis\\core'],
qgis.core.__file__ returns 'D:\\Program Files\\QGIS 2.18\\apps\\qgis-ltr\\python\\qgis\\core\\__init__.pyc,
dir(qgis.core) returns full list of QGIS standard core classes.
If I run the .py script, in which PyCharm's code validator is showing incorrect import object by red underline, the script is executed without any error and I can easly print the imported QgsFeature object. Why my PyCharm's editor see different content of qgis.core module then console ran also from IDE on exactly the same interpreter?
Allow me to qualify myself. I lost interest for adventures and trail blazing in computing some time ago. I won some big battles and lost some too. And I unnecessary wasted a lot of time that could have been better spent. Nowadays, I just wait a reasonable period of time before upgrading anything, and wait a spell before downloading the newest anything, and pretty much avoid trying to do things outside a conventional path.
So your post got me curious and I looked around some. I found very little information about PyCharm being used building QGIS plugins. That pretty much tells me all I want to know: that it's not real common for PyCharm to be used in building QGIS plugins.
But there was quite abit of information about building QGIS plugins with python. There's even a QGIS Tutorial & Tips on the subject. QGIS suggests using the Qt Creator and further goes on to say that QGIS itself is written using the QT framework. "For plugin development, we use an application called Qt Creator."
If you overcome your current predicament I'd like to hear how you accomplished it.
For what it's worth, here's the link to the QGIS Tutorials & Tips.
https://www.qgistutorials.com/en/docs/building_a_python_plugin.html
Finally, the problem solved itself. It was something wrong with building a binary skeletons of C++ libraries, what PyCharm does every time the application is launched. My PyCharm didn't want to do it and I found many similar problems with other libraries (e.g. Qt) that people were struggling with. After two days my PyCharm just fixed itself and created all needed binary skeletons for my QGIS libraries.
I am using pythonnet ( https://github.com/pythonnet/pythonnet ) and have referenced a .Net library within the same solution space but it does not seem to pull in any information to use for Intellisense. In fact, VS gives an error:
"Unable to resolve "ReferenceLibarary". Intellisense may be missing for this module.
So how do I add intellisense to my "module"?
Even though there is an error, the script can still be run (in or out of VS). I have thorough XML already set up as there is also a sandcastle project for a chm output. Can the sandcastle project output the appropriate content that is missing?
Peter Stevens answer seems fine to me. His solution uses IronPython to generate stubs. These stubs are used in turn by the CPython auto-completion mechanism, in for example, Visual Studio Code.
Let's say you have a c# dll located in c:\dev\HTL\bin\release\HTL.dll you want to access from Python and need auto-completion to help code. With IronPython installed and the ironpython-stubs module downloaded, you can do the following:
ipy -m ironstubs make HTL --path="c:\\dev\\HTL\\bin\\release\\HTL.dll" -output stubs.min
Then, as in Peter's response above, in VS Code use File | Preferences | Settings to add the full stubs.min path to "python.autoComplete.extraPaths". Having imported the target dll into your *.py file in the VS Code editor you should now be able to dot to auto-completion for the classes contained in that dll.
Use the ironpython-stubs module.
Make sure your dll is in sys.path, then run from ironpython
ipy -m ironstubs make --output stubs.min
then set "python.autoComplete.extraPaths" to \ironpython-stubs\release\stubs.min
Works like a charm
I write Python programs with eclipse IDE in Linux Backtrack5.
Scapy,Python 2.6 are available in Backtrack.Using auto-completion feature I've seen in all modules,But when i run the program, I see the following error:
from scapy.contrib.ospf import OSPF_Hdr
ImportError : No module named contrib.ospf
I've just started programming with Python, and many of the sources that I've encountered this problem.
The problem for me is very interesting, when coding everything is OK, but it is not running!
In addition to resolving this problem, I want to know why this problem occurs??And in general how to fix?
Maybe my question is a duplicate, but a complete solution and we need it, in which case it does not duplicate any other person who asks the question.
Your search path is wrong.
2 common causes are:
Pythonversions in development and execution are different. Are you sure you use python2.6 both times? Because not all packages are available for all versions of python.
-the search path in your IDE is different from the search path in your shell. I don't have experience with eclipse for python, but it happened to me with other IDEs.
So in bash you can write:
export PYTHONPATH="$PYTHONPATH:/where/module/lives/"
to include the path.
How to include this in your bashprofile:
http://blog.streamitive.com/2011/12/01/export-pythonpathpythonpath/
You can include the path via python itself:
http://www.johnny-lin.com/cdat_tips/tips_pylang/path.html
When to include via bash and when to include via python itself:
include via python for controlled programms, when you know 100% your module will always be in the same location.
Else, you should set the path explicitly. Maybe in a startup shell script, in the crontab command, or wherever you are calling your python program.
The PYTHONPATH the IDE is using isn't the same as the PYTHONPATH you are running the program with
I have had trouble setting up the pycharm ide on my macosx10.7 with python3..
I have scoured every resource available and tried hundreds of approaches, at this point I must accept my incompetence and seek help via this channel.
In my research, I notice a lack of ground-up explanations on python integration into macosx and how to configure pycharm to import modules, run code within the editor, etc. If i ever solve this I will make a very detailed tutorial.
I have imported python3 successfully, it looks like it is linked appropriately from /sys/lib/frameworks to /usr/lib ...etc -- version control is working just fine.
I think my issue is either in setting environmental variables (tried the program to fix this and tried macports) and in the script needed to execute. it will catch errors throughout but final product does not run in python and returns printout of :
/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2 /Users/anon/Desktop/pythonpractice/Py_Ex/classes.py
Process finished with exit code 0
i really need to get this configuration sound for my python programming class. please help (I've been through every line of pycharm website) .. preferably is there a way to map it via terminal? thanks for anyone who took the time to read this.
Summary of the discussion above:
Python 3.2.2 installation was broken on this Mac, installing ActiveState Python 3.2.2 from scratch and configuring it in PyCharm has fixed the problem.
Python path to be used in PyCharm settings: /Library/Frameworks/Python.framework/Versions/3.2/bin/python3
Incompatible third-party plug-ins may break PyCharm, uninstall/disable them in Preferences | Plugins.
Ensure the latest PyCharm version is installed.
User's code depends on the graphics.py module which was not in the project or in the PYTHONPATH. Putting it into the project has solved the problem.
Most likely the wrong Run/Debug configuration was used in PyCharm, the easiest way to run or debug such scripts is by using the editor context menu Run and Debug actions. PyCharm creates the configuration automatically and debugging works fine as shown of the screenshot:
If one wants to configure and debug it, he can use the code.zip file to get started.
Sorry for the comments mess above, but it was not possible to move it into chat as user had only 1 reputation point, hence not able to use the chat feature of StackOverflow.
I am experimenting with the idea of writing Processing apps with Jython on MacOSX Snow Leopard using Eclipse/PyDev. Simple sketches are working pretty well, but when I attempt to use the OpenGL APIs the app crashes and throws this error:
Exception in thread "Animation Thread" java.lang.UnsatisfiedLinkError: no jogl in java.library.path
Note that this is a runtime error, not a build error. It looks like the JVM can't find the jogl libs, though I have added them to the External Libraries in the project settings. In fact I've added all of the following:
core.jar
jogl.jar
gluegen-rt.jar
opengl.jar
libgluegen-rt.jnilib
libjogl_awt.jnilib
libjogl_cg.jnilib
libjogl.jnilib
Inspecting the Python path reveals that all of these files are in fact on the Python path but when I look at the Java path using:
System.getProperty("java.library.path") » » I see only this » »
/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
Clearly something is wrong with the linking of these libs. What could it be?
UPDATE:
I found the issue to be that the paths to the native libraries for the .jar files were not being added to the Java path. PyDev does not appear to accomodate setting the path to the native library of a linked .jar file like the standard Eclipse Java project settings do. So as a fix I was able to add the directory of the native libs throug a VM argument in the Run Configurations.
UPDATE 2:
I've written an article on the env configuration: http://bit.ly/yHjIw9
I replied also with a tweet, but I believe the answer to be setting the LD_LIBRARY_PATH environment variable to point to the folder containing your native libs. (edit: sorry on OSX this is DYLD_LIBRARY_PATH)