We Keep Coding

How to add a folder to Python path?

Basically, I can only reference my other files as modules when they are in a very specific location:
C:\Users\Dave\Desktop\Programming\Python.
If I want to create a new folder for a large project with multiple modules, say
C:\Users\Dave\Desktop\Programming\Python\Project1,
I can no longer import any modules and keep getting a ModuleNotFoundError. I've looked into it and it seems I need to add that folder to the Python Path somehow, but I couldn't find any answers on how to do it. My computer runs on Windows 10 if that matters.

I think the immediate solution to your problem/the answer to your question would be to use sys.path.append() at the top of your script.
import sys
sys.path.append("<ABSOLUTE/PATH/TO/YOUR/CUSTOM/MODULES/FOLDER>")
import custom_module
This isn't an ideal solution for any kind of prod use, however. Based on what you wrote in your question, this may not be what you're looking for. More info might help to craft a more stable solution:
Are you using virtual environments when running python? Do you just run python script.py or is there a specific version of python you're using that's installed elsewhere than the common C:\Program Files\Python?
When you say that you work on a project with multiple modules, does that mean there are custom modules that you/someone wrote or is it just that that project uses non-standard library modules, i.e. you had to pip install them?
Can we get an example of the code you're running and the folder structure of your project/where the modules are that you need?

Writing gem5 configuration scripts with Pycharm

In order to develop complex gem5 python configuration scripts with more convenient IDE the gem5 lib has to be added
to the project. However, for those who are not experts in python and Pycharm (Such as myself)
there are some difficulties.
For those who are unfamiliar with gem5, this is
a short explanation:
gem5 is an open source simulator that inspects hardware architecture.
It can be downloaded from github: link to github.
The installation process is described at the following link.
<gem5_installation_dir>/gem5/configs/learning_gem5/part1
Inside the above path, there is a basic python script file: simple.py
This file contains some imports.
Editing this file with Pycharm requires some dependencies that are located in the gem5 installation directory.
The questions:
1) How to add those dependencies of gem5 into Pycharm?
2) How to config the Pycharm with gem5 execution command?
Any help would be appreciated.

A few pointers from what I see at gem5 d9cb548d83fa81858599807f54b52e5be35a6b03 (May 2020) under gem5/configs/learning_gem5/part1/two_level.py:
from common: common is at configs/common which gets found because of the above m5.util.addToPath('../../') call, so add configs/ to the PYTHONPATH as shown at: PyCharm and PYTHONPATH
import m5 comes from src/python/m5 so add src/python to the PYTHONPATH as above
from caches import * comes from the sibling learning_gem5/part1/caches.py, so likely this will be found automatically by PyCharm. Otherwise add that directory to the PYTHONPATH.
from m5.objects import *: this is likely the one you are really interested in as it contains all the interesting objects, but unfortunately PyCharm simply cannot handle it I believe since the SimObjects are added dynamically to that namespace at startup in a very convoluted way via PyBind11 native modules + code generation.
A description of how this works in more detail can be found here, but basically every SimObject class goes through some heavy code autogeneration to make this work, say e.g. src/cpu/simple/AtomicSimpleCPU.py due to SimObject('AtomicSimpleCPU.py') in src/cpu/simple/SConscript.
As of 2017, PyCharm said they did not have plans for a proper native C/C++ extension setup: https://intellij-support.jetbrains.com/hc/en-us/community/posts/206018984-Developing-Python-extension-in-C-using-PyCharm
With that said, I've found that it is not worth to use an IDE for the Python part of gem5. This is because the Python tends to be very simple to understand with IPDB (or impossible to setup an IDE for), and if you just grep stuff you tend to quickly guess what is going on. For C++ though I do recommend setting up Eclipse: How to setup Eclipse IDE for gem5 development?
Related: Add custom modules to PyCharm Linter

python power shell issues

Edited to include a specific example
I am learning to use python and work in a windows 10 environment (although on 3 separate computers). I have gotten past the "add python to %PATH% issue" but continue to see odd outcomes when I call simple scripts from powershell compared to running them from the python terminal. For example, I often see that lines of code are duplicated, even simple print statements. A second repeating issue is that some (but not all) modules work fine in the python terminal but can't be found when running a script from powershell, despite using pip install within powershell to install it in the first place.
Rather then seeking help on a specific issue, I'm hoping for some guidance into how powershell and python interact that might help me understand or identify some commonalities in these issues I keep experiencing that are common from within powershell. If thats as simple as redirecting me towards another source even that'd be apprecaited. But my search results always turn up the common issue of powershell not knowing what python is, and needing to add python to the PATH. But as indicated, I have already gotten past this and can get %50 of my python to work from powershell. There must be something else perhaps obvious to others that I'm missing that can help me understand why some things aren't working while others are.
Thanks for taking the time to consider my problem and any advice is greatly apprecaited.
Here is an example from some code I am trying to get working, but in troubleshooting I have taken out a lot of code and am now only running what is shown that defines a dataframe and a print statement. Everything works line by line in python, but when I call the script from powershell, the print statement executes twice.
#!/usr/bin/python3
import pandas as pd
joedata = {'fpr': [0.2,0.4,0.8], 'tpr':[0.9,0.5,0.1]}
joeframe = pd.DataFrame(data=joedata)
print(joeframe)

Concerning modules, you probably have multiple versions of Python installed on your system. Calling pip in the Powershell doesn't mean that it will install for the Python installation which is called by default when you execute Python in the Powershell. So you should try to figure out on which version pip is installing packages and which version is actually used when executing a script.
EDIT: I tested the example that you gave in Powershell and I don't have the problem, it's working fine.
Moreover, for me, without doing any special configuration, in Powershell the commands pip and python refer to the same version of Python. If by default everything is ok, you should consider uninstalling Python and just running the installer again and let it manage the PATH, etc.

change the priority of pythonpath

When I was loading certain modules [namely pygments.lexers Bash Lexer and pygments.formatters LatexFormatter] I was was getting an error that python couldn't find the modules. I then realised that this problem was being caused by my PYTHONPATH, which is set up for using paraview with python. It brings its own version of pygments, which doesn't work with nbconvert from the jupyter notebook for some reason [Note it is not totally disfunctional, as PythonLexer, and a few others were called without a problem, it was only the ones that I've mentioned above that couldn't be found].
I have a similar problem with mayavi, which wouldn't work with paraview's version of vtk.
Both of these problems can be resolved simply enough by commenting out the python path in the bashrc, but obviously then paraview won't work.
Is there any way to, for example, reduce the priority of the PYTHONPATH so that the system codes in /etc... are called preferentially, but paraview can still find the ones that it needs in the PYTHONPATH?
I am using python 2.76 on linux mint 17.3, paraview is version 4.4.0, installed from source code as per here

Ordering the entries in PYTHONPATH is partly right, but the system paths don't seem to get included until you run python, and then they get put at the end. So to put a system path in front, explicitly add it:
export PYTHONPATH="[path/to/system/files]:$PYTHONPATH"
It is kind of a hack, because the system path you add will be duplicated in sys.path. But it works.

Yes, you can import sys and manipulate sys.path as an ordinary list at runtime. You can rearrange what's there, or just insert at the beginning (sys.path.insert(0, 'path')). Do this before your import statement. If this will cause problems elsewhere, put it back after your import statement.
Note, this is fairly hacky. But it sounds like you might have a case for it, although I have not looked at these specific tools together.
Edit: this is more relevant if you want to control the python path at the level of individual imports within the course of one execution of python. If you want to control the path at the level of one full execution of Python, you can also set the python path on the command line for just that execution like this:
PYTHONPATH=/replacement/path/here python your_script.py
This is more verbose (unless you wrap it in a shell script or alias) than just calling python, but it lets you control the path one script at a time, wheres putting it into .bashrc/.bash_profile or similar changes it for your whole shell session.

export PYTHONPATH=$PYTHONPATH:<your_path> will give priority to system paths and only if something is not there will look on your path.
export PYTHONPATH=<your_path>:$PYTHONPATH will search <your_path> firstly and then $PYTHONPATH for what it does not find.
If something exists in both but you want to use one version of it for one program and the other for another then you might want to look on different bashrc profiles.

Python3/MacOSX integration into pycharm

I have had trouble setting up the pycharm ide on my macosx10.7 with python3..
I have scoured every resource available and tried hundreds of approaches, at this point I must accept my incompetence and seek help via this channel.
In my research, I notice a lack of ground-up explanations on python integration into macosx and how to configure pycharm to import modules, run code within the editor, etc. If i ever solve this I will make a very detailed tutorial.
I have imported python3 successfully, it looks like it is linked appropriately from /sys/lib/frameworks to /usr/lib ...etc -- version control is working just fine.
I think my issue is either in setting environmental variables (tried the program to fix this and tried macports) and in the script needed to execute. it will catch errors throughout but final product does not run in python and returns printout of :
/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2 /Users/anon/Desktop/pythonpractice/Py_Ex/classes.py
Process finished with exit code 0
i really need to get this configuration sound for my python programming class. please help (I've been through every line of pycharm website) .. preferably is there a way to map it via terminal? thanks for anyone who took the time to read this.

Summary of the discussion above:
Python 3.2.2 installation was broken on this Mac, installing ActiveState Python 3.2.2 from scratch and configuring it in PyCharm has fixed the problem.
Python path to be used in PyCharm settings: /Library/Frameworks/Python.framework/Versions/3.2/bin/python3
Incompatible third-party plug-ins may break PyCharm, uninstall/disable them in Preferences | Plugins.
Ensure the latest PyCharm version is installed.
User's code depends on the graphics.py module which was not in the project or in the PYTHONPATH. Putting it into the project has solved the problem.
Most likely the wrong Run/Debug configuration was used in PyCharm, the easiest way to run or debug such scripts is by using the editor context menu Run and Debug actions. PyCharm creates the configuration automatically and debugging works fine as shown of the screenshot:
If one wants to configure and debug it, he can use the code.zip file to get started.
Sorry for the comments mess above, but it was not possible to move it into chat as user had only 1 reputation point, hence not able to use the chat feature of StackOverflow.