We Keep Coding

Python Environment Setup seems complicated and unsolvable

So, I've been a dev for about 3 years, primarily working in TypeScript and NodeJS. I'm trying to broaden my skillset by learning python (and eventually expanding my learning of Computer Vision, Machine learning, etc.) but I feel incredibly frustrated by trying to get Python to work consistently on my machine. Surely I'm doing something wrong, but I just can't really understand what it is.
I've run into these problems mostly when using ML packages (TensorFlow, Whisper, OpenCV (although I was eventually able to resolve this), so I don't know if it's related to M1 support of one of the common dependencies, etc.
My current understanding of python is that:
M1 support of python is version dependent at best.
venv is the only environment manager I should need to use
I should use pyenv to install python versions so as to not conflict with os-installed python (macos dependencies)
I'll use the latest project I'm working on as an example.
My Machine and Environment
Mac M1 Pro, MacOS Monterey 12.6
pyenv 2.3.9
Python 3.7.13
Fish Shell version 3.5.1
My general workflow to get a project started:
Create virtual environment using venv
python3 -m venv <some_environment_name>
Open created directory using VSCode, and activate the virtual environment
Here is where I encounter my first issue, which seems to be persistent.
➤ source /Users/paal/src/whisper_transcription/bin/activate.fish
functions: Function '_old_fish_prompt' does not exist
~/src/whisper_transcription/bin/activate.fish (line 18):
functions -c _old_fish_prompt fish_prompt
^
in function 'deactivate' with arguments 'nondestructive'
called on line 30 of file ~/src/whisper_transcription/bin/activate.fish
from sourcing file ~/src/whisper_transcription/bin/activate.fish
(Type 'help functions' for related documentation)
functions: Function 'fish_prompt' does not exist
~/src/whisper_transcription/bin/activate.fish (line 47):
functions -c fish_prompt _old_fish_prompt
^
from sourcing file ~/src/whisper_transcription/bin/activate.fish
(Type 'help functions' for related documentation)
fish: Unknown command: _old_fish_prompt
~/src/whisper_transcription/bin/activate.fish (line 71):
_old_fish_prompt
^
in function 'fish_prompt'
in command substitution
(whisper_transcription)
So, what I do to resolve this is add the following if statement to the fish.config file.
if type -q $program
_old_fish_prompt
end
Looking at GitHub issues, this seems to be a persistent issue for fish shell and this seems to at least temporarily resolve it.
Or, I just switch to ZSH.
OK, so with that resolved I move on. The environment is activated, I'm using ZSH now, and I can successfully run a python script that prints "hello world" to the console.
Then comes the nightmare of installing any packages. It seems like any project I start has some weird edge case of compatibility issues. Between M1 processors, Python Versions, builds not working correctly, etc.
for example..
import whisper
... # rest of file
This with any other code or even by itself throws the following error:
Traceback (most recent call last):
File "main.py", line 1, in <module>
import whisper
File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/__init__.py", line 12, in <module>
from .decoding import DecodingOptions, DecodingResult, decode, detect_language
File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/decoding.py", line 514
if prefix := self.options.prefix:
^
SyntaxError: invalid syntax
This appears to be some problem with the Python Version. From what I understand, the := operator isn't valid syntax until Python 3.8. However, dependencies of whisper (PyTorch) only seems to be supported up to version 3.7.9
So, you can see, it seems like I just end up in these bizarre circular problems where some dependency of some package I want to use isn't supported by either the platform or the current python version, and they seem basically unsolvable (at least with my current knowledge of python)
Why is this seemingly so complicated? I'm clearly doing something wrong here, and obviously I'm out of my comfort and knowledge zone, but these issues feel very daunting and opaque, and difficult to actually troubleshoot this in any consistent or clear way.
Is there a resource that makes this stuff more clear? Is Python development on M1 chips just this broken at the moment? How can I get past these seemingly basic issues to start actually learning?
I'm not necessarily looking for a solution to this specific problem here, but if theres general advice about environment management and how to make things work somewhat reliably, I'm fine troubleshooting. I just feel like every time I start trying to learn, I end up in these rabbit holes that take hours and hours to fix and sometimes don't even really resolve things.

Unable to import modules in pycharm in spite being installed in the used python interpreter

Even after installing dash - getting module import error.
Pycharm does not throw any compilation errors on the line as well.
How could this be resolved?

I tried everything as suggested in comments and stuff known to me but could not resolve this - have done this a thousand times earlier but don't know what went wrong this time.
The only solution the=at worked was to create a new project, copy files over manually and then create a fresh interpreter and VE.

AttributeError: dlsym(RTLD_DEFAULT, run): symbol not found - meaning

I am attempting to use the PyMultinest package. The full error text I am encountering is AttributeError:dlsym(RTLD_DEFAULT, run): symbol not found in the __getitem__() function in ctypes/__init__.py. I'll include more text and code below, but I am mostly trying to understand what this error is telling me - my Google Fu is apparently lacking, and the StackExchange questions I have seen relating to this error seem to be hyper-focused on solving a specific instance of this error. So - What is this error trying to tell me is wrong?
More context. I attempt to execute the PyMultinest (PMN) package as directed in the PMN documentation. PMN is, effectively, a Python wrapper for a C program. Running PMN requires a fair bit of setup code (several ancillary functions need to be defined, as well as a host of variables), which I'm not including here by default because it's ... a lot, but I can if needed. The PMN execution line I use is
pmn.run(Loglike, Prior, ndims, n_live_points=1000, n_params=n_params, outputfiles_basename='./'+ProjectName+'/temp_', resume=False, verbose=True)
This returns the error traceback
File "[redacted]", line 139, in <module>
pmn.run(Loglike, Prior, ndims, n_live_points=1000, n_params=n_params, outputfiles_basename='./'+ProjectName+'/temp_', resume=False, verbose=True)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymultinest/run.py", line 254, in run
lib.run(*args_converted)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
func = self.__getitem__(name)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py", line 386, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, run): symbol not found
If it helps, I have determined that the name variable being passeed through self.__getitem__(), and thus into self._FuncPtr(), is run. Although, that might be obvious looking at the AttributeError message.
I am running Python 3.8 (as shown above) on a MacOS machine. Last summer, I was able to execute PMN on this machine using extremely similar code to that which I am using now. I am currently trying to optimize my code from last summer, which is why I'm surprised it isn't simply "working".
So far my attempts at fixing this have been mostly centered on reinstalling PMN. I have done a clean install (pip uninstall/pip install) of the PMN package, as well as following the PMN documentation to rebuild the C portion of the package. I have included the source directories of the C software in my Path variables - or at least, I tried to, I am assuming I was successful, but I'm not very familiar with Macs.
Ultimately, I just wish I understood what Python was telling me with this error better. It would help me direct my own attempts at solving the issue. I suspect it is saying "We don't know where to find this 'run' command you're asking for," in which case I need to figure out why my Path variable changes aren't working. Am I on the right path?

Some hints how to resolve this are given in these issues:
https://github.com/JohannesBuchner/PyMultiNest/issues/160
https://github.com/JohannesBuchner/PyMultiNest/issues/163
It is important to cleanly recompile (empty build folder) when the environment changes, so that cmake recognises changed libraries and compilers.

Check if you are running everything with python3 and not python (works as python 2.x is you have both version 2 and 3 installed)
Check is all the PyMultiNest python files are rewritten in python 3 style, e.g. $ print "something" becomes $ print("something")
My supervisor was accidentally running on python 2.7 and got the same error, so your error might as well be due to mismatch of python version usages.

Run command flutter pub cache clean and flutter clean and after that run flutter pub get now relaunch your emulator it worked for me.

Invalid Salt on Local versus Heroku (Encoding UTF-8 problems)

The Issue:
It appears there is some sort of encoding/decoding issue going on with flask-bcrypt for my Mac. I'd like to know if there is an easy solution to fix this so I can run my local with a similar setup to my Heroku version.
Comparison:
Local
If I use .decode("utf-8") with generate_password_hash it returns AttributeError: 'str' object has no attribute 'decode'. Without decode, it saves my password to the database properly and I'm able to login with no issues.
When I test bcrypt.generate_password_hash [without decode] in terminal on local it
returns like '$2b$12$yTBsCi.....`
Heroku
If I use .decode("utf-8") with generate_password_hash it saves to the database properly and I"m able to login. If I don't use decode("utf-8"), it saves the password incorrectly (something like: \x243261243132245a4e64696e4d7062327......) and when I try to login it returns ValueError: Invalid salt.
When I test bcrypt.generate_password_hash [without decode] on heroku bash it returns
b'$2b$12$amVQ6f2.d......'
Current Setup:
Both are running Python 3.6.4. Both Heroku and my local version have identical requirements installed. bcrypt==3.1.4 Flask-bcrypt==0.7.1 py-bcrypt==0.4
Documentation:
https://flask-bcrypt.readthedocs.io/en/latest/
In Python 3, you need to use decode(‘utf-8’) on generate_password_hash(), like below:
pw_hash = bcrypt.generate_password_hash(‘hunter2’).decode(‘utf-8’)
Final Thoughts:
Any solutions to get my Mac encoding/decoding correctly? Perhaps because MacOS relies on Python 2.7 it's messing it up? But this shouldn't be the case within my virtualenv.....
I've run echo $LANG and looked at the psql server_encoding on local/heroku and they are both identical for utf8.

So! This is a mind boggling problem for a lot of people. Now that I have found a solution, I'm back to post the results.
Solution Part A: [The most important piece is in Solution Part B, this is a minor fix]. The first thing I realized was that I was using double quotes around my utf-8. I then rechecked the documentation and realized it was a single quote .decode('utf-8'). As soon as I did this, it seemed to fix the problem (at least temporarily). Why that fixes it, I'm uncertain, but that's something to try and a small detail to overlook. After a few days of more pushes, the bug popped back up.
Solution Part B: This is the much larger problem that I learned. After hours of searching I stumbled on a google search that was something along the lines of "Flask-bcrypt py-bcrypt conflict". There were some ambiguous stack overflow questions and github issues that lead me to realize there was likely something conflicting between these packages. So naturally I removed them an reinstalled py-bcrypt, flask-bcrypt, and bcrypt. Same problem. Then I did it again in a different order....new problem....about ffi being missing. So then I uninstalled all of them and ONLY installed flask-bcrypt. Well wouldn't you know...that fixed the problem for Heroku and my local. It turns out that when you install flask-bcrypt it auto installs bcrypt. I left py-bcrypt uninstalled entirely. It seems bcrypt and py-crypt really don't like each other.
Hope that fixes it for you!
(note: I do have this feeling that bcrypt is still accessing py-bcrypt in some way, maybe through headers or the python package itself. Regardless, removing it from my virtual environment did the trick).
(alternative solution: I heard if these packages just really don't play nice, you can use generate_password_hash from something like werkzeug or try installing the cryptography package. I didn't want to go that route though, but worth a shot).

Using msi.py to install Python3.4

So I downloaded source for Python3.4 and used VisualStudio 2010 to build a Python executable for my Windows7 machine. Ultimately, I want to use this to embed Python support in my application.
The first time I tried to execute my app, PyInitialise ends up aborting, and presents the error message:
"Py_Initialize: unable to load the file system codec"
So now I'm thinking, D'oh, you never installed your Python build after building it. After much googling, I find myself running msi.py. When I do, I see:
File "msi.py", line 934
raise ValueError, "Could not find "+srcdir+"/../"+pat
^
SyntaxError: invalid syntax
This looks to me like I'm trying to run pre-Python3.X code. But I'm so sure this MSI tool came with my Python3.4 source distribution. That said, I've been staring at this for so long things are starting to blend to a blur.
Any clues about where to look or what to try next?
Should I be worrying about MSI? Is there some other answer to my codec problem?