We Keep Coding

Python Environment Setup seems complicated and unsolvable

So, I've been a dev for about 3 years, primarily working in TypeScript and NodeJS. I'm trying to broaden my skillset by learning python (and eventually expanding my learning of Computer Vision, Machine learning, etc.) but I feel incredibly frustrated by trying to get Python to work consistently on my machine. Surely I'm doing something wrong, but I just can't really understand what it is.
I've run into these problems mostly when using ML packages (TensorFlow, Whisper, OpenCV (although I was eventually able to resolve this), so I don't know if it's related to M1 support of one of the common dependencies, etc.
My current understanding of python is that:
M1 support of python is version dependent at best.
venv is the only environment manager I should need to use
I should use pyenv to install python versions so as to not conflict with os-installed python (macos dependencies)
I'll use the latest project I'm working on as an example.
My Machine and Environment
Mac M1 Pro, MacOS Monterey 12.6
pyenv 2.3.9
Python 3.7.13
Fish Shell version 3.5.1
My general workflow to get a project started:
Create virtual environment using venv
python3 -m venv <some_environment_name>
Open created directory using VSCode, and activate the virtual environment
Here is where I encounter my first issue, which seems to be persistent.
➤ source /Users/paal/src/whisper_transcription/bin/activate.fish
functions: Function '_old_fish_prompt' does not exist
~/src/whisper_transcription/bin/activate.fish (line 18):
functions -c _old_fish_prompt fish_prompt
^
in function 'deactivate' with arguments 'nondestructive'
called on line 30 of file ~/src/whisper_transcription/bin/activate.fish
from sourcing file ~/src/whisper_transcription/bin/activate.fish
(Type 'help functions' for related documentation)
functions: Function 'fish_prompt' does not exist
~/src/whisper_transcription/bin/activate.fish (line 47):
functions -c fish_prompt _old_fish_prompt
^
from sourcing file ~/src/whisper_transcription/bin/activate.fish
(Type 'help functions' for related documentation)
fish: Unknown command: _old_fish_prompt
~/src/whisper_transcription/bin/activate.fish (line 71):
_old_fish_prompt
^
in function 'fish_prompt'
in command substitution
(whisper_transcription)
So, what I do to resolve this is add the following if statement to the fish.config file.
if type -q $program
_old_fish_prompt
end
Looking at GitHub issues, this seems to be a persistent issue for fish shell and this seems to at least temporarily resolve it.
Or, I just switch to ZSH.
OK, so with that resolved I move on. The environment is activated, I'm using ZSH now, and I can successfully run a python script that prints "hello world" to the console.
Then comes the nightmare of installing any packages. It seems like any project I start has some weird edge case of compatibility issues. Between M1 processors, Python Versions, builds not working correctly, etc.
for example..
import whisper
... # rest of file
This with any other code or even by itself throws the following error:
Traceback (most recent call last):
File "main.py", line 1, in <module>
import whisper
File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/__init__.py", line 12, in <module>
from .decoding import DecodingOptions, DecodingResult, decode, detect_language
File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/decoding.py", line 514
if prefix := self.options.prefix:
^
SyntaxError: invalid syntax
This appears to be some problem with the Python Version. From what I understand, the := operator isn't valid syntax until Python 3.8. However, dependencies of whisper (PyTorch) only seems to be supported up to version 3.7.9
So, you can see, it seems like I just end up in these bizarre circular problems where some dependency of some package I want to use isn't supported by either the platform or the current python version, and they seem basically unsolvable (at least with my current knowledge of python)
Why is this seemingly so complicated? I'm clearly doing something wrong here, and obviously I'm out of my comfort and knowledge zone, but these issues feel very daunting and opaque, and difficult to actually troubleshoot this in any consistent or clear way.
Is there a resource that makes this stuff more clear? Is Python development on M1 chips just this broken at the moment? How can I get past these seemingly basic issues to start actually learning?
I'm not necessarily looking for a solution to this specific problem here, but if theres general advice about environment management and how to make things work somewhat reliably, I'm fine troubleshooting. I just feel like every time I start trying to learn, I end up in these rabbit holes that take hours and hours to fix and sometimes don't even really resolve things.

Creating or Using a VENV File/ Portability

Venv File I have a project that utilizes YoloV5, OpenCV, as well as many other python libraries. After finally finishing work on the project, I wanted to share it with my teammates. However, I was unable to get my code to run anywhere other than my machine. There was a Venv file that was generated automatically somewhere along the way, either by my IDE or a jupyter notebook. I tried giving out this file to others, but it is not only massive (~1GB), but it also didn't work. I will consistently get syntax errors that do not exist in my environment. For example, I will get:
"SyntaxError: Non-ASCII character '\xf0' in file detect.py on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
"
If I remove that line, I get:
" File "detect.py", line 120
model.warmup(imgsz=(1 if pt else bs, 3, *imgsz)) # warmup
^
SyntaxError: invalid syntax
"
Mind you, none of these errors exist in my environment.
I have tried looking up various commands to somehow link the venv file to the environment or to the python file. I thought that, perhaps, python wasn't looking for the file, so it wasn't using it. I also tried installing other packages and installing all the dependencies manually. However, none of these things worked. I am a little lost on where to go from here, I have heard of tools like Conda, which apparently might help manage things like this, but I am wondering if anyone could give advice on how to proceed, or maybe if anyone else has personally used YoloV5 and encountered this problem themselves

Error: 'python35_d.dll is missing' while running C++ code with Python in MS Visual Studio 2015

I am trying to embed Python in my C++ code to use WordNet through nltk. I am new to Python and I am using MS Visual Studio 2015 to do the same but I'm facing problems to use Python on MSVS 15.
So far I've,
installed Python 3.5.1,
included this path: ...\Python\Python35-32\include, which let me include the Python.h library,
added this path: ...\Python\Python35-32\libs to access the lib files, and python35.lib as dependency in Linker\Input setting of MSVS15,
then there an error occurred, demanding python35_d.lib, so I compiled Python's source in Debug mode & copied that file to that ...\libs folder and it solved the problem.
This all happened when I was trying to run this only-
Py_Initialize();
PyRun_SimpleString("print (\"Hello World!\\n\")");
Py_Finalize();
Now I'm trying to run a Python function from a .py file. I'm trying to run this from the Python tutorials but I'm getting this error -
While debugging, I have found that when this macro gets called: Py_DECREF(), this error occurs.
Do I only need to copy python35_d.dll from PCBuild\win32 to C:\Windows\[System|System32|SysWOW64]\ to solve the problem as I don't know if it'll have any side-effects?
Although the file is already present in ...\Python\Python35-32\DLLs\.
Is there a proper setup to be done before running Python in C/C++ using MSVS, like this for Unix-like systems? (I mean by adding other dependencies or folders to the settings)
I've been reading Python documentation continuously and even other stackoverflow questions but I'm still struggling a lot (I don't know why) even to do simple things.
Can somebody guide/enlighten me for something I may be missing because I'm continuously getting errors here & there, please?
Thanks.

copy python35.dll into the folder where .sln file is present

cloudera-scm-agent fails when using python 2.7

Not sure how I can get around this...on the cloudera-manager site it says their software requires either pyhton2.6 or python2.7
however when i try to start the cloudera-scm-agent it complains that:
/opt/versioned_apps/cm-4.8.2/lib64/cmf/agent/build/env/bin/python: error while loading shared libraries: libpython2.6.so.1.0: cannot open shared object file: No such file or directory
I'm running centos7 (which is not supported out of the box).
To make matters worst, I don't know anything about python either (sorry)...so if i need to install anything, please provide step-by-step instructions :-)

Correct mt.exe syntax to resolve pyodbc import problems

I'm deploying a Python app on Windows for the first time (Apache & mod_wsgi/Flask/Sqlalchemy).
I'd like to use pyodbc to get Sqlalchemy talking to mssql.
I was able to install a binary build of pyodbc with no problem, and it works when I use it from the command line. However, mod_wsgi is logging import errors when it hits import pyodbc
This thread on the pyodbc issue tracker describes my issue exactly, and seems to implicate Windows manifests. However, I can't figure out how to use mt.exe to resolve it.
In PowerShell:
C:\my\site-packages> C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\mt.exe
-inputresource:c:\Windows\SysWOW64\python27.dll;#2 -manifest
-outputresource:pyodbc.pyd#2'
mt.exe : command line error c10100a9: Some operation on the input manifests must
be specified (even if it is just to pipe the input to the output).
Use the /? option for help on usage and samples.`
In cmd:
C:\my\site-packages>"C:\Program Files\Microsoft SDKs"\Windows\v7.1\Bin\mt.exe
-inputresource:C:\Windows\SysWOW64\python27.dll#2 -outputresource:pyodbc.pyd#2
mt.exe : general error c101008c: Failed to read the manifest from the resource
of file "C:\Windows\SysWOW64\python27.dll#2". The system cannot find the file
specified.
What am I doing wrong?!

I'm the author of the thread on the pyodbc issue tracker.
Unless I'm missing something it looks like the file "C:\Windows\SysWOW64\python27.dll" doesn't exist. Maybe you have a version other than 2.7?
Otherwise, make sure you have installed the official windows packages for python and verify that importing pyodbc succeeds with that version. If it does, you should be able to substitute the path to your local python.dll (e.g. python27.dll) from the python installation root or elsewhere.

Well, this is embarrassing: I appear to be missing some semicolons. I know that I'd run the command properly at least a few times though, so I will list the other things I changed before noticing the mistake in the syntax:
Made triply sure everything was 32-bit
Downloaded Visual Studio C++ 2008 Express (was just running VC++ 2008 redistributable before, and this gave me a different version of mt.exe)
Uninstalled anything with "2010" and "C++" in its name
Ran the command in cmd; for whatever reason, it does not work at all in PowerShell.
Thanks again to Alfred for his epically detailed and helpful post over at the pyodbc mailing list.