We Keep Coding

Using embedded python, loading a *.pyd outside of DLLs directory fails

I have a C++ application (X-Plane) for which there is a plugin which permits the use of python scripts (XPPython3 plugin). The plugin is written in C, using python CAPI, and works great, allowing me to write python scripts which get executed within the C++ application.
On Windows 10, I want to extend my python features by importing imgui. I have a python cython-built pyd file (_imgui.cp39-win_amd64.pyd).
If I place the pyd file in C\Program Files\Python39\DLLs, it works as expected: C++ application calls CAPI to python, which loads script which imports and executes imgui code.
If I place the pyd file anywhere else, embedded python either reports "module not found" -- if the pyd isn't on sys.path(), or if it is on sys.path():
ImportError: DLL load failed while importing _imgui: The parameter is incorrect.'
Changes using: os.add_dll_directory(r'D:\somewhere\else')
Does not effect whether the module is found or not, nor does it change the 'parameter incorrect' error. (see https://bugs.python.org/issue36085 for details on this change. -- my guess is add_dll_directory changes lookup for DLLs, but not for pyd?) sys.path appears to be used for locating pyd.
Yes, the pyd is compiled with python3.9: I've compiled it both with mingw and with visual studio toolchains, in case that might be a difference.
For fun, I moved python-standard _zoneinfo.pyd from Python39\DLLs and it fails in the same way in embedded python: "The parameter is incorrect". So, that would appear to rule out my specific pyd file.
The key question is/are:
Other than placing a pyd file under PythonXX\DLLs, is there a way to load a PYD in an embedded python implementation? (I want to avoid having to tell users to move my pyd file into the Python39\DLLs directory... because they'll forget.)
Note that using IDLE or python.exe, I can load pyds without error -- anywhere on sys.path -- so they don't have to be under Python39\DLLs. It's only when trying to load from embedded python that the "Parameter is incorrect" appears. And of course, this works flawlessly on Mac.
(Bonus question: what parameter? It appears to be python passing through a windows error.)

There seems to be a simple answer, though I suspect it's better characterized as a python bug.
There is nothing magical about Python39\DLLs directory.
The problem is using absolute vs relative paths in sys.path.
Python can find modules using absolute or relative paths. So if zippy.py is in folder foobar,
sys.path.append('foobar')
import zippy
# Success
Python and find, BUT NOT LOAD pyd files using relative paths. For example, move _zoneinfo.pyd from PythonXX\LDDs to foobar
sys.path.append('foobar')
import _zoneinfo
# ImportError: DLL load failed while importing _zoneinfo: The parameter is incorrect.'
Instead, use absolute path, and it will find and load PYD:
sys.path.append(r'c:\MyTest\foobar')
import _zoneinfo
# Success

So, there is actually a way to do this—that is, ship your application with the desired libraries. The solution is to use an embedded distribution and ship this with your application. You can find the correct distribution on the official Python download page corresponding to your desired version (here's the link to the lastest 3.9 release which seems to be what you're using: https://www.python.org/downloads/release/python-392/). Look for the Windows Embeddable Package.
You can then simply drop in your .pyd file alongside the standard library files (note that if your third-party library is dependent on any other libraries, you will have to include them, as well). Shipping your application with an embeddable distribution should not only solve your current issue, but will also mean that your application will work regardless of which version of Python a user has installed (or without having Python installed at all).

Paraview: paraview.simple import, all paths are set

I am trying to run a simple python program, importing the paraview.simple module:
from paraview.simple import *
I get the following error:
Error: Could not import vtkCommonComputationalGeometry
I looked at similar posts on different fora and tried everything that was suggested there, but nothing worked for me. My Python path includes:
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/bin/
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/
My LD_LIBRARY_PATH includes:
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/vtkmodules/
Does anybody know how to fix it?
Update:
I think there is an underline issue regarding the Unicode variant my python interpreter is using. I now get the following error:
Unicode error
ImportError: /home/nick/ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/vtkmodules/vtkCommonCorePython.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
Does anybody know a fix?

You may want to use the pvpython program that is bundled with ParaView. It is basically a python interpreter but already setup with the correct paths.
If you want to use an external interpreter, you have to setup the PYTHONPATH environment variable to ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/ and the LD_LIBRARY_PATH (on linux, PATH on windows) to ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib.
See also the ParaViewTutorial pdf from the download page (https://www.paraview.org/download/), at 3.1 Starting the Python Interpreter

Python 3.3.5 time module missing

When I try to import time I get : No module named time
I have tried other time modules(datetime and timeit) and they work fine. I decided to check my installation and I can't find time.py anywhere. I checked the Lib, Scripts, libs and include folders, but can't find it anywhere.
Anyone know what I can do to fix this? Maybe download the .py and put it in Lib myself?
I am using Python 3.3.5 with PyCharm IDE. Only extra scripts I've installed is EasyInstall and PRAW.

The import does work. When PyCharm said No module named time, I assumed I would get a compiler error and started trying to fix it.
However when I eventually just ran the code it worked fine. I expect PyCharm doesn't detect the time module as it's a dll and not a py as noted by Martijn in the comments. This is on PyCharm Community Edition 4.0.4.

I tried playing with virtualenv and a host of other things, but I eventually went to Preferences -> Build, Execution, Deployment -> Console -> Python Console, and in the "starting script" box, I added two lines:
sys.builtin_module_names.append('sys')
sys.builtin_module_names.append('time')
This got rid of errors I had with both sys and time. Once I did that, I even get autocomplete for both of those modules... weird.

YouCompleteMe/Python can complete for built-in libs, but not site-packages

I just installed ycm, everything looks good, but I found small problem. The problem is as following:
import os # os is built-in library
os. # ycm helps to complete members of the class.
import numpy # numpy is not built-in library, where its location is site-packages.
numpy. # nothing happened. ycm shows 'pattern not found' message.
I think, this would be a simple problem. But I could not find the solution yet. I think, there is some configuration file in which I can define 'search path' for my project.
It would be grateful if I can find a way to solve it.
Best,
Je-Hoon Song

I had the same issue with module 'mpmath' and fixed it in the following manner:
First I retrieved the path where the module was located:
%python3
>>>import mpmath
>>>print(mpmath.__file__)
/usr/lib/python3.4/site-packages/mpmath/__init__.py
Here I found the path of all my "installed" python3 packages to be:
/usr/lib/python3.4/site-packages/
I then simply added to my PYTHONPATH environment variable this path:
%export PYTHONPATH=/usr/lib/python3.4/site-packages/
Then when I used vim sample.py typing import mpmath and following it up with mpmath. YCM showed me all the autocompletions for the mpmath module.
Hope this helps.

I use anaconda python to be my python interpreter in ycm to solve this.
First I modified my vimrc according to full pythong setting in vim.
Then I change g:ycm_python_interpreter_path by
let g:ycm_python_interpreter_path = '/usr/local/anaconda3/bin/python3.8'
In this way I didn't change the system environment variables.
Addtional Info 1:
I think the main problem is that,
my Python interpreter for YCM is my system python (/usr/local/opt/python#3.9/bin/python3.9 ),
which only has limited locally built libraries.
So using the libraries comes with anaconda (/usr/local/anaconda3/bin/python3.8 ) can solve.
Additional Info 2:
By reading :YcmDebugInfo, the main different after edited g:ycm_python_interpreter_path is that:
-- Python completer debug information:
-- Python interpreter: /usr/local/opt/python#3.9/bin/python3.9
-- Python path: ['/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/usr/local/lib/python3.9/site-packages']
-- Python version: 3.9.6
change to
-- Python completer debug information:
-- Python interpreter: /usr/local/anaconda3/bin/python3.8
-- Python path: ['/usr/local/anaconda3/lib/python38.zip', '/usr/local/anaconda3/lib/python3.8', '/usr/local/anaconda3/lib/python3.8/lib-dynload', '/usr/local/anaconda3/lib/python3.8/site-packages', '/usr/local/anaconda3/lib/python3.8/site-packages/aeosa', '/usr/local/anaconda3/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg']
-- Python version: 3.8.8
Additional Info 3: about how to read list of locally installed pyton modules
use https://stackoverflow.com/a/740018/11226687
e.g. in my case
$ /usr/local/opt/python#3.9/bin/python3.9
>>> help('modules')
# only return limitted modules
$ /usr/local/anaconda3/bin/python3
>>> help('modules')
# list out all the modules included in Anaconda, including numpy/matplotlib/scipy ect

numpy is kind of a difficult library because it dynamically builds its namespace on import, making it hard for static code analysis tools to know when you're write the code what names should be available. Because the names available in the namespace numpy are only really known at runtime, YCM probably doesn't have any useful suggestions for you.

One simple way to fix is activate your python environment, then open vim. For example
(django_mdn) ➜ locallibrary git:(master) ✗ vim
and in the vim run :echo $PATH.
Then you should be able to see that your venv path is at the first like this:
/Users/gwanghyeongim/.virtualenvs/django_mdn/bin:/usr/local/opt/tcl-tk/bin:...
Then see if your python packages are auto-complete.
It worked.
If you want to set a certain site-packages to be auto complete permanently, you need to make a file called .ycm_ extra_conf.py in your project root directory or global_extra_conf.py and set vim configuration if you want to set it globally.
P.S.
By running export PYTHONPATH=/usr/lib/python3.4/site-packages/ in the shell before opening vim didn't work for me. Besides, unless setting PYTHONPATH permanently, which will cause issue, you will have to set export PYTHONPATH everytime you want dependencies to be auto complete.

Python Throws ImportError: No module named..." Error

I'm trying to install the Chilkat library for Python in order to use its encryption functionality, but being new to Python in every possible way, I'm getting stuck entirely too early. I've installed the library as instructed by the docs and verified that the files are in the "right place" (/usr/local/lib/python2.7/site-packages/) on my Ubuntu 12.04 server.
I've also downloaded the test script. When I try to run it, however:
ImportError: No module named chilkat
I know this is stupid basic, but here I am. In the docs they do mention a possible issue with sys.prefix. That (presumably default, since I've never touched it) value on my machine is /usr. I moved everything there, but still get the same error.
Help? Where is the most "pythonic" place to put these files and how can I get Python to recognize them universally?
Thanks.

For anyone searching, I just ended up adding site-packages to my sys.path by adding a .pth file to dist-packages which was already in my path.
echo "/usr/local/lib/python2.7/site-packages" > /usr/local/lib/python2.7/dist-packages/site-packages.pth