I'm trying to run this code in vscode for a school task, but I keep getting an error of a missing file 'libc.dll' I am somewhat confused of this file since I am new to this and not sure where to look for this file...
Can someone explain to me what does this error actually mean and why I can't locate the file?
from ctypes import *
libc = CDLL("libc.dll")
libc.printf("hello everybody\n".encode('ascii'))
The error I'm getting:
Traceback (most recent call last):
File "c:\Users\User\Desktop\2-3.py", line 3, in
libc = CDLL("libc.dll")
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\ctypes_init_.py", line 376, in init
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'libc.dll' (or one of its dependencies). Try using the full path with constructor syntax.
[GNU]: The GNU C Library (glibc) (if this is what your libc.dll refers to) states (emphasis is mine):
The GNU C Library project provides the core libraries for the GNU system and GNU/Linux systems, as well as many other systems that use Linux as the kernel.
So, LibC is part of Nix world (which Win is not in).
MS's equivalent is CRT (also referred to as VCRT, MSVCRT, UCRT, ...: [MS.Learn]: Microsoft C runtime library (CRT) reference).
According to [Python.docs]: ctypes - Loading dynamic link libraries (make sure to also read the notes):
Here are some examples for Windows. Note that msvcrt is the MS standard C library containing most standard C functions, and uses the cdecl calling convention.
Example:
>>> import ctypes as cts
>>> import sys
>>>
>>> sys.version
'3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]'
>>>
>>>
>>> crt = cts.CDLL("msvcrt")
>>>
>>> crt.printf
<_FuncPtr object at 0x0000019AE5A48040>
You should also check:
[SO]: Can't import dll module in Python (#CristiFati's answer)
[SO]: C function called from Python via ctypes returns incorrect value (#CristiFati's answer) (when calling printf (or any function as a matter of fact))
Note: I also tried (a while ago) your code on Nix emulators (MSYS2, Cygwin) but they didn't work. The only thing that would work for sure is WSL2 ([MS.Learn]: Install Linux on Windows with WSL), as it has a genuine Linux kernel, but I'm not sure if that's relevant for you.
Related
I ask this question in the context of most other questions being 5-10 years old on this topic and given the following
Windows 7 OS
Several versions of a .pyd compiled dll at different file path locations e.g.
/ver1.0/lib/my_dll.pyd
/ver1.1/lib/my_dll.pyd
my_dll.pyd is imported from a secondary file as from my_dll import *, I am not in control of the method or at liberty to alter
I can successfully switch between two .pyd versions as follows
import sys
sys.path.append(r'\path\to\ver_1.0')
import my_dll # imports ver 1.0
print my_dll.release_version()
sys.path.pop(-1)
del sys.modules['my_dll'] # remove module dict ref
from IPython import get_ipython # just in case you run in iPython
get_ipython().magic('reset -sf') # need to remove history!
del my_dll # delete the import object
import _ctypes # release the library handle
import ctypes
dll = ctypes.CDLL('my_dll.pyd')
_ctypes.FreeLibrary(dll._handle) # for some reason it needs to be
_ctypes.FreeLibrary(dll._handle) # done twice.
sys.path.append(r'\path\to\ver_1.1')
import my_dll
print my_dll.release_version()
The failures of reload are overcome by reloading the C++ .pyd as per the following output
Version 1.0
Version 1.1
my issue is that then I am left with an extremely flimbsy python ecosystem that will crash at almost any new function call.
Is there a more updated methodology of doing this?
References
ctypes unload dll
How can I unload a DLL using ctypes in Python?
forcing ctypes.cdll.LoadLibrary() to reload library from file
Python runtime: recompiling and reusing C library
https://bugs.python.org/issue14597
My initial goal is to open a dll file on Cygwin using ctypes. However I found some issues with it. I dug up to sys.dl which returns an unknown Permission denied only on IPython.
With python everything looks fine:
$ ls
my.dll
$ python
Python 2.7.8 (default, Jul 28 2014, 01:34:03)
[GCC 4.8.3] on cygwin
>>> import dl
>>> dl.open('my.dll')
<dl.dl object at 0xfffaa0c0>
With ipython I get the error:
$ ipython
Python 2.7.8 (default, Jul 28 2014, 01:34:03)
In [1]: import dl
In [2]: dl.open('my.dll')
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-2-c681630fa713> in <module>()
----> 1 dl.open('my.dll')
error: Permission denied
I investigated on this using strace. The output log for `IPython is huge, more than 4MB. Fortunately, I identified some weird things:
symlink.check(C:\Users\user\Home\projects\foo\my.dll, 0x28AB88) (0x4022)
35 2705178 [main] python2.7 16924 path_conv::check: this->path(C:\Users\user\Home\projects\foo\my.dll), has_acls(1)
37 2705215 [main] python2.7 16924 cwdstuff::get: posix /cygdrive/c/Users/user/Home/projects/foo
32 2705247 [main] python2.7 16924 cwdstuff::get: (C:\Users\user\Home\projects\foo) = cwdstuff::get (0x8006ECF0, 32768, 0, 0), errno 11
--- Process 14376, exception c0000138 at 7726163E
3286 2708533 [main] python2.7 16924 seterrno_from_win_error: /home/corinna/src/cygwin/cygwin-1.7.35/cygwin-1.7.35-1.i686/src/src/winsup/cygwin/dlfcn.cc:174 windows error 182
42 2708575 [main] python2.7 16924 geterrno_from_win_error: unknown windows error 182, setting errno to 13
36 2708611 [main] python2.7 16924 dlopen: ret 0x0
Who is /home/corinna? I have no corinna user in my installation, neither on my Windows. Corinna does not come from my installation. Is it some hard-coded stuff?
Now, here is what I get from strace for python:
symlink.check(C:\Users\user\Home\projects\foo\my.dll, 0x28B728) (0x4022)
26 10440048 [main] python 12604 path_conv::check: this->path(C:\Users\user\Home\projects\foo\my.dll), has_acls(1)
23 10440071 [main] python 12604 cwdstuff::get: posix /cygdrive/c/Users/user/Home/projects/foo
25 10440096 [main] python 12604 cwdstuff::get: (C:\Users\user\Home\projects\foo) = cwdstuff::get (0x8006ECF0, 32768, 0, 0), errno 0
3405 10443501 [main] python 12604 dlopen: ret 0x5B9C0000
dlopen is returning 0x0 in IPython while it is returning 0x5B9C0000 for python. I notice that cwdstuff::get is raising an error before dlopen is called.
EDIT
I sent a message to Cygwin's mailing list and the answer of Corinna regarding this issue is:
This is not Cygwin's fault, AFAICS. Cygwin never loads functions by
ordinal. This is also a bit on the lean side as far as information is
concerned. One can't see how the process calls dlopen, for instance.
Corinna
How to solve this issue?
My earlier tests using ctypes
Initially when I asked my question I was just playing with ctypes.
I am working on Cygwin 32-bit and Windows 7.
With IPython I got an OSError when I tried to load a dll using cdll.LoadLibrary.
Two ideas:
1) in the next cell, type %pdb, and then interactively "print self._name" to see what it is.
2) Use a full path to cdll.LoadLibrary("foo.dll") to see if that works.
Once you know what the issue is, then you can decide whose bug it is, and report it (could be a ctypes issue, but probably ipython)
More ideas:
Check that the user accessing the DLL is the same. You do that like this:
import getpass
print(getpass.getuser())
Check what's the current process actually doing. I haven't used cygwin but in the linux shell the executable strace should show you this.
Usage: get the PID of your current process: import os; os.getpid()
After this, you can use (from outside the python/ipython console the command strace -p <the pid> -e file. After this setup, you can try to load your DLL.
Remarks: the -e file flag should be written exactly like that. The word file tells strace to report all file operations that the process makes. If no differences occur when running on python/ ipython, you can try dropping the -e file flag. Then you will see all system calls that the process makes. I haven't worked on windows like that, so this might not work at all there, but on linux at least, this should report to you everything that the process did. You could see there at least all the files that got opened, but more interesting things could be found there as well. If the outputs are identical, then the problem can be debugged further in python/ ipython. This would require basically what #Doug Blank suggested, but I'd also recommend investigating every name (variable) that gets touched. The self, _dlopen and mode names also sound like they might contain useful information.
Otherwise, do a dir(self) and dir(_dlopen) to see what other properties you could find there that might have gotten modified by IPYthon.
Try these first, and after that we can help you dig further.
Perhaps Python executable and IPython kernel use different manifest files which define loading policy?
Try appending the DLL path to sys.path in both cases.
Check admin rights (UAC) in both cases.
Use dependency walker to figure out dependencies of this DLL. Maybe the problem comes from dependencies?
Possibly your machine has multiple copies of this DLL?
Finally you can use Process Explorer to see the list of DLLs loaded in both cases and see any differences.
I'm working on a very similar issue:
ipython notebook & script difference - loading DLLs
I am trying to cross compile Python 2.7.3 for an arm based embedded device. I have managed to compile it successfully (based on these instructions: http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html) and all of the tests pass on the target device so I'm confident that the build process works. I've cross compiled sqlite3 (version 3.8.5) and included it in the python cross compile process which it seems to pick up fine (it is no longer listed in the modules which were not found at the end of the build process).
I'm having difficulty actually trying to import the sqlite3 library on the target device, I get the error listed below (python is running with the -v flag).
Python 2.7.3 (default, Jul 7 2014, 19:06:12)
[GCC 3.4.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
import sqlite3 # directory /mnt/card/arm-python/lib/python2.7/sqlite3
# /mnt/card/arm-python/lib/python2.7/sqlite3/__init__.pyc matches /mnt/card/arm-python/lib/python2.7/sqlite3/__init__.py
import sqlite3 # precompiled from /mnt/card/arm-python/lib/python2.7/sqlite3/__init__.pyc
# /mnt/card/arm-python/lib/python2.7/sqlite3/dbapi2.pyc matches /mnt/card/arm-python/lib/python2.7/sqlite3/dbapi2.py
import sqlite3.dbapi2 # precompiled from /mnt/card/arm-python/lib/python2.7/sqlite3/dbapi2.pyc
dlopen("/mnt/card/arm-python/lib/python2.7/lib-dynload/datetime.so", 2);
import datetime # dynamically loaded from /mnt/card/arm-python/lib/python2.7/lib-dynload/datetime.so
dlopen("/mnt/card/arm-python/lib/python2.7/lib-dynload/time.so", 2);
import time # dynamically loaded from /mnt/card/arm-python/lib/python2.7/lib-dynload/time.so
dlopen("/mnt/card/arm-python/lib/python2.7/lib-dynload/_sqlite3.so", 2);
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/card/arm-python/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/mnt/card/arm-python/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: File not found
It seems to be complaining about a "file not found" but I've been through all of the paths listed in the output an all of the files seem to exist. Is there anything I can do to diagnose this problem further?
I've managed to get it working although I don't think I fully understand what's going on, I'm not that familiar with compiling C/C++ code and how the whole static/shared libraries and linking works, maybe someone can shed some light on what's actually going on here. Anyway, here's how I've resolved it.
First I ran strace (Linux debugging utility) on the python process:
strace /mnt/card/arm-python/bin/python
This spits out a load of output as the python process starts, once it's settled down I tried to import the sqlite library:
import sqlite3
This will then spit out a load more output, most of it relating to opening the files involved in the module your are importing. I then noticed it managed to open _sqlite3.so but shortly afterwards it tried to load libsqlite3.so which wasn't found on the library paths ($LD_LIBRARY_PATH).
open("/mnt/card/arm-python/lib/python2.7/lib-dynload/_sqlite3.so", O_RDONLY|O_LARGEFILE) = 5
...
open("/lib/libsqlite3.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/libsqlite3.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
I copied libsqlite3.so.0.8.6 from the /lib directory within the cross compiled sqlite library on my build machine to /mnt/card on the embedded arm device and renamed it to libsqlite3.so.0. I then added /mnt/card to the $LD_LIBRARY_PATH as the existing locations in that path reside on the read-only filesystem.
I then tried to import sqlite3 again and it all seem to be working fine. So what is the role of the stuff in the lib-dynload and also the role of libsqlite3.so.0?
I have installed Python 2.6.7 in $HOME/local of a machine which already has a default Python in /usr (I don't have admin access on this machine). The default Python is compiled in 32bits and my local installation is a 64bits. For some unknown reasons my local Python library (which I call as an embedded python interpreter from within a C program) search for the modules in the default (wrong) installation.
This is the result of "import random"
Traceback (most recent call last):
File "test.py", line 3, in <module>
import random
File "/tmp/work/mdorier1/local/lib/python2.6/random.py", line 45, in <module>
from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
ImportError: /usr/lib/python2.6/lib-dynload/math.so: wrong ELF class: ELFCLASS32
As you can see, the import statement correctly search "random.py" in the local installation of Python, but the import statement in random.py go search for math.so in the wrong location, which ends in an error since the default location has 32 bits modules.
I guessed there is a problem with an environment variable, and I tried
import sys
sys.path
to get
['/tmp/work/mdorier1/local/lib/python26.zip',
'/tmp/work/mdorier1/local/lib/python2.6',
'/tmp/work/mdorier1/local/lib/python2.6/plat-linux2',
'/tmp/work/mdorier1/local/lib/python2.6/lib-tk',
'/tmp/work/mdorier1/local/lib/python2.6/lib-old',
'/usr/lib/python2.6/lib-dynload',
'/tmp/work/mdorier1/local/lib/python2.6/site-packages']
I noticed that on of the paths indeed points to the default installation of Python. My questions thus are:
- Why this path shows up here, as the local installation has nothing to do with the default one?
- How do I change it (in a clean and permanent way)? This path should be the path to lib-dynload in the local installation instead.
Thanks
You probably don't have your Python's bin directory in the PATH variable before the system Python.
Or perhaps you simply compiled your Python incorrectly and did not use:
./configure --prefix=/tmp/work/mdorier1/local
so now it thinks that its files are somewhere else.
import itertools
print itertools#ok
the code is ok
but i can't find the itertools file.
who can tell me where is the 'itertools file'
my code is run python2.5
import itertools
print itertools.__file__
Traceback (most recent call last):
File "D:\zjm_code\mysite\zjmbooks\a.py", line 5, in <module>
print itertools.__file__
AttributeError: 'module' object has no attribute '__file__'
>>> import itertools
>>> itertools.__file__
'/usr/lib64/python2.6/lib-dynload/itertools.so'
The fact that the filename ends with .so means it's an extension module written in C (rather than a normal Python module). If you want to take a look at the source code, download the Python source and look into Modules/itertoolsmodule.c (you can also view it in your browser at http://svn.python.org/view/python/trunk/Modules/itertoolsmodule.c?view=log).
Edit (as an answer to the comment below): it also works with Python 2.5:
Python 2.5.2 (r252:60911, Oct 5 2008, 19:29:17)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import itertools
>>> itertools.__file__
'/usr/lib/python2.5/lib-dynload/itertools.so'
If you're looking for the source file (in C, of course), it's for example online here.
For the windows users (I'm running Python 2.7.2, Win7x64, default installer package) the call to __file__ will flame out as #zjm1126 has noted, I suspect the problem being that itertools is a builtin on the windows package. If you'd picked say, exceptions? You'd get the same behaviour on another platform (e.g. Python 2.6.1 on my macbook) - Windows just happens to have a few more builtins like itertools.
It's not strictly an answer as such, but you could parse sys.modules which would give you a hint as to where it's coming from:
>>> import sys
>>> sys.modules['itertools']
<module 'itertools' (built-in)>
which points at itertools being built-in to your python executable.
Also, the imp.find_module response is providing the same information: the weird return tuple is by spec (see: http://docs.python.org/2/library/imp.html#imp.find_module) and telling you that the module is of type 6, which is the enumeration for imp.C_BUILTIN
try this
>>> import imp
>>> imp.find_module("itertools")
update:
since yours is None, another go through a manual way. Do a sys.path
>>> import sys
>>> sys.path
['', '/usr/lib/python2.6/lib-dynload' ]
then depending on your system, use your system's search facility to find it. on my linux system
$ find /usr/lib/python2.6/lib-dynload -type f -iname "*itertools*"
/usr/lib/python2.6/lib-dynload/itertoolsmodule.so
OR, just search the entire system for the file with name "itertools".