I'm running a python script on a shared hosting server which until this morning had MySQL version 4. Now it has version 5. My python script can no longer connect to MySQL, as it can't find libmysqlclient_r.so.14:
$ python my_script.py
Traceback (most recent call last):
File "my_script.py", line 6, in ?
import MySQLdb
File "/home/lib/python2.4/site-packages/PIL-1.1.6-py2.4-linux-i686.egg/__init__.py", line 19, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 7, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 6, in __bootstrap__
ImportError: libmysqlclient_r.so.14: cannot open shared object file: No such file or directory
There are various other versions of libmysqlclient in /usr/lib:
/usr/lib/libmysqlclient.so.15
/usr/lib/libmysqlclient.so.14
/usr/lib/mysql/libmysqlclient.la
/usr/lib/mysql/libmysqlclient.so
/usr/lib/mysql/libmysqlclient_r.so
/usr/lib/mysql/libmysqlclient_r.a
/usr/lib/mysql/libmysqlclient_r.la
/usr/lib/mysql/libmysqlclient.a
/usr/lib/libmysqlclient.so
/usr/lib/libmysqlclient_r.so
/usr/lib/libmysqlclient_r.so.15
/usr/lib/libmysqlclient_r.so.15.0.0
/usr/lib/libmysqlclient.so.15.0.0
So my question is this: how can I tell python (version 2.4.3) which version of libmysqlclient to use?
You can't tell the dynamic linker which version of a library to use, because the SONAME (full name of the library + interface) is part of the binary.
In your case, you can try to upload libmysqlclient_r.so.14 to the host and set LD_LIBRARY_PATH accordingly, so tell the dynamic linker which directories to search additionally to the system dirs when resolving shared objects.
You can use ldd to see if it LD_LIBRARY_PATH works:
$ ldd $path_to/_mysql.so
...
libmysqlclient_r.so.14 => $path_to_lib/libmysqlclient_r.so.14
...
Otherwise, there will be an error message about unresolved shared objects.
Of course that can only be a temporary fix until you rebuild MySQLdb to use the new libraries.
You will have to recompile python-mysql (aka MySQLdb) to get it to link to the new version of libmysqlclient.
If your host originally set up the environment rather than you compiling it, you'll have to pester them.
/usr/lib/libmysqlclient.so.14
This looks like a remnant of the old libmysqlclient, and should be removed. The _r and .a (static) versions are gone and you don't really want a mixture of libraries still around, it will only risk confusing automake.
Whilst you could make a symbolic link from libmysqlclient_r.so.14 to .15, that'd only work if the new version of the client happened to have the same ABI for the functions you wanted to use as the old - and that's pretty unlikely, as that's the whole point of changing the version number.
One solution is to set your PYTHONPATH environment variable to have some local directory, and copy over (or link, I suppose) the version of the mysql lib you want.
Related
I want to use other versions of library for my pwn study in pwntools, but EOF error occurred.
I tried to solve this issue , changed ubuntu versions 3 times (18.04 desktop -> 14.04 desktop -> 18.04.0 server), reinstall python and pwntools 4 times.
currently, versions are ubuntu 18.04.0 server, Python 2.7.15rc1, pwntools 3.12.2
I tried using other versions library for my pwn study in pwntools.
like this:
p = process("./binary_name",env={"LD_PRELOAD" : "./libc_name"})
and tried also
env = {"LD_PRELOAD": os.path.join(os.getcwd(), "libc_name")}
p = process("./binary_name",env=env)
and excute python code, Error occurred
I already set the permisson of libc to chmod 777, but result is same.
[*] Process './aeiou' stopped with exit code -4 (SIGILL) (pid 77469)
Traceback (most recent call last):
File "ex4.py", line 6, in <module>
p.sendlineafter(">>","3")
File "/home/synod2/.local/lib/python2.7/site- packages/pwnlib/tubes/tube.py", line 747, in sendlineafter
~~~~~~~~~~~~~~
EOFError
I dont know why EOF error occurred. but, because of 3 differents version ubuntu give the same error, I think I missed install something.
but I don't know what I missed!
Maybe you should try it on Ubuntu 16.
Obviously your binary file is dynamic linked. So when the program need to call some libc function such as read. It will pass some information to the dynamic linker, then the linker will calculate the real address of the read function.
but functions in libc has a version attribute. So if you try to use LD_PRELOAD on Ubuntu 18.04. the dynamic linker would try to find sth like read_2_27 in you 2.23-version-libc which only have read_2_23. so your program would fail to execute.
UPDATE:
another solution is to tell the excutable file to use the correct version of ld.so
elf file has a segment(INTERP) in which save the path to the ld.so to use. you can just change it to the path to ld.so you want to use.
BTW, you can find many version of ld.so in the repository
When I import a module I built, I get this boost-python related error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: dlopen(./myMod.so, 2): Symbol not found: __ZN5boost6python7objects15function_objectERKNS1_11py_functionERKSt4pairIPKNS0_6detail7keywordES9_E
Referenced from: ./myMod.so
Expected in: flat namespace
in ./myMod.so
What does this actually mean? Why was this error raised?
Description
The problem was caused by mixing objects that compiled with libc++ and object that compiled with libstdc++.
In our case, the library myMod.so (compiled with libstdc++) need boost-python that compiled with libstdc++ (boost-python-libstdc++ from now). When boost-python is boost-python-libstdc++, it will work fine. Otherwise - on computer that its boost-python has compiled with libc++ (or another c++ library), it will have a problem loading and running it.
In our case, it happens because that libc++ developers intentionally changed the name of all of their symbols to prevent you (and save you) from mixing code from their library and code from a different one: myMod.so need a function that take an argument from the type. In libc++, this type's name is std::__1::pair. Therefore, this symbol was not found.
To understand why mixing two version of the same API is bad, consider this situation: There are two libraries: Foo and Bar. They both have a function that takes a std::string and uses it for something but they use a different c++ library. When a std::string that has been created by Foo will be passed to Bar, Bar will think that this is an instance of its c++ library's std::string and then bad things can happen (they are a completely different objects).
Note: In some cases, there would be no problem with two or more different versions of the same API in a completely different parts of a program. There will be a problem if they will pass this API's objects between them. However, checking that can be very hard, especially if they pass the API object only as a member of another object. Also, a library's initialization function can do things that should not happen twice. Another version may do these things again.
How to solve that?
You can always recompile your libraries and make them match each other.
You can link boost-python to your library as a static library. Then, it will work on almost every computer (even one that doesn't has boost-python installed). See more about that here.
Summary
myMod.so need another version of boost-python, one that compiled with a specific c++ library. Therefore, It would not work with any another version.
In my case I was receiving:
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmlsec.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '_xmlSecDSigNs'
BACKGROUND:
M1 MacBook Pro with Montery
I was working with a python virtualenv (using pyenv) to use an earlier version of python3.8 (3.8.2), while my system had 3.8.10 installed natively.
While I was in the activated 3.8.2 virtualenv I noticed the path in dlopen() was pointing to the package in the native python install NOT the virtualenv install.
SOLUTION:
In my case, I did not need the native 3.8 version at all so I simply removed it and this solved the problem.
I encounter the same problem.
Expected in: flat namespace
Add the linker flag fixes the problem
-lboost_python37
change the dynamic library name to the one installed on the os.
By the way, my os is macOS High Sierra and I use brew to install boost_python3.
Symbol not found means the definition of the declared function or variable was not found. When a header file of a shared object is compiled with your program, linker adds symbols of declared functions and objects to your compiled program. When your program is loaded by the OS's loader, the symbols are resolved so that their definition will be loaded. It is only at this time where if the implementation is missing, loader complains it couldn't find the definition due to may be failing to resolve the actual path to the library or the library itself wasn't compiled with the implementation/source file where the definition of the function or object resides. There is a good article on this on the linux journal http://www.linuxjournal.com/article/6463.
In my case I was just failing to import all the required sources (c++ files) when compiling with Cython.
From the string after "Symbol not found" you can understand which library you are missing.
One of the solutions I found was to uninstall and reinstall it using the no-binary flag, which forces pip to compile the module from source instead of installing from precompiled wheel.
pip install --no-binary :all: <name-of-module>
Found this solution here
Here's what I've learned (osx):
If this is supposed to work (i.e. it works on another computer), you may be experiencing clang/gcc issues. To debug this, use otool -l on the .so file which is raising the error, or a suspect library (in my example it's a boost-python dylib file) and examine the contents. Anything in the /System/ folder is built with clang, and should be installed somewhere else with the gcc compiler. Never delete anything in the /System folder.
.so files are dynamic libraries (so = shared object). On Windows they are called .dll (dynamic-link library). They contain compiled code which contains functions available for usage to any executable which links them.
What is important to notice here is that those .so are not Python files. They were probably compiled from C or C++ code and contain public functions which can be used from Python code (see documentation on Extending Python with C or C++).
On your case, well, you have a corrupt .so. Try reinstalling the affected libraries, or Python, or both.
Problem
I had this same issue when running puma as part of Rails app
LoadError:
dlopen(/Users/alucard/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/puma-5.6.4/lib/puma/puma_http11.bundle, 0x0009): symbol not found in flat namespace '_ERR_load_crypto_strings'
/Users/alucard/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/puma-5.6.4/lib/puma/puma_http11.bundle
Solution
It was solved just by installing puma gem again gem install puma
I'm currently using the Python 3.4 Mac OS X build from Python.org. I'm using a Python module that depends on a library that I built in Macports. The script does not run out-of-the-box:
Traceback (most recent call last):
File "magnetx.py", line 6, in <module>
import yara
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so, 2): Library not loaded: /usr/local/lib/libyara.3.dylib
Referenced from: /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so
Reason: image not found
I can fix this if I set an environment variable
export DYLD_FALLBACK_LIBRARY_PATH="/opt/local/lib:$DYLD_FALLBACK_LIBRARY_PATH"
Unfortunately, it does not satisfy cx_freeze. It keeps looking in /usr/local/lib, when it should be looking in /opt/local/lib.
copying
/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so
-> build/exe.macosx-10.6-intel-3.4/yara.so copying /usr/local/lib/libyara.3.dylib ->
build/exe.macosx-10.6-intel-3.4/libyara.3.dylib error: [Errno 2] No
such file or directory: '/usr/local/lib/libyara.3.dylib'
I could probably build Python in Macports, but that seems like it should be unnecessary. Any ideas on how to fix this?
On OS X, dependent libraries are referenced using absolute paths. The path that gets copied into your binary depends on the so-called "install name" of the library you link against at build time. In your case, the yara.so does not reference the library you would like it to load. Let's explore a couple of reasons why this could be the case, and a couple of ways to fix that:
I've verified that libyara.dylib as installed by MacPorts (on my system) has an install name of /opt/local/lib/libyara.0.dylib. Sometimes, build systems that don't use a cross-platform library build tool and don't expect the peculiarities of OS X mess this up (and use relative paths or /usr/local/lib). If this was the case, it would be a bug in the software's build system, which could be manually fixed using install_name_tool(1)'s -id flag (before linking against the library).
Your copy of yara.so may have been built against a different version of libyara.dylib that resides in /usr/local/lib. That would explain why your yara.so does not contain the correct absolute path to the MacPorts copy of libyara.dylib, but it would also prevent the error you're seeing from happening in the first place, unless you had a copy in /usr/local/lib at build time and deleted it later on. As you've already seen, you can instruct OS X' loader to also search different paths using the DYLD_* series of environment variables. My take on why this doesn't work for cx_freeze is that it doesn't pay attention to the DYLD_* series of variables.
If you are sure that the copy of libyara.dylib yara.so expects to find in /usr/local/lib is binary-compatible with the one in /opt/local/lib, you can manually modify the library load commands in yara.so to point to the latter path using install_name_tool(1)'s -change old new parameter, e.g. install_name_tool -change /usr/local/lib/libyara.3.dylib /opt/local/lib/libyara.0.dylib /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so. This is essentially modifying the binary with the change the loader did for you when you set DYLD_FALLBACK_LIBRARY_PATH. Since the library major version numbers seem to be different, this may not be a safe assumption.
If you don't know whether yara.so is compatible with MacPorts' build of libyara.0.dylib, you can and should recompile yara.so. If the re-compile went right, you should be able to check the library load commands using otool -L yara.so and see the paths beginning with /opt/local in there (provided that otool -D /opt/local/lib/libyara.0.dylib correctly points to itself).
Edit: I've just re-checked and noticed that my MacPorts build's library version number differs from the one your system expects. That sounds a lot like case number 2 to me.
I've just installed MySQL on my 10.10.1 MAC, and I'm trying to link it to Python. However I get this error when I'm trying to import the package.
import MySQLdb as mdb
File "build/bdist.macosx-10.5-x86_64/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/ME/.python-eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so, 2): Library not loaded: libssl.1.0.0.dylib
Referenced from: /Users/ME/.python-eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so
Reason: image not found
[Finished in 0.2s with exit code 1]
I have no idea what "Image not found" means, or how to solve this.
MySQL is connected to the server in the background, and I installed the package through the terminal window using: easy_install MySQL-python
Any suggestions on where I went wrong?
Check where your _mysql.so is linking to:
otool -L /Library/Python/2.7/site-packages/_mysql.so
In my case I built the mysql source in /tmp/ and once
tmp cleaned up my dylib went away. I never re-linked to a
reliable location like /Applications/MAMP/Library/lib where
my mysql libraries live (because I copied them here after the build).
So I went back and rebuilt everything and made sure to run the command to
update the package link to the correct dylib:
sudo install_name_tool -change <temp file location>/mysql-5.5.29/libmysql/libmysqlclient.18.dylib /Applications/MAMP/Library//lib/libmysqlclient.18.dylib /Library/Python/2.7/site-packages/_mysql.so
I followed the setup found here:
http://dreamconception.com/tech/how-to-install-mysqldb-mysql-python-on-mamp/
NOTE: That I didn't need to do the step regarding changing the configure.cmake
file. When I looked at the configure.cmake file it made sense to me so I ran
it as-is and it worked.
Also note your libmysqlclient.<num>.dylib might be something different so
do a find . -name "libmysqlclient.*.dylib" to get the right one
Now re-run the otool command and see that the linking was updated.
References (and other solutions):
Python import MySQLdb error - Mac 10.6
http://dreamconception.com/tech/how-to-install-mysqldb-mysql-python-on-mamp/
The solution to the problem can be found in this link Python: MySQLdb and "Library not loaded: libmysqlclient.16.dylib"
_mysql.so refers to libmysqlclient.16.dylib. That is, the shared library that serves as the bridge between Python and the MySQL client library, _mysql.so, refers to the dynamic library for the MySQL client library, and that library cannot be loaded for some reason.
Questions you need to answer:
Is there a libmysqlclient.16.dylib anywhere on your system? If not, you need to install the MySQL client software.
If so, is the directory containing that library in your DYLD_LIBRARY_PATH setting? If not, try adding it.
If so, you'll have to ensure that the libmysqlclient.16.dylib file is not corrupt. My copy, installed in /opt/local/lib/mysql5/mysql/libmysqlclient.16.dylib, courtesy of MacPorts, has MD5 signature c79ee91af08057dfc269ee212915801a and is 1,462,376 bytes in size. What does your copy look like?
I am trying to load a *.pyd with Python, but I receive the well known "Import Error: DLL load failed: the specified procedure can not be found." error.
I have already done the following:
1.) Investigated the *.pyd with Dependency Walker. GPSVC.DLL and IESHIMS.DLL came up as missing, but delay loaded, IEFRAME.DLL aslo came up as missing an export, but was also delay-loaded. It's my understanding that these are not used, and are delay load anyway, so they should not be the problem.
2.) Did an "import foo" on foo.pyd in the python command window, with ProcMon watching. ProcMon shows event "LoadImage" on "foo.pyd" with result SUCCESS.
This seems to imply that the *.pyd file loaded correctly.
So what am I missing. My windows diagnostics are telling me all is well, but python is telling me the thing cannot be loaded (usually due to a missing dll or symbol).
Ideas?
Thanks!
Is the .pyd file for the same version of Python you're using? Loading a .pyd file for the wrong Python version can produce that error message.
Dependency Walker can show you which pythonNN.dll it links to.
If you have a file foo.pyd, for import foo to succeed, there must be an externally accessible function named initfoo. Dependency Walker will show this (typically the ONLY function) if it exists. initfoo needs to be called by Python to initialise the foo module.
Note: I would expect a more explicit error message if this were the problem:
>>> import fubar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define init function (initfubar)
>>>
You say that you are "trying to load a *.pyd file". Is that just a strange way of describing import foo or is it something else?
Did you create the pyd? If not, who did? Have you asked them? Is this pyd available on the web so that others could try to load/import it?
Ok here is the answer:
The windows diagnostics (depends, procmon, etc) were showing the DLL (or pyd) loading fine.
Python was showing that it was not loading fine.
I found that the windows tools were referring to a different Python26.dll hiding in my C:\Window\SysWOW64 folder.
This second Python26.dll (found in SysWOW64) has a symbol that is missing in the primary python26.dll (installed by the windows python installer, found in C:\Python26).
This symbol "_PyByteArray_empty_string", was apparently needed by my *.pyd file.
So when loading via windows diagnostics, the SysWOW64 dll was found, and the *.pyd loaded properly. When loading from python, the dll in C:\Python26\ was found, the symbol was missing, and load failed.
So that is WHY the problem manifested. The question now is: Why are there two versions of Python26.dll floating around, one with _PyByteArray_empty_string, and one without?
I'm using Python 2.6.6. Perhaps this symbol is removed in 2.6.6 but present in some older 2.6.x release?