I installed python 2.7.17 on a windows 10 machine. I then wanted to test its openssl version by running the following inside python:
import ssl
print ssl.OPENSSL_VERSION_INFO
I am getting (1, 0, 2, 20, 15)
I wanted to upgrade to version 1.1.1.
Doing pip freeze I get:
cffi==1.14.0
cryptography==2.8
enum34==1.1.6
ipaddress==1.0.23
pycparser==2.19
pyOpenSSL==19.1.0
six==1.14.0
These seem to be the latest in pip for pyOpenSSL and cryptography.
The openssl I have installed (as part of git bash) is 1.1.1, however, this is not the same version used inside python.
How do I upgrade the version of openssl included in python to 1.1.1 or greater?
EDIT:
In response to the comments, following is a result of python -m OpenSSL.debug:
C:\Users\assaf>python -m OpenSSL.debug
pyOpenSSL: 19.1.0
cryptography: 2.8
cffi: 1.14.0
cryptography's compiled against OpenSSL: OpenSSL 1.1.1d 10 Sep 2019
cryptography's linked OpenSSL: OpenSSL 1.1.1d 10 Sep 2019
Pythons's OpenSSL: OpenSSL 1.0.2t 10 Sep 2019
Python executable: C:\Python27\python.exe
Python version: 2.7.17 (v2.7.17:c2f86d86e6, Oct 19 2019, 21:01:17) [MSC v.1500 64 bit (AMD64)]
Platform: win32
sys.path: ['', 'C:\\WINDOWS\\SYSTEM32\\python27.zip', 'C:\\Python27\\DLLs', 'C:\\Python27\\lib', 'C:\\Python27\\lib\\plat-win', 'C:\\Python27\\lib\\lib-tk', 'C:\\Python27', 'C:\\Python27\\lib\\site-packages']
As stated above, cryptography and pyOpenSSL are the latest in pip.
The only way to get Python 2.7 to use the current OpenSSL 1.1.1d version for its ssl module is to rebuild it with that version of OpenSSL. For *nix platforms, this is not too hard; it only involves adjusting some initial configuration files. A quick test on macOS showed that the Python 2.7 source code was actually compatible with the OpenSSL 1.1.1d API so that looked promising.
For Windows however, rebuilding Python 2.7 with OpenSSL 1.1.1d is harder. This is because the build system, which relies on Microsoft's msbuild, is not as easy to adjust as the make-based build system on *nix. You can see the associated Visual Studio solution, projects and property files in Python's PCbuild subdirectory.
Additionally, the differences between the OpenSSL 1.0.2 and 1.1.1 versions is slightly larger on Windows, because library names have changed as well, whereas they have stayed the same for the *nix version.
The required modifications to Python's build system for Windows are not too extensive though. It looks like I was able to achieve what you are looking for, after making the change explained at the bottom of this answer. The actual build can be done with the following command run from Python's PCbuild subdirectory:
> build --no-tkinter --no-bsddb -e "/p:PlatformToolset=v141"
for a 32-bits build, or
> build --no-tkinter --no-bsddb -e -p x64 "/p:PlatformToolset=v141"
for a 64-bits build. The --no- options are used to speed up the process and focus on the OpenSSL aspect. v141 stands for Visual Studio 2017, you need to be in a Visual Studio shell for this to work. After that, the following reproduced your test showing the use of OpenSSL 1.1.1d with Python 2.7.17:
> win32\python.exe
Python 2.7.17 (tags/v2.7.17-dirty:c2f86d86e6, Feb 20 2020, 01:04:36) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> print ssl.OPENSSL_VERSION
OpenSSL 1.1.1d 10 Sep 2019
I did not do any testing beyond this.
In summary, the changes to achieve this include
Removed the libeay project from the solution. This was used to build OpenSSL 1.0.2 from source but it is not compatible with OpenSSL 1.1.1. Instead, the solution now relies on the prebuilt OpenSSL 1.1.1d binaries as provided by the Python repo in cpython-bin-deps. Rebuilding OpenSSL yourself as part of the build process is possible but requires more modifications.
Modified the get_externals.bat batch script to download the OpenSSL 1.1.1d prebuilt libraries from the aforementioned cpython-bin-deps repository.
Modified the openssl.props property file that configures several of the _ssl and _hashlib project settings, and made some changes to those projects themselves as well. The adjustments mostly taken from the v3.8.1 release of Python, to link with the new OpenSSL libraries and set the include paths correctly. With this modification, OpenSSL is no longer statically linked as it used to be in 2.7.17, but dynamically linked.
Applied patches to the files Modules/posixmodule.c and Modules/timemodule.c similar to this patch, to allow for building with Visual Studio 2017 -- the same version the prebuilt OpenSSL binaries are created with.
By the way, more stuff than needed gets downloaded in the current build process, including the OpenSSL and nasm source code. This is only to allow for as few modifications to the original build scripts as possible.
If you are interested in the details, you can check out the associated commit in this fork of the cpython repo which I did just for the purpose of clarifying my answer. It is based on the original tag v2.7.17. You should be able to reproduce the build by checking out that branch v2.7.17_ossl_1.1.1 and running the build command in PCbuild as mentioned above. If enough people are interested, I may clean it up and keep it around.
Related
Encountering an JPG image which wouldn't load with pyexiv2 v2.6.4, I upgraded to pyexiv2 v2.7, via pip install, which went through without complaints.
Now pyexiv2 doesn't load at all anymore, complaining
FileNotFoundError: Could not find module 'C:\Program Files\Python\3.8\lib\site-packages\pyexiv2\lib\exiv2.dll'. Try using the full path with constructor syntax.
The file exists (os.path.exists("...")==True just before the _dlopen() throwing the exception), and was just installed fresh with the package. I am running "Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 64 bit (AMD64)] on win32".
Any ideas? Thanks!
EDIT: Two more bits of information:
I vaguely remember that when installing the previous pyexiv2 version, the installation compiled something called exiv2. This did not happen during installing pyexiv2 v2.7. Although I do find only the windows subdirectories in the install directory.
Replacing v2.7 by v2.6.4 again (from a safety copy :-) does not help - now it's broken as well :-(
Looking at Python docs for the ctypes module, I saw a hint that maybe it's a problem with additional libraries required by exiv2.dll.
After installing "Visual Studio 2022 Release Candidate" from Visual Studio Download (be sure to check the "build tools" checkbox when the installer asks for what to install), all is well again.
I have upgraded my RHEL OS from 6.7 to 7.5. After upgrading, I found some issues when trying to run yum. Below are the details.
# yum repolist
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
/lib64/libc.so.6: version `GLIBC_2.14' not found (required by /lib64/libgcc_s.so.1)
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
2.6.6 (r266:84292, Aug 9 2016, 06:11:56)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
If you cannot solve this problem yourself, please go to
the yum faq at:
http://yum.baseurl.org/wiki/Faq
After getting this error, I just installed python2.7 and GLIBC 2.14. But when I am trying to install GLIBC 2.14 from my current GLIBC version 2.12, it is throwing some error. Below are the steps that I am using to install GLIBC 2.14:
tar xvfz glibc-2.14.tar.gz
cd glibc-2.14
mkdir build
cd build
../configure --prefix=/opt/glibc-2.14
make
sudo make install
export LD_LIBRARY_PATH=/opt/glibc-2.14/lib:$LD_LIBRARY_PATH
In step5, I am getting error. Below are the details:
# ../configure --prefix=/opt/glibc-2.14
checking for forced unwind support... no
configure: error: forced unwind support is required
I am unaware of this error "unwind support is required".Please let me know how to setup/install forced unwind in Redhat 7.5.
add libc_cv_forced_unwind=yes when configure:
../configure --prefix=/opt/glibc-2.14 libc_cv_forced_unwind=yes
and make
I am using python in the MSYS2 environment. The MSYS2 has its own built MINGW python version. Also I can install the official python from the www.python.org.
Here is the problem: If I want to write a python code need to know the python version is MINGW or the official one, how can I do it?
Here are some ways I can image.
Use the "sys.prefix" object. It tells the installation directory. the MSYS2 usually installed in the directory X:\msys2\.... and the official one install in the X:\Python27\ as default. But users may change the installation directory. So this is not a good way.
Use the "sys.version" object can get the version strings show with the compiler name. It shows the MINGW python compiled by GCC, the official one compiled by MSC. But there may have some possibility that there is an other version's python also built by GCC or MSC.
Is there any more elegant way can do this?
Mingw python3 build patched official sysconfig.get_platform() function so it returns "mingw" and can be used for distinguishing from official python builds.
https://github.com/Alexpux/MINGW-packages/blob/abd06ca92d876b9db05dd65f27d71c4ebe2673a9/mingw-w64-python2/0410-MINGW-build-extensions-with-GCC.patch#L54
Another option is to check if GCC is in sys.version.
For example:
import sys
MSYS2 = ("GCC" in sys.version)
print(MSYS2)
> True
This works because the sys.version information has the type of compiler used, and MinGW uses GCC.
python -c "import sys; print(sys.version)"
3.7.4 (default, Aug 15 2019, 18:17:27) [GCC 9.2.0 64 bit (AMD64)]
sys.platform gives msys when in msys-Python.
After successfully compiling Mesos 0.16.0, running the tests fails when checking the PythonFramework. All other tests pass successfully.
Steps I used for building:
./bootstrap
mkdir build
cd build
../configure CXX=g++4.7 CC=gcc-4.7
make
Then, when running the tests;
make check
The results look like this:
[...]
[ RUN ] ExamplesTest.PythonFramework
../../src/tests/script.cpp:78: Failure
Failed
python_framework_test.sh exited with status 1
[ FAILED ] ExamplesTest.PythonFramework (201 ms)
[...]
Environment:
OS X 10.9.1 (Mavericks)
Python 2.7.5 (default, Aug 25 2013, 00:04:04) [GCC 4.2.1 Compatible
Apple LLVM 5.0 (clang-500.0.68)] on darwin
gcc-4.7 (GCC) 4.7.3 Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There
is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
How do I build proper Mesos Python bindings that pass the tests within this environment?
Update:
My former answer has mostly become obsolete as of Mesos 0.17.0 due to the fact that this version does fully handle clang compilage (Yay!). So there is no need to compile it using gcc anymore - just go ahead and use Xcode's clang (Xcode commandline utilities).
In case you still get into trouble getting the Python bindings to work, please add a comment or new question here on StackOverflow or post into the Mesos Mailing list.
Mesos version 0.16.0 or lower:
How to fix the Python bindings of Mesos on OS X (10.9).
Install Python 2.7.3 via homebrew
Find out which versions are available
brew versions python
2.7.6 git checkout 3c86d2b /usr/local/Library/Formula/python.rb
2.7.5 git checkout a04b443 /usr/local/Library/Formula/python.rb
2.7.3 git checkout 865f763 /usr/local/Library/Formula/python.rb
2.7.4 git checkout 280581d /usr/local/Library/Formula/python.rb
[...]
Select Python 2.7.3
cd /usr/local/Library/Formula/
git checkout 865f763 /usr/local/Library/Formula/python.rb
brew install python
Make sure you do not force installing a universal build (32 + 64bit) as that would again cause the same issue explained below. The default is 64bit only and that is just fine.
Rebuild Mesos in connection with your custom Python installation
rm -rf build
rm -rf ~/.python-eggs
mkdir build
cd build
../configure CXX=g++-4.7 CC=gcc-4.7 PYTHON=/usr/local/bin/python
make
make check
You should now see a properly functioning test, hence a perfectly fine Mesos Python binding:
[ RUN ] ExamplesTest.PythonFramework
[ OK ] ExamplesTest.PythonFramework (1682 ms)
As asking users to install a custom Python version often is just wrong but in this case appears to be inevitable, let me draft an explanation of the issue. And maybe one of the readers knows a better workaround.
Manually executing that test using the verbosive output setting does help identifying the exact problem.
bin/mesos-tests.sh --gtest_filter="*.PythonFramework" --verbose
Traceback (most recent call last): File
"/Users/till/Documents/Development/github/mesos-master/build/../src/examples/python/test_framework.py",
line 23, in
import mesos File "build/bdist.macosx-10.9-intel/egg/mesos.py", line 26, in File
"build/bdist.macosx-10.9-intel/egg/_mesos.py", line 7, in
File "build/bdist.macosx-10.9-intel/egg/_mesos.py", line 6, in
bootstrap ImportError: dlopen(/Users/till/.python-eggs/mesos-0.16.0-py2.7-macosx-10.9-intel.egg-tmp/_mesos.so,
2): Symbol not found: __ZNSoD0Ev Referenced from:
/Users/till/.python-eggs/mesos-0.16.0-py2.7-macosx-10.9-intel.egg-tmp/_mesos.so
Expected in: flat namespace in
/Users/till/.python-eggs/mesos-0.16.0-py2.7-macosx-10.9-intel.egg-tmp/_mesos.so
The important detail is the fact that the dynamic linkage of that native Python egg failed.
The reasoning is to be found within the distutils build step of this module when building mesos 0.16. The Python distutils derive their build-settings directly from python-config. As your Python was built using clang, the distutils will try to build your native egg using clang as well.
Issues:
Mesos' autoconf phase did not propagate the compiler settings into the distutils build phase. So even though Mesos itself is being built using gcc-4.7 in the above description, the egg is being built using clang. The result is a mishmash of libc++ and stdlibc++ which are not ABI compatible.
that part is being fixed right now, Mesos will use the same compiler in the distutils build phase as well (see MESOS-798 and MESOS-799). Chances are pretty high that when you read this answer, that particular issue was already taken care of.
The default OS X Python distutils do enforce building universal binaries (i386 + x86_64) using parameters that only the gcc-frontend of clang supports. There appears to be no workaround, hence all dynamically linked dependencies of that egg will have to be built for both architectures as well (this appears to be a leftover from OS X 10.6).
Mesos itself is linked statically into that egg, hence it does not have to be built as a universal binary for the egg to build and function on a 64bit platform. It will however fail to execute on a 32bit platform.
As long as Mesos does not support clang compilage (hence being linked against libc++), the only proper workaround seems to be to install a differently compiled Python. A quick and easy solution is using homebrew to install Python 2.7.3. Note: do not install Python 2.7.6 (the current default of homebrew) as that one has issues in connection with its autoconf developer macro (see MESOS-617)
I'm working on a Django project that is using an ldap authentication module. This is working on our server but I am running into issues getting this running on my windows dev machine.
My environment is using virtualevn and when trying to install pip python-ldap I receive the following message:
error: Unable to find vcvarsall.bat
Does anyone have any idea what could be going wrong?
To expand on #Brandon's answer, to install using the pre-built wheel:
Ensure you have pip 19.2+ installed:
$ pip --version
pip 19.2.3
Check your Python version and architecture (32/64 bit) https://stackoverflow.com/a/10966396/1026:
$ python -c 'import sys; print(sys.version)'
3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]
Download the matching pre-built *.whl from https://www.lfd.uci.edu/~gohlke/pythonlibs/#python-ldap
For example given the above Python I picked "python_ldap‑3.2.0‑cp37‑cp37m‑win_amd64.whl"
Install it with:
pip install path\to\your.whl
Unfortunately, many Python modules have trouble installing on Windows. The error you're receiving is one that I was never able to get fixed, even given the vast amount of information available on the web. Give this link a try for a pre-compiled version: http://www.lfd.uci.edu/~gohlke/pythonlibs/#python-ldap