python - code works fine on Linux but crashes on OSX - python

I'm running on OSX Mavericks and have python 3.3.3 installed.
Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 16 2013, 23:39:35)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin)
I'm trying to run this (unfinished) code (a simple HTTPS GET and POST with cookies), but it crashes with an unknown error:
import urllib.request as rqst
import re
import http.cookiejar
def LoginICorsi():
loginUrl = "https://www2.icorsi.ch/auth/shibboleth/"
rqst.install_opener(rqst.build_opener(rqst.HTTPCookieProcessor(http.cookiejar.CookieJar())))
pageBytes = rqst.urlopen(loginUrl)
pageString = pageBytes.read().decode("utf-8")
action = re.findall(r'action="([^"]*)', pageString)[0]
loginPostUrl = "https://wayf.switch.ch" + action
loginPostUrl = loginPostUrl.replace("&", "&")
print("Posting USI to "+loginPostUrl)
postDATA = "user_idp=https://login2.usi.ch/idp/shibboleth".encode("utf-8")
usiLoginRequest = rqst.Request(loginPostUrl)
usiLoginRequest.add_header("Content-Type", "application/x-www-form-urlencoded")
usiLoginUrl = rqst.urlopen(usiLoginRequest, data=postDATA)
usiLoginResult = usiLoginUrl.read().decode("utf-8")
print(usiLoginResult)
The problem is that this code works on Ubuntu
Python 3.3.3 (default, Nov 20 2013, 00:22:18)
[GCC 4.8.2] on linux
so I suppose the code is correct.
Moreover, the code worked on my Mac when I set the system proxy for HTTPS to a proxy with Fiddler.
This is the traceback pastebin.
What am I missing? Does this have something to do with OSX, python, the HTTPS server or just my mac?
The problem arises when handshaking with the server, but I don't why it works on Linux.

Highly, highly recommend you migrate away from urllibs and onto python requests module.
Seriously, it's worth the 5 minutes to easy_install/pip and port your code. It will handle most SSL issues a lot more graceful and a lot less painful.

Related

directory with slash or backslash in pycharm vs google colab

I have a basic question about using slash or backslash.
here is my code:
import os
path1 = r'c:\super'
path2 = r'c:/super'
print(os.path.basename(path1))
print(os.path.basename(path2))
the result from pycharm is:
super
super
the result from google colab is:
c:\super
super
why do they perform so differently on both platforms?
I believe is an issue of 'GCC implementation' of the python compiler.
I've tried this:
import os
import sys
path=[]
path.append('c:\super')
path.append('c:\a\b\super')
path.append(r'c:\super')
path.append(r'c:\a\b\super')
path.append('c:\\a\\b\\super')
path.append('c:/a/super')
path.append(r'c:/a/super')
for p in path:
print (os.path.basename(p))
print(sys.version)
Colaboratory reports
c:\super
c:\super
c:\super
c:\a\b\super
c:\a\b\super
super
super
3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
The same code on repl-it (for instance) get:
answer from repl.it
But the same code in 3.8.5 / Intel gets
answer from local Intel

PyLint not finding class from Google Appengine SDK

We have a Python Application running on Google App Engine. For local development, we have the Google Appengine SDK installed. The setup is a little messed up as the application itself adds things to the sys.path. PyLint of course does not pick up those changes happening somewhere in some function.
I tried cleaning up the PyLint log by adding the Google Appengine SDK to the PYTHONPATH, which worked pretty good. Locally, I can now run PyLint for a specific set of folders and it reports a 10/10 with no errors (but a lot of locally-disabled statements). The same code is reporting exactly 1 issue on our CI system (We use Codeship). The message is Module 'google.appengine.ext.ndb' has no 'JsonProperty' member (no-member).
The Googe Appengine SDK is installed inside the Codeship container and is added to the PYTHONPATH. Other functions/classes from the SDK get picked up correctly. The google.appengine.ext.ndb.init.py looks like this:
"""NDB -- A new datastore API for the Google App Engine Python runtime."""
__version__ = '1.0.10'
__all__ = []
from tasklets import *
__all__ += tasklets.__all__
from model import * # This implies key.*
__all__ += model.__all__
from query import *
__all__ += query.__all__
from context import *
__all__ += context.__all__
The google.appengine.ext.ndb.model.py contains the JsonProperty class and has some magic as well:
# Update __all__ to contain all Property and Exception subclasses.
for _name, _object in globals().items():
if ((_name.endswith('Property') and issubclass(_object, Property)) or
(_name.endswith('Error') and issubclass(_object, Exception))):
__all__.append(_name)
As I said, it works locally but does not on Codeship. Locally, `pylint --version prints
pylint 1.5.0,
astroid 1.4.5
Python 2.7.9 (default, Apr 2 2015, 15:33:21)
[GCC 4.9.2]
while on Codeship, it says
pylint 1.5.0,
astroid 1.4.4
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2]
I am not sure what is going on here. There should be no major differences between Python 2.7.6 and 2.7.9, right? I have no idea how to further debug this either. Any hints are appreciated.

python import cx_Oracle error on cygwin

I tried to install cx_Oracle from pypi source since there is no available port for it in cygwin. I did make some changes as suggested in http://permalink.gmane.org/gmane.comp.python.db.cx-oracle/2492 and modified my setup.py. However, I still get the following error :-
$ python
Python 2.7.3 (default, Dec 18 2012, 13:50:09)
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
/usr/lib/python2.7/site-packages/cx_Oracle-5.1.3-py2.7-cygwin-1.7.24-i686.egg/cx_Oracle.py:3: UserWa
rning: Module cx_Oracle was already imported from /usr/lib/python2.7/site-packages/cx_Oracle-5.1.3-p
y2.7-cygwin-1.7.24-i686.egg/cx_Oracle.pyc, but /home/zerog/cx_Oracle-5.1.3 is being added to sys.pat
h
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.cygwin-1.7.24-i686/egg/cx_Oracle.py", line 7, in <module>
File "build/bdist.cygwin-1.7.24-i686/egg/cx_Oracle.py", line 6, in __bootstrap__
ImportError: Exec format error
>>>
If someone can please help me fix this ?
TIA.
Fixed this by specifying the path to instantclient as below :
$ export PATH=$PATH:/cygdrive/d/Tools/instantclient_11_2
(Other, possibly important stuff) :
$ echo $LD_LIBRARY_PATH
/cygdrive/d/Tools/instantclient_11_2
$ echo $ORACLE_HOME
/cygdrive/d/Tools/instantclient_11_2
Now, I get :-
$ python
Python 2.7.3 (default, Dec 18 2012, 13:50:09)
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
>>>
It's hard to pin down from the error message alone, but I am guessing that you have two different copies of cx_Oracle in your sys.path. The error message is complaining that a different version of the same module had already been import-ed.
Presumably the pristine upstream version is installed system-wide in /usr/lib/python2.7/site-packages/cx_Oracle-5.1.3-py2.7-cygwin-1.7.24-i686.egg, and your modified version in /home/zerog/cx_Oracle-5.1.3.
Does it work if you pare down sys.path so that only the original, or only your modified version, is included?
(You might want to use virtualenv if you need to switch back and forth between two versions frequently.)
I ran into this error "Exec format error."
For me, this was likely caused by a mismatch between cygwin being installed as 64 bit, but the instant client being installed as 32 bit. Double check that everything (oracle, cygwin) is either 32 bit or 64 bit.
What fixed my issue:
Since my cygwin is 64 bit (see uname -a, and look for x86_64), I downloaded the 64 bit instant client from oracle's website, and unzipped
I set the env vars in .profile, to point where it was unzipped:
export ORACLE_HOME=/cygdrive/c/oracle/instantclient_x64_11_2
export LD_LIBRARY_PATH=$ORACLE_HOME
export DYLD_LIBRARY_PATH=$ORACLE_HOME
export TNS_ADMIN='//optional/path/to/your/oracle/tns/files/'
source ~/.profile
To test, you should now be able to run this python command with no error:
import cx_Oracle
To verify the path is correct, if you run ls, you should see something like
ls $ORACLE_HOME
adrci.exe genezi.exe oci.sym ociw32.dll ojdbc6.jar
oraocci11.dll oraociei11.sym uidrvci.exe vc9
adrci.sym genezi.sym ocijdbc11.dll ociw32.sym orannzsbb11.dll
oraocci11.sym orasql11.dll uidrvci.sym xstreams.jar
BASIC_README oci.dll ocijdbc11.sym ojdbc5.jar orannzsbb11.sym
oraociei11.dll orasql11.sym vc8

Running SL4A facade APIs from host python shell

Objective:
Trying to run SL4A facade APIs from python shell on the host system (windows 7 PC)
My environment:
1. On my windows 7 PC, i have python 2.6.2
2. Android sdk tools rev 21, platform tools rev 16
3. API level 17 supported for JB 4.2
4. I have 2 devices ( one running android 2.3.3 and another android 4.2.2) both running Python for android and SL4A
I'm trying these commands as specified at http://code.google.com/p/android-scripting/wiki/RemoteControl
Here are the commands which i'm trying on the python shell:
Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import android
>>> droid=android.Android
>>> droid.makeToast("Hello")
Traceback (most recent call last):
File "", line 1, in
AttributeError: type object 'Android' has no attribute 'makeToast'
Before this i'm performing the port forwarding and starting a private server as shown below
$ adb forward tcp:9999 tcp:4321
$ set AP_PORT=9999
Also set the server on the target listening on port 9999 ( through SL4A->preferences->serverport.
Please help to understand where i'm doing mistake which gives the above error while trying droid.makeToast("Hello") ?
Try
import android
droid=android.Android()
droid.makeToast("Hello")
(you are missing parentheses after android.Android)
Hope you solved the problem. This is what I found, so maybe for somebody else it will be useful.
You need to run all your commands inside the DOS shell...otherwise you will get that error[11001].
If you still want to run you commands outside the DOS shell - add a new environment variable as described here:
http://www.smartphonedaq.com/installation.page

Python ssl login hanging on Debian

I am trying to use Python 2.7 mechanize to log into Mint.com with the following code:
import mechanize
br = mechanize.Browser()
br.open("https://wwws.mint.com/login.event")
this works just fine on OSX, but it hangs on debian. The issue seems to be ssl-related; the traceback ends with
File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
EDIT:
the issue persists on Debian using urllib2. As suggested in the comments, it seems the issue is actually ssl related. Why would this be a problem on Debian and not OSX?
So things look ok on Fedora:
[bharrington#leviathan ~]$ python
Python 2.7.5 (default, Aug 22 2013, 09:31:58)
[GCC 4.8.1 20130603 (Red Hat 4.8.1-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mechanize
>>> br = mechanize.Browser()
>>> br.open("https://wwws.mint.com/login.event")
<response_seek_wrapper at 0x29b6440 whose wrapped object = <closeable_response at 0x29b6320 whose fp = <socket._fileobject object at 0x298d150>>>
>>> br.title()
'Mint > Start Here'
>>>
Which leads me to wonder about the SSL/OpenSSL libraries in use. Just to test it out can you run from the command line:
$ openssl s_client -connect wwws.mint.com:443
You should see the SSL certificate for mint displayed, along with the full certificate chain verification, and the final line: " Verify return code: 0 (ok)"
While I highly doubt it is an SSL issue directly I figure it's worth checking into. Additionally, verify the version of mechanize. Debian is notable for using stable versions of code (not new versions). The version I verified with of mechanize was 0.2.5
This is a manifestation of an incompatibility between recent versions of OpenSSL and certain Web servers. Apple's doing their best to eliminate OpenSSL on OS X, so they're only applying security patches (OpenSSL has been difficult to support as part of the OS, not to mention "minor" updates introducing issues like this one), whereas Debian is using a more recent OpenSSL 1.0.1.
#Brian Redbeard’s suggestion to check with the openssl command-line is a good one — it hung for me on wwws.mint.com when I just tried it.
This question on ServerFault finally provided the answer. The SSLLabs test linked there identifies long handshake intolerance as the issue, which affects OpenSSL 1.0.1 and later, and links to an OpenSSL bug with some potential workarounds.
Either using -no_tls1_2, as one of the OpenSSL developers recommends, or reducing the cipher list with the -cipher argument, causes OpenSSL 1.0.1 to successfully handshake with wwws.mint.com (as well as another server I was trying to contact).
For my purposes — a script that isn't going to be distributed — I monkeypatched ssl.wrap_socket as follows:
import ssl
old_wrap_socket = ssl.wrap_socket
def wrap_socket(sock, keyfile=None, certfile=None,
server_side=False, cert_reqs=ssl.CERT_NONE,
ssl_version=ssl.PROTOCOL_SSLv3, ca_certs=None,
do_handshake_on_connect=True,
suppress_ragged_eofs=True, ciphers=None):
return old_wrap_socket(sock, keyfile, certfile,
server_side, cert_reqs, ssl_version,
ca_certs, do_handshake_on_connect,
suppress_ragged_eofs, ciphers)
ssl.wrap_socket = wrap_socket
import mechanize
The default value for ssl_version is ssl.PROTOCOL_SSLv23; by changing it to PROTOCOL_SSLv3 it successfully connects.
You could guard this patch using a test such as ssl.OPENSSL_VERSION_INFO[:3] >= (1, 0, 1).
This should likely be reported as a Debian OpenSSL bug if it hasn't been already.

Categories