OpenShift Python mongoDB environment variables not set / can't connect - python

This is in my application file head:
import os
import sys
from cgi import parse_qs, escape
import pymongo
from pymongo import MongoClient
I have the mongoDB 2.4 gear installed, and am trying to connect via
client = MongoClient('mongodb:$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/')
I get the errors:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 239, in __init__
res = uri_parser.parse_uri(entity, port)
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/uri_parser.py", line 269, in parse_uri
nodes = split_hosts(hosts, default_port=default_port)
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/uri_parser.py", line 209, in split_hosts
nodes.append(parse_host(entity, port))
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/uri_parser.py", line 137, in parse_host
raise ConfigurationError("Port number must be an integer.")
pymongo.errors.ConfigurationError: Port number must be an integer.
looks like OPENSHIFT_MONGODB_DB_PORT isn't set
print OPENSHIFT_MONGODB_DB_PORT --> NameError: name 'OPENSHIFT_MONGODB_DB_PORT' is not defined
Same with OPENSHIFT_MONGODB_DB_HOST
What would I need to do, to set up a connection?
Update:
I was able to connect directly via client by hardcoding info from rockmongo
client = MongoClient('mongodb://admin:password#[ip addr]:[port]/')
but when I do
client = MongoClient('mongodb:admin:password#%s:%s/' % os.environ['OPENSHIFT_MONGODB_DB_HOST'], os.environ['OPENSHIFT_MONGODB_DB_PORT']))
I get
[error] (<type 'exceptions.KeyError'>, KeyError('OPENSHIFT_MONGODB_DB_HOST',), <traceback object at 0x7f7bc8367248>)

The OpenShift connection variables are defined as environment variables, they cannot be accessed as normal Python variables. So the print statement you gave does not work, the following should;
import os
print os.environ['OPENSHIFT_MONGODB_DB_PORT']
You should change your code to;
client = MongoClient('mongodb:%s:%s/' % (os.environ['OPENSHIFT_MONGODB_DB_HOST'], os.environ['OPENSHIFT_MONGODB_DB_PORT))
You can refer to an example here.

Related

Unable to use "socket" in my vscode using python

I was just learning socket programming in python and was unable to run my socket commands in vs code.
the code i wrote is:-
import socket
import threading
PORT=5050
SERVER =socket.gethostbyname(socket.gethostname)
print(SERVER)
The code produces this error:
Traceback (most recent call last):
File "c:\Users\Gurkirat Singh\Desktop\tempCodeRunnerFile.py", line 1, in <module>
import socket
File "c:\Users\Gurkirat Singh\Desktop\socket.py", line 3, in <module>
from xmlrpc.client import Server
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\xmlrpc\client.py", line 136, in <module>
import http.client
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\http\client.py",
line 789, in <module>
class HTTPConnection:
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\http\client.py",
line 837, in HTTPConnection
def __init__(self, host, port=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
AttributeError: partially initialized module 'socket' has no attribute '_GLOBAL_DEFAULT_TIMEOUT' (most likely due to a circular import)
Looking at your stack trace, you seem to have a file named socket.py:
File "c:\Users\Gurkirat Singh\Desktop\socket.py", line 3, in <module>
from xmlrpc.client import Server
Having a file named socket.py in your workspace causes import problems because Python won't import its library module socket but instead will import your own file.
Rename your file from socket.py to something else like mysock.py and delete anything from your Desktop directory called socket.pyc if it exists.

Getting "TypeError: 'NoneType' object is not iterable" while doing parallel ssh

I am trying to do parallel ssh on servers. While doing this, i am getting "TypeError: 'NoneType' object is not iterable" this error. Kindly help.
My script is below
from pssh import ParallelSSHClient
from pssh.exceptions import AuthenticationException, UnknownHostException, ConnectionErrorException
def parallelsshjob():
client = ParallelSSHClient(['10.84.226.72','10.84.226.74'], user = 'root', password = 'XXX')
try:
output = client.run_command('racadm getsvctag', sudo=True)
print output
except (AuthenticationException, UnknownHostException, ConnectionErrorException):
pass
#print output
if __name__ == '__main__':
parallelsshjob()
And the Traceback is below
Traceback (most recent call last):
File "parallelssh.py", line 17, in <module>
parallelsshjob()
File "parallelssh.py", line 10, in parallelsshjob
output = client.run_command('racadm getsvctag', sudo=True)
File "/Library/Python/2.7/site-packages/pssh/pssh_client.py", line 520, in run_command
raise ex
TypeError: 'NoneType' object is not iterable
Help me with the solution and also suggest me to use ssh-agent in this same script. Thanks in advance.
From reading the code and debugging a bit on my laptop, I believe the issue is that you don't have a file called ~/.ssh/config. It seems that parallel-ssh has a dependency on OpenSSH configuration, and this is the error you get when that file is missing.
read_openssh_config returns None here: https://github.com/pkittenis/parallel-ssh/blob/master/pssh/utils.py#L79
In turn, SSHClient.__init__ blows up when trying to unpack the values it expects to receive: https://github.com/pkittenis/parallel-ssh/blob/master/pssh/ssh_client.py#L97.
The fix is presumably to get some sort of OpenSSH config file in place, but I'm sorry to say I know nothing about that.
EDIT
After cleaning up some of parallel-ssh's exception handling, here's a better stack trace for the error:
Traceback (most recent call last):
File "test.py", line 11, in <module>
parallelsshjob()
File "test.py", line 7, in parallelsshjob
output = client.run_command('racadm getsvctag', sudo=True)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/pssh_client.py", line 517, in run_command
self.get_output(cmd, output)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/pssh_client.py", line 601, in get_output
(channel, host, stdout, stderr, stdin) = cmd.get()
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/gevent/greenlet.py", line 480, in get
self._raise_exception()
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/gevent/greenlet.py", line 171, in _raise_exception
reraise(*self.exc_info)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/gevent/greenlet.py", line 534, in run
result = self._run(*self.args, **self.kwargs)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/pssh_client.py", line 559, in _exec_command
channel_timeout=self.channel_timeout)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/ssh_client.py", line 98, in __init__
host, config_file=_openssh_config_file)
TypeError: 'NoneType' object is not iterable
This was seemingly a regression in the 0.92.0 version of the library which is now resolved in 0.92.1. Previous versions also work. OpenSSH config should not be a dependency.
To answer your SSH agent question, if there is one running and enabled in the user session it gets used automatically. If you would prefer to provide a private key programmatically can do the following
from pssh import ParallelSSHClient
from pssh.utils import load_private_key
pkey = load_private_key('my_private_key')
client = ParallelSSHClient(hosts, pkey=pkey)
Can also provide an agent with multiple keys programmatically, per below
from pssh import ParallelSSHClient
from pssh.utils import load_private_key
from pssh.agent import SSHAgent
pkey = load_private_key('my_private_key')
agent = SSHAgent()
agent.add_key(pkey)
client = ParallelSSHClient(hosts, agent=agent)
See documentation for more examples.

Python Hive: thrift.transport.TTransport.TTransportException: None

Suppose Hive is installed in say, "g" cluster. I do not have access to the Gold Cluster. I'm doing my python development work in "s" cluster. I can access Hive from the "s" cluster and run queries.
I have the below code to connect to Hive from a Python script running in "s" cluster.
some_table is a table that already exists in Hive. I would like to execute a simple select * from some_table command to get some results.
import sys
sys.path.append("/usr/lib/hive/lib/py")
from hive_service import ThriftHive
from hive_service.ttypes import HiveServerException
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
transport = TSocket.TSocket('what-ever-server', what-ever-port)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = ThriftHive.Client(protocol)
transport.open()
print "connect success"
client.execute("SELECT * FROM some_table")
print client.fetchAll()
print "executed"
But I get the below error after "connect success" is printed. I am assuming that the connection was successful.
Traceback (most recent call last):
File "hiveConnect.py", line 30, in <module>
row = client.execute("SELECT * FROM some_table")
File "/usr/lib/hive/lib/py/hive_service/ThriftHive.py", line 68, in execute
self.recv_execute()
File "/usr/lib/hive/lib/py/hive_service/ThriftHive.py", line 79, in recv_execute
(fname, mtype, rseqid) = self._iprot.readMessageBegin()
File "/usr/lib/hive/lib/py/thrift/protocol/TBinaryProtocol.py", line 137, in readMessageBegin
name = self.trans.readAll(sz)
File "/usr/lib/hive/lib/py/thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz-have)
File "/usr/lib/hive/lib/py/thrift/transport/TTransport.py", line 155, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.DEFAULT_BUFFER)))
File "/usr/lib/hive/lib/py/thrift/transport/TSocket.py", line 94, in read
raise TTransportException('TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: None
What am I doing wrong in this code? I am not experiencing any error while connecting to Hive using the server-name and port, so I'm assuming everything is fine there and that the connection to Hive is not the issue.
Well after a bit more research I found that it was actually HiveServer2 that was being used with a port number of 10,000. After that I had to install pyhs2 for it to work properly.

SUDS Exception Imported Schema Failed

I'm getting the error:
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at
(http://www.w3.org/2001/XMLSchema.x sd), failed
when passing a Doctor (constructed with ImportDoctor) to the suds Client constructor.
I'm working on two Windows machines, both of them got the same version of suds installed, but only one of them rises the error above.
Could someone guide me here to know why this error rises?, so I can figure out what's missing on the machine where it happens?.
Thanks in advance!!!.
UPDATE: I don't really know if this is important, but it's worth noting that my Windows machine that is rising the error is an Amazon Web Services instance. At my local machine everything's working well!.
UPDATE: Here's some code I ran at the python interpreter of the machine I mentioned. Here you can detail how the error is rising...
>>> from suds.client import Client
>>> from suds.xsd.doctor import ImportDoctor, Import
>>> missing_import = Import("http://www.w3.org/2001/XMLSchema")
>>> missing_import.filter.add("http://tempuri.org/")
>>> doctor = ImportDoctor(missing_import)
>>> client = Client("http://etcfulfill.ebooks.com/Fulfillment.asmx?wsdl")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "suds\client.py", line 112, in __init__
self.wsdl = reader.open(url)
File "suds\reader.py", line 152, in open
d = self.fn(url, self.options)
File "suds\wsdl.py", line 159, in __init__
self.build_schema()
File "suds\wsdl.py", line 220, in build_schema
self.schema = container.load(self.options)
File "suds\xsd\schema.py", line 95, in load
child.dereference()
File "suds\xsd\schema.py", line 323, in dereference
midx, deps = x.dependencies()
File "suds\xsd\sxbasic.py", line 422, in dependencies
raise TypeNotFound(self.ref)
suds.TypeNotFound: Type not found: '(schema, http://www.w3.org/2001/XMLSchema, )'
>>> client = Client("http://etcfulfill.ebooks.com/Fulfillment.asmx?wsdl", doctor=doctor)
No handlers could be found for logger "suds.xsd.sxbasic"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "suds\client.py", line 112, in __init__
self.wsdl = reader.open(url)
File "suds\reader.py", line 152, in open
d = self.fn(url, self.options)
File "suds\wsdl.py", line 159, in __init__
self.build_schema()
File "suds\wsdl.py", line 220, in build_schema
self.schema = container.load(self.options)
File "suds\xsd\schema.py", line 93, in load
child.open_imports(options)
File "suds\xsd\schema.py", line 305, in open_imports
imported = imp.open(options)
File "suds\xsd\sxbasic.py", line 542, in open
result = self.download(options)
File "suds\xsd\sxbasic.py", line 567, in download
raise Exception(msg)
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at (http://www.w3.org/2001/XMLSchema.xsd), failed
UPDATE:
I realized that suds connections always open in TCP increasing ports, and if it reaches the maximum TCP port (65535) then it starts opening again from the minimum TCP port available, so there's no problem with this.
The problem shows up when using suds ImportDoctor, because it has to open a previous connection to the location where the import should be retrieved, and for some reason, if the system reaches the maximum TCP port count, then suds somehow assumes that there's no TCP port available to open the connection for obtaining the import, and in consecuence it throws the exception:
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at (http://www.w3.org/2001/XMLSchema.xsd), failed
I repeat, this only happens if suds has to open this previous connection for obtaining the import. If ImportDoctor is not used, then suds has no problem if the TCP port count reaches its maximum, it just restarts at the minimum port available.
Does anyone has any clue on how to resolve this issue???. I'd really appreciate the help!!!.
I've figured out what the problem was. The schema that was missing from the WSDL I was trying to use with suds was:
http://www.w3.org/2001/XMLSchema
And the XSD file for this schema is at:
http://www.w3.org/2001/XMLSchema.xsd
So when I used suds ImportDoctor to add this schema import, sometimes the w3.org domain was denying my access (don't know why really) and that's why this error was rising:
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at (http://www.w3.org/2001/XMLSchema.xsd), failed
What did I do to solve this problem?. I just downloaded this schema to my machine and used suds ImportDoctor to retrieve this import locally.
And that was it!!!. Confusing bug!!!. But SOLVED.

SSLError on Google App Engine (local dev-server)

When I try to use boto library on App Engine, I get the next error:
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\google\appengine\ext\webapp\_webapp25.py", line 701, in __call__
handler.get(*groups)
File "E:\Probes\pruebas\pruebasAWS\main.py", line 26, in get
conn = S3Connection('<KEY1>','<KEY2>')
File "E:\Probes\pruebas\pruebasAWS\boto\s3\connection.py", line 148, in __init__
path=path, provider=provider)
File "E:\Probes\pruebas\pruebasAWS\boto\connection.py", line 231, in __init__
self.http_unretryable_exceptions.append(ssl.SSLError)
AttributeError: 'module' object has no attribute 'SSLError'
I´ve installed OpenSSL and Python 2.7. OpenSSL and SSL library for python are running, and when i deploy the App to the Google Infrastructure, it works fine. The problem comes when I try to execute the app on my local machine.
The code is:
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from boto.s3.connection import S3Connection
import hashlib
class MainHandler(webapp.RequestHandler):
def get(self):
conn = S3Connection('<KEY1>','<KEY2>')
bucket = conn.create_bucket(hashlib.md5('noTRePeaTedBuCket').hexdigest()+"probe")
if bucket:
self.response.out.write('Bucket creado')
else:
self.response.out.write('Bucket NO creado')
The actual issue here is that AppEngine mucks around with things to make it impossible to import certain standard, builtin python modules such as ssl.
There was some conversation about this on the boto IRC and one of the users came up with this patch:
https://github.com/samba/boto/commit/6f1ab73d92ff6fb2589362bbdadf6bbe66811e7e
Some form of this will probably be merged into boto master soon.

Categories