How to fix invalid URL error while using http.client? - python

I am trying to hit an api using http.client of pyhton3 to simulate how I would do the same in a web browser.
However, http.client feels that url is inappropriate.
This is what I am trying to do.
import http.client
connection = http.client.HTTPSConnection("https://analyticsapi.zoho.com/api/EmailAddress/WorkspaceName/TableName?ZOHO_ACTION=IMPORT&ZOHO_OUTPUT_FORMAT=XML&ZOHO_ERROR_FORMAT=XML&ZOHO_API_VERSION=1.0&authtoken=************&ZOHO_IMPORT_TYPE=APPEND&ZOHO_AUTO_IDENTIFY=TRUE&ZOHO_ON_IMPORT_ERROR=ABORT&ZOHO_CREATE_TABLE=TRUE&ZOHO_FILE=/home/dev1/Desktop/Zoho/temporary.csv")
connection.request("GET", "/")
response = connection.getresponse()
print("Status: {} and reason: {}".format(response.status, response.reason))
connection.close()
And this is the error I am getting.
$ python3 pyToTestPushingCSV.py
Traceback (most recent call last):
File "/usr/lib/python3.5/http/client.py", line 798, in _get_hostport
port = int(host[i+1:])
ValueError: invalid literal for int() with base 10: '//analyticsapi.zoho.com/api/usename/ATable/InsideTable?ZOHO_ACTION=IMPORT&ZOHO_OUTPUT_FORMAT=XML&ZOHO_ERROR_FORMAT=XML&ZOHO_API_VERSION=1.0&authtoken=****
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "pyToTestPushingCSV.py", line 3, in <module>
connection = http.client.HTTPSConnection("https://analyticsapi.zoho.com/api/usename/ATable/InsideTable?ZOHO_ACTION=IMPORT&ZOHO_OUTPUT_FORMAT=XML&ZOHO_ERROR_FORMAT=XML&ZOHO_API_VERSION=1.0&authtoken=****************&ZOHO_IMPORT_TYPE=APPEND&ZOHO_AUTO_IDENTIFY=TRUE&ZOHO_ON_IMPORT_ERROR=ABORT&ZOHO_CREATE_TABLE=TRUE&ZOHO_FILE=/home/dev1/Desktop/Zoho/temporary.csv")
File "/usr/lib/python3.5/http/client.py", line 1233, in __init__
source_address)
File "/usr/lib/python3.5/http/client.py", line 762, in __init__
(self.host, self.port) = self._get_hostport(host, port)
File "/usr/lib/python3.5/http/client.py", line 803, in _get_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
http.client.InvalidURL: nonnumeric port: '//analyticsapi.zoho.com/api/usename/ATable/InsideTable?ZOHO_ACTION=IMPORT&ZOHO_OUTPUT_FORMAT=XML&ZOHO_ERROR_FORMAT=XML&ZOHO_API_VERSION=1.0&authtoken=*************&ZOHO_IMPORT_TYPE=APPEND&ZOHO_AUTO_IDENTIFY=TRUE&ZOHO_ON_IMPORT_ERROR=ABORT&ZOHO_CREATE_TABLE=TRUE&ZOHO_FILE=/home/dev1/Desktop/Zoho/temporary.csv'
When I hit the URL with my Browser it gives a good response back in XML which in short, succeeds.
This is where I referred for documentation
Can you pin point where I went wrong?

As per documentation HTTPS support is only available if Python was compiled with SSL support (through the ssl module).
Also the default port for https connection is 443. The module seems to be hitting that port by default.

Related

Getting Errors while using telnetsrv in Python3

I'm creating a telnet server using telnetsrv and green. I'm using python3. Modified the green.py from from telnetsrvlib import TelnetHandlerBase, command to from .telnetsrvlib import TelnetHandlerBase, command and SocketServer to socketserver for python3 compatibility. I'm facing the following errors while connecting to the servers.
[+] Welcome to Telnet server
Traceback (most recent call last):
File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
File "/usr/local/lib/python3.8/dist-packages/gevent/baseserver.py", line 34, in _handle_and_close_when_done
return handle(*args_tuple)
File "/usr/local/lib/python3.8/dist-packages/telnetsrv/telnetsrvlib.py", line 495, in streamserver_handle
cls(request, address, server)
File "/usr/local/lib/python3.8/dist-packages/telnetsrv/green.py", line 14, in __init__
TelnetHandlerBase.__init__(self, request, client_address, server)
File "/usr/local/lib/python3.8/dist-packages/telnetsrv/telnetsrvlib.py", line 482, in __init__
SocketServer.BaseRequestHandler.__init__(self, request, client_address, server)
File "/usr/lib/python3.8/socketserver.py", line 748, in __init__
self.setup()
File "/usr/local/lib/python3.8/dist-packages/telnetsrv/green.py", line 18, in setup
TelnetHandlerBase.setup(self)
File "/usr/local/lib/python3.8/dist-packages/telnetsrv/telnetsrvlib.py", line 519, in setup
self.sock = self.request._sock
AttributeError: 'Telnet_handler' object has no attribute 'request'
2022-08-13T15:21:34Z <Greenlet at 0x7f8902e428c0: _handle_and_close_when_done(<bound method TelnetHandlerBase.streamserver_handl, <bound method StreamServer.do_close of <StreamServ, (<gevent._socket3.socket [closed] at 0x7f8902a181c)> failed with AttributeError```
*This is my code*
```#!/usr/bin/python3
import gevent, gevent.server
from telnetsrv.green import TelnetHandler, command
class Telnet_handler(TelnetHandler):
WELCOME = "Welcome to my server."
print(WELCOME)
server = gevent.server.StreamServer(("", 8023),Telnet_handler.streamserver_handle)
try:
print("[+] Welcome to Telnet server")
server.serve_forever()
except KeyboardInterrupt:
print("[-] Connection Failed \n Exiting Telnet Server ")```
[1]: https://i.stack.imgur.com/GuyNX.png

Onion Server with Stem and CherryPy in Python

I want to create an Onion-Website with CherryPy in Python (using Stem to access the Tor network). My problem is the deployment of the site. For that I adopted the example from this article: https://jordan-wright.com/blog/2014/10/06/creating-tor-hidden-services-with-python/ (great article, but originally in combination with Flask, but this should also work with CherryPy and vice versa). Only difference is that Flask operates on port 5000 and CherryPy on 8080 (I changed that in the code below).
My final test-code looks like this:
import cherrypy
from stem.control import Controller
class HelloWorld(object):
#cherrypy.expose
def index(self):
return "Hello world!"
if __name__ == '__main__':
port = 8080
host = "127.0.0.1"
hidden_svc_dir = '/home/python/'
print(" * Getting controller")
controller = Controller.from_port(address="127.0.0.1", port=9151)
try:
controller.authenticate(password="")
controller.set_options([
("HiddenServiceDir", hidden_svc_dir),
("HiddenServicePort", "80 %s:%s" % (host, str(port)))
])
svc_name = open(hidden_svc_dir + "/hostname", "r").read().strip()
print(" * Created host: %s" % svc_name)
except Exception as e:
print(e)
cherrypy.quickstart(HelloWorld())
My /etc/tor/torrc contains this:
ControlPort 9051
HashedControlPassword xxx
CookieAuthentication 1
(I also tried with/without Password and with/without Cookies set to 1 and 0 - all with the same error message.)
Though in other codes I use Stem / Tor and they work properly, this one throws the folling error message:
* Getting controller
Traceback (most recent call last):
File "/home/xxx/.local/lib/python3.6/site-packages/stem/socket.py", line 535, in _make_socket
control_socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "onion_server_1.py", line 20, in <module>
controller = Controller.from_port(address="127.0.0.1", port=9151)
File "/home/xxx/.local/lib/python3.6/site-packages/stem/control.py", line 1033, in from_port
control_port = stem.socket.ControlPort(address, port)
File "/home/xxx/.local/lib/python3.6/site-packages/stem/socket.py", line 503, in __init__
self.connect()
File "/home/xxx/.local/lib/python3.6/site-packages/stem/socket.py", line 172, in connect
self._socket = self._make_socket()
File "/home/xxx/.local/lib/python3.6/site-packages/stem/socket.py", line 538, in _make_socket
raise stem.SocketError(exc)
stem.SocketError: [Errno 111] Connection refused
I have no more ideas why this does not work.
A working code (adopted from mine above) would be great.

AsyncHTTPClient with https auth proxy

I am trying to configure AsyncHTTPClient with auth proxy to access https websites. Is it possible to do with authenticated proxy?
from tornado import httpclient, ioloop
config = {
'proxy_host': proxy_host,
'proxy_port': proxy_post,
"proxy_username": proxy_username,
"proxy_password": proxy_password
}
httpclient.AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
def handle_request(response):
if response.error:
print("Error:", response.error)
else:
print(response.body)
ioloop.IOLoop.instance().stop()
http_client = httpclient.AsyncHTTPClient()
http_client.fetch("https://twitter.com/",
handle_request, **config)
ioloop.IOLoop.instance().start()
I get these errors after running the code above
Traceback (most recent call last):
File "C:\Users\Adam\Anaconda3\envs\sizeer\lib\site-packages\tornado\curl_httpclient.py", line 130, in _handle_socket
self.io_loop.add_handler(fd, self._handle_events, ioloop_event)
File "C:\Users\Adam\Anaconda3\envs\sizeer\lib\site-packages\tornado\platform\asyncio.py", line 103, in add_handler
self.asyncio_loop.add_writer(fd, self._handle_events, fd, IOLoop.WRITE)
File "C:\Users\Adam\Anaconda3\envs\sizeer\lib\asyncio\events.py", line 507, in add_writer
raise NotImplementedError
NotImplementedError
Traceback (most recent call last):
File "C:\Users\Adam\Anaconda3\envs\sizeer\lib\site-packages\tornado\curl_httpclient.py", line 130, in _handle_socket
self.io_loop.add_handler(fd, self._handle_events, ioloop_event)
File "C:\Users\Adam\Anaconda3\envs\sizeer\lib\site-packages\tornado\platform\asyncio.py", line 97, in add_handler
raise ValueError("fd %s added twice" % fd)
ValueError: fd 700 added twice
ERROR:asyncio:Future exception was never retrieved
future: <Future finished exception=HTTP 599: SSL certificate problem: unable to get local issuer certificate>
tornado.curl_httpclient.CurlError: HTTP 599: SSL certificate problem: unable to get local issuer certificate
Process finished with exit code -1
I'm not sure if this is the only problem here, but the NotImplementedError is because Python 3.8 on Windows uses a different event loop implementation that is incompatible with Tornado. You need to add asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) to the beginning of your main file/function.
I suspect you may also need to use the ca_certs argument to tell libcurl where to find the trusted root certificates for your proxy.

Connection to remote MySQL db from Python 3.4

I'm trying to connect to two MySQL databases (one local, one remote) at the same time using Python 3.4 but I'm really struggling. Splitting the problem into three:
Step 1: connect to the local DB. This is working fine
using PyMySQL. (MySQLdb isn't compatible with Python 3.4, of
course.)
Step 2: connect to the remote DB (which needs to
use SSH). I can get it to work from the Linux command prompt but not
from Python... see below.
Step 3: connect to both at the
same time. I think I'm supposed to use a different port for the
remote database so that I can have both connections at the same time
but I'm out of my depth here! If it's relevant then the two DBs will
have different names. And if this question isn't directly related,
please tell me and I'll post it separately.
Unfortunately I'm not really starting in the right place for a newbie... once I can get this working I can happily go back to basic Python and SQL but hopefully someone will take pity on me and give me a hand to get started!
For Step 2, my code is below. It seems to be quite close to the sshtunnel example which answers this question Python - SSH Tunnel Setup and MySQL DB Access - though that uses MySQLdb. For the moment I'm embedding the connection parameters – I'll move them to the config file once it's working properly.
import dropbox, pymysql, shlex, shutil, subprocess
from sshtunnel import SSHTunnelForwarder
import iot_config as cfg
def CloseLocalDB():
localcur.close()
localdb.close()
def CloseRemoteDB():
# Disconnect from the database
# remotecur.close()
# remotedb.close()
# Close the SSH tunnel
# ssh.close()
print("end of CloseRemoteDB function")
def OpenLocalDB():
global localcur, localdb
localdb = pymysql.connect(host=cfg.localdbconn['host'], user=cfg.localdbconn['user'], passwd=cfg.localdbconn['passwd'], db=cfg.localdbconn['db'])
localcur = localdb.cursor()
def OpenRemoteDB():
global remotecur, remotedb
with SSHTunnelForwarder(
('my_remote_site', 22),
ssh_username = "my_ssh_username",
ssh_private_key = "/etc/ssh/my_private_key.ppk",
ssh_private_key_password = "my_private_key_password",
remote_bind_address = ('127.0.0.1', 3308)) as server:
remotedb = None
#Following line gives an error if uncommented
# remotedb = pymysql.connect(host='127.0.0.1', user='remote_db_user', passwd='remote_db_password', db='remote_db_name', port=server.local_bind_port)
#remotecur = remotedb.cursor()
# Main program starts here
OpenLocalDB()
CloseLocalDB()
OpenRemoteDB()
CloseRemoteDB()
This is the error I'm getting:
2016-04-21 19:13:33,487 | ERROR | Secsh channel 0 open FAILED: Connection refused: Connect failed
2016-04-21 19:13:33,553 | ERROR | In #1 <-- ('127.0.0.1', 60591) to ('127.0.0.1', 3308) failed: ChannelException(2, 'Connect failed')
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 60591)
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/sshtunnel.py", line 286, in handle
src_address)
File "/usr/local/lib/python3.4/dist-packages/paramiko/transport.py", line 834, in open_channel
raise e
paramiko.ssh_exception.ChannelException: (2, 'Connect failed')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/socketserver.py", line 613, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python3.4/socketserver.py", line 669, in __init__
self.handle()
File "/usr/local/lib/python3.4/dist-packages/sshtunnel.py", line 296, in handle
raise HandlerSSHTunnelForwarderError(msg)
sshtunnel.HandlerSSHTunnelForwarderError: In #1 <-- ('127.0.0.1', 60591) to ('127.0.0.1', 3308) failed: ChannelException(2, 'Connect failed')
----------------------------------------
Traceback (most recent call last):
File "/home/pi/Documents/iot_pm2/iot_ssh_example_for_help.py", line 38, in <module>
OpenRemoteDB()
File "/home/pi/Documents/iot_pm2/iot_ssh_example_for_help.py", line 32, in OpenRemoteDB
remotedb = pymysql.connect(host='127.0.0.1', user='remote_db_user', passwd='remote_db_password', db='remote_db_name', port=server.local_bind_port)
File "/usr/local/lib/python3.4/dist-packages/pymysql/__init__.py", line 88, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/pymysql/connections.py", line 678, in __init__
self.connect()
File "/usr/local/lib/python3.4/dist-packages/pymysql/connections.py", line 889, in connect
self._get_server_information()
File "/usr/local/lib/python3.4/dist-packages/pymysql/connections.py", line 1190, in _get_server_information
packet = self._read_packet()
File "/usr/local/lib/python3.4/dist-packages/pymysql/connections.py", line 945, in _read_packet
packet_header = self._read_bytes(4)
File "/usr/local/lib/python3.4/dist-packages/pymysql/connections.py", line 981, in _read_bytes
2013, "Lost connection to MySQL server during query")
pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query')
Thanks in advance.
Answering my own question because, with a lot of help from J.M. Fernández on Github, I have a solution: the example that I copied at the beginning uses port 3308 but port 3306 is the standard. Once I'd changed this it started working.

Python determine if HTTPConnection failed to connect

In Python, how I can determine if HTTPConnection failed to connect to server, for example, by bad server address?
import httplib
import sys
if len(sys.argv) != 2:
print 'Usage: server'
quit()
con = httplib.HTTPConnection(argv[1]) # how I can determine if this fails?
Try this after instantiating the object con.connect().
If the connection is unable to be made, this call will throw an exception.
Example:
>>> import httplib
>>> con = httplib.HTTPConnection('127.0.0.1')
>>> con.connect()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/httplib.py", line 772, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 111] Connection refused
You could try adding try() and except() statements.

Categories