This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
What RPC module should I use to implement RCP in Python and be able to change connection method later?
I am looking for RPC solution that can be used over different protocols like SSH, telnet and HTTP.
It has to be Python 2.5 compatible.
You're likely going to have to roll your own, but much of the heavy lifting in transport code could be done in other modules:
paramiko for ssh
telnetlib for telnet
urllib(2) for http.
You'll still have to address the issue of data format, but that is independent of transport protocol (feel free to deliver XML-RPC or JSON or any other format over these transports).
Related
This question already has an answer here:
HTTP Requests using a range of IP address on python
(1 answer)
Closed 8 years ago.
I'm writing a python script that will send http requests concurrently to the urls mentioned in a file using python. The script works fine for a single IP address. The OS I'm using is linux. I've generated virtual IP addresses like eth0:1,eth0:2 etc. I want to send HTTP requests using these virtual IP addresses along with the eth0 IP address concurrently. I use the requests module for http requests and threading module for concurrent requests. Kindly help me. I'm trying to develop a web testing tool.
I think you wanted to avoid "Crawl Delay" and do faster crawl on one server!
In this case, Remote Web Server will recognize request from only one IP!!
I think using parallel + curl + python script is more simple and best way.
or use https://pypi.python.org/pypi/pyparallelcurl/0.0.4
or use a lot of servers.
and refer to https://code.google.com/p/httplib2/issues/detail?id=91
This question already has answers here:
Python requests - print entire http request (raw)?
(9 answers)
Closed 8 years ago.
I am looking for a recipe for writing and reading the raw data generated by a requests transaction from files rather than a socket. By "raw data" I mean the bytes just before they are written to or read from the underlying socket. I've tried:
Using "hooks". This seems to be mostly deprecated as the only remaining hook is "response".
mount()ing a custom Adapter. Some aggressive duck-typing here provides access to the underlying httplib.HTTPConnection objects, but the call stack down there is complicated and quite brittle.
The final solution does not need to be general-purpose as I am only interested in vanilla HTTP functionality. I won't be streaming or using the edgier parts of the protocol.
Thanks!
Spawn a thread (import threading). Run an HTTP server in there. You can generate a unique port on demand by socket.socket().bind(0). In the HTTP server, just write the incoming data to a file (perhaps named by timestamp and incoming port number). Then send your requests there.
What's the easiest way to establish an emulated TCP connection over HTTP with python 2.7.x?
Server: a python program on pythonanywhere (or some analogue) free hosting, that doesn't provide a dedicated ip. Client: a python program on a Windows PC.
Connection is established via multiprocessing.BaseManager and works fine when testing both server and client on the same machine.
Is there a way to make this work over HTTP with minimal additions to the code?
P.S. I need this for a grid computing project.
P.P.S. I'm new to python & network & web programming, started studying it several days ago.
Found this: http://code.activestate.com/recipes/577643-transparent-http-tunnel-for-python-sockets-to-be-u/. Appears to be exactly what I need, though I don't understand how to invoke setup_http_proxy() on server/client side. Tried setup_http_proxy("my.proxy", 8080) on both sides, but it didn't work.
Also found this: http://docs.python.org/2/library/httplib.html. What does the HTTPConnection.set_tunnel method actually do? Can I use it to solve the problem in question?
Usage on the client:
setup_http_proxy("THE_ADRESS", THE_PORT_NUMBER) # address of the Proxy, port the Proxy is listening on
The code wraps sockets to perform an initial HTTP CONNECT request to the proxy setup to get an HTTP Proxy to proxy the TCP connection for you but for that you'll need a compliant proxy (most won't allow you to open TCP connections unless it's for HTTPS).
HTTPConnection.set_tunnel basically does the same thing.
For your use case, a program running on free hosting, this just won't work. Your free host probably will only allow you to handle http requests, not have long running processes listen for tcp connections(which the code assumes).
You should rethink your need to tunnel and organize your communication to post data (and poll for messages from the server, unless they're answers to the stuff you post). Or you can purchase a VPS hosting that will give you more control over what you can host remotely.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
TCP client-server SIGPIPE
I would like know what does this error mean?
You are using sockets and Http protocol.
It simply means your TCP connection has been closed by the other end or broken due to some other reason. By broken it means a 3 way handshake is required again before starting data transfer. As mentioned in the comments, being on listening end i.e. server, you normally cannot initiate the connection. So should simply close this socket and proceed ahead.
However, if you were a client, you should probably call api similar to connect again and proceed once it is successful.
Broken pipe on SO
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
What is the practical / hard limit on socket connections per server
When I hit about 480 TCP connections, it doesn't accept + connections.
Where it is the max of TCP connections?
Windows 2008..
It depends on the Windows 2008 edition, Web and Foundation editions have connection limits while Standard, Enterprise, and Datacenter do not.
Hope that helps...
Python does not limit connections. It is controlled/limited by the underlying operating system and network stack.
However, if you are working directly with python's socket module, you can adjust some parameters using socket.setsockopt(level, optname, value) before you bind to the port.
you can also tweak windows registry settings to enable more concurrent connections.
see: http://technet.microsoft.com/en-us/magazine/2007.12.network.aspx
the TcpNumConnections, MaxUserPort,and EnableConnectionRateLimiting tcp settings are good places to start.