I need to use http_wait link with telethon, are there already made functions in the library to use that specific method?
I need to receive messages as soon as they occur is large broadcast channels, now the updates come 5-20 seconds late
Clients using the Telegram API, such as Telethon, connect to the Telegram servers directly via a TCP socket. While connected, Telegram decides when and where to deliver the updates. Telegram's API doesn't really offer a way to "poll" for these updates.
If Telegram is delivering them slowly, it's probably to reduce load, or because the channel is too large, or because the client is not being actively used. In essence, it's not something the library can "fix".
Related
I am working on MQTT and using python paho-mqtt https://pypi.python.org/pypi/paho-mqtt
I am unable to understand how can I publish msg to a specific client or list of clients?
I'll appreciate your help.
This isn't directly possible with strict MQTT, although some brokers may offer that functionality, or you can construct your application so that the topic design works to do what you need.
Although I do agree that in some cases it would be useful to send a message to a particular client (or list of clients) that's simply not how the publish/subscribe messaging paradigm works. Read more on the publish-subscribe pattern on Wikipedia. If all your system needs to do is send messages to unique clients, then I would perhaps suggest thinking of a different architecture for the system you are designing. That being said, you can leverage off pub/sub to achieve what you want using a clever topic design architecture.
For example, let's assume all clients are part of a group (list), you could think of the following topic design:
Unique per client: P2P/< client-name >
List/Group subscription: LIST/< list-name >
For example, P2P/user12345 and LIST/QA where only user12345 subscribes to P2P/user12345 but all users of the QA group subscribe to LIST/QA.
It would be the client's responsibility to ensure that it is subscribed to its own topic(s) (or if your broker allows it, you could also add the topics administratively to non-clean clients).
With this design, a publisher would be able to send a message to a specific user or all members of a defined group (list).
I'm looking to start a web project using Flask and its SocketIO plugin, which depends on gevent (something something greenlets), but I don't understand how gevent relates to the webserver. Does using gevent restrict my server choice at all? How does it relate to the different levels of web servers that we have in python (e.g. Nginx/Apache, Gunicorn)?
Thanks for the insight.
First, lets clarify what we are talking about:
gevent is a library to allow the programming of event loops easily. It is a way to immediately return responses without "blocking" the requester.
socket.io is a javascript library create clients that can maintain permanent connections to servers, which send events. Then, the library can react to these events.
greenlet think of this a thread. A way to launch multiple workers that do some tasks.
A highly simplified overview of the entire process follows:
Imagine you are creating a chat client.
You need a way to notify the user's screens when anyone types a message. For this to happen, you need someway to tell all the users when a new message is there to be displayed. That's what socket.io does. You can think of it like a radio that is tuned to a particular frequency. Whenever someone transmits on this frequency, the code does something. In the case of the chat program, it adds the message to the chat box window.
Of course, if you have a radio tuned to a frequency (your client), then you need a radio station/dj to transmit on this frequency. Here is where your flask code comes in. It will create "rooms" and then transmit messages. The clients listen for these messages.
You can also write the server-side ("radio station") code in socket.io using node, but that is out of scope here.
The problem here is that traditionally - a web server works like this:
A user types an address into a browser, and hits enter (or go).
The browser reads the web address, and then using the DNS system, finds the IP address of the server.
It creates a connection to the server, and then sends a request.
The webserver accepts the request.
It does some work, or launches some process (depending on the type of request).
It prepares (or receives) a response from the process.
It sends the response to the client.
It closes the connection.
Between 3 and 8, the client (the browser) is waiting for a response - it is blocked from doing anything else. So if there is a problem somewhere, like say, some server side script is taking too long to process the request, the browser stays stuck on the white page with the loading icon spinning. It can't do anything until the entire process completes. This is just how the web was designed to work.
This kind of 'blocking' architecture works well for 1-to-1 communication. However, for multiple people to keep updated, this blocking doesn't work.
The event libraries (gevent) help with this because they accept and will not block the client; they immediately send a response and when the process is complete.
Your application, however, still needs to notify the client. However, as the connection is closed - you don't have a way to contact the client back.
In order to notify the client and to make sure the client doesn't need to "refresh", a permanent connection should be open - that's what socket.io does. It opens a permanent connection, and is always listening for messages.
So work request comes in from one end - is accepted.
The work is executed and a response is generated by something else (it could be a the same program or another program).
Then, a notification is sent "hey, I'm done with your request - here is the response".
The person from step 1, listens for this message and then does something.
Underneath is all is WebSocket a new full-duplex protocol that enables all this radio/dj functionality.
Things common between WebSockets and HTTP:
Work on the same port (80)
WebSocket requests start off as HTTP requests for the handshake (an upgrade header), but then shift over to the WebSocket protocol - at which point the connection is handed off to a websocket-compatible server.
All your traditional web server has to do is listen for this handshake request, acknowledge it, and then pass the request on to a websocket-compatible server - just like any other normal proxy request.
For Apache, you can use mod_proxy_wstunnel
For nginx versions 1.3+ have websocket support built-in
I have this AutobahnPython server up and running fine.
https://github.com/tavendo/AutobahnPython/blob/master/examples/websocket/streaming/streaming_server.py
I want to attach a HTML5 Front end for capture of web cam video and audio.
How do I get the HTML5 Blob to send through the socket I just created in HTML5 to the python socket server I also have running?
Is it sendMessage?
https://autobahnpython.readthedocs.org/en/latest/websocketbase.html#autobahn.websocket.WebSocketProtocol.sendMessage
Be prepared, doing what you want, and doing it right (which means flow-control), is an advanced topic. I try to give you a couple of hints. You might be also interested in reading this.
WebSocketProtocol.sendMessage is part of the AutobahnPython API. To be precise, it is part of the message-based basic API. Whereas the streaming server above uses the advanced API for receiving, it uses the basic API for sending (since the sent data is small, and there is no need for flow control)
Now, in your case, the web cam is the "mass data" producer. You will want to flow-control the sending from the JS to the server. Since if you just send out WebSocket messages from JS as fast as you get data from cam, your upstream connection might not keep up, and the browser's memory will just run away. Read about bufferedAmount which is part of the JS WebSocket API.
If you just want to consume data is it flow into your server, above AutobahnPython streaming server example is a good starting point since: you can process WebSocket data as it comes in. Other WebSocket frameworks will first buffer up a complete message until they give the message to you.
If you want to redistribute the data received by your server again to other connected client, you will want flow-control on the server's outgoing leg also. And then you will need the advanced API for sending also. See the reference or the streaming (producer) client examples - you can adjust the code to run inside your server.
Now if above all does not make sense to you .. it's a non-trivial thing. Try reading the first link to the Autobahn forum, and more about flow-control. It is also non-trivial since the JS WebSocket API has only limited machinery for doing this kind of flow-control, without falling back to invent your own scheme at app level. Well. Anyway, hope that helps a little.
I'm using GAE + Python to create an application that needs to send real-time updates of sensitive data to clients and I wanted to know if the App Engine Channel API is secure or not. Will using HTTPS be enough or do channels require their own security protocol?
Also, what is the underlying implementation of the App Engine Channel API? Websockets, SSE? It seems like it really only provides one way communication from server to client through the channel, and then has the client use a standard HTTP request to communicate with the server.
Connections to the channel API are made over HTTPS, regardless of how your page was loaded, so it's not possible to eavesdrop on the contents of a channel API connection. As long as you keep the channel key secret, then, your channel is a secure communications channel between your app and the client.
Channels are implemented using long polling (comet).
Because channels are long-term-alive connections between server and a client, channels are not allowed to use resource consuming security approaches in many cases due to performance consideration. As it is declared in the official manual, The server only receives update messages from clients via HTTP requests. And as far as I know, even Dropbox sends its long-term notify message via HTTP, using a very short notify only to tell whether there is something new.
Fortunately, there are two ways to ensure your security.
Only notify your client via the HTTP channel when some states change. After that, let the client decide whether a further request, which can be a secure communication, should be made. And this is the most common way channels are used.
Although this is not the way I personally recommended, you can encrypt your data yourself and put these encrypted data run on the insecure HTTP channel.
I am developing a testbed for cloud computing environment. I want to establish multiple client connection to a server. What I want is that, server first of all send a data to all the clients specifying sending_interval and then all the clients will keep on sending their data with a time gap of that time_interval (as specified by the server). Please help me out, how can I do the same using python socket program. (i.e. I want multiple client to single server connectivity and also client sending data with the time gap specified by server). Will be great-full if anyone can help me. Thanks in advance.
This problem is easily solved by the ZeroMQ socket library. It is production stable. It allows you to define publisher-subscriber relationships, where a publishing process will publish data on a port regardless of how many (0 to infinite) listening processes there are. They call this the PUB-SUB model; it's in their docs (link below).
It sounds like you want to set up a bunch of clients that are all publishers. They can subscribe to a controlling channel, which which will send updates to their configuration (how often to write). They also act as publishers, pushing out their own data at an interval specified by default/config channel/socket.
Then, you have one or more listening processes that listen to all the clients' published messages. Perhaps you could even have two listening processes, one for backup or DR, or whatever.
We're using ZeroMQ and loving the simplicity it gives; there's no connection errors because the publisher doesn't care if anyone is listening, and the subscriber can start before the publisher and if there's nothing there to listen to, it can just loop around and wait until there is.
Bindings are available in ALL languages (it's freaky). The Python binding isn't pure-python, it does require a C compiler, but is frighteningly fast, and the pub/sub example is a cut/paste, 'golly, it works!' experience.
Link: http://zeromq.org
There are MANY other methods available with this library, including message queues, etc. They have relatively complete documentation, too.
Multi-Client and Single server Socket programming can be achieved by Multithreading in Socket Programming. I have implemented both the method:
Single Client and Single Server
Multiclient and Single Server
In my GitHub Repo Link: https://github.com/shauryauppal/Socket-Programming-Python
What is Multi-threading Socket Programming?
Multithreading is a process of executing multiple threads simultaneously in a single process.
To understand well you can visit Link: https://www.geeksforgeeks.org/socket-programming-multi-threading-python/, written by me.