I am developing a bot with module pyTelegramBotAPI, which is working via webhooks with installed Flask+Gunicorn as server for webhooks. Gunicorn is working with 5 workers for better speed, structure of my project looks like that:
app.py
bot.py
In bot.py i have a function for processing updates:
def process_the_update(update):
logger.info(update)
update = types.Update.de_json(update)
bot.process_new_updates([update])
In app.py i imported this function, so, whenever an update comes, app.py will call this function, and bot will handle the update. In my bot user can call a command, which will use an external api to get some information. The problem is, that this external api has limit for 3 requests per second. I need to configure a bot with such rate limit. First i thought of doing it with Queue with code like this:
lock_queue = Queue(1)
requests_queue = Queue(3)
def api_request(argument):
if lock_queue.empty():
try:
requests_queue.put_nowait(time.time())
except queue.Full:
lock_queue.put(1)
first_request_time = requests_queue.get()
logger.info('First request time: ' + str(first_request_time))
current_time = time.time()
passed_time = current_time - first_request_time
if passed_time >= 1:
requests_queue.put_nowait(time.time())
lock_queue.get()
else:
logger.info(passed_time)
time.sleep(1 - passed_time)
requests_queue.put_nowait(time.time())
lock_queue.get()
else:
lock_queue.put(1)
first_request_time = vk_requests_queue.get()
logger.info('First request time: ' + str(first_request_time))
current_time = time.time()
passed_time = current_time - first_request_time
if passed_time >= 1:
requests_queue.put_nowait(time.time())
lock_queue.get()
else:
logger.info(passed_time)
time.sleep(1 - passed_time)
requests_queue.put_nowait(time.time())
lock_queue.get()
result = make_api_request(argument) # requests are made too by external module.
return result
The logic was that, as i thought, because module pyTelegramBotAPI uses threads for faster updates handling, all threads would check requests_queue, which will have time of 3 last api_requests, and so the time of the first of 3 requests made will be compared to the current time (to check, if a second passed). And, because i needed to be sure, that only one thread would do this kind of comparison simultaneously, i made lock_queue.
But, the problem is that, firstly, gunicorn uses 5 workers, so there will be always possibility, that all messages from users will be handled in different processes, and these processes will have their own queues. And, secondly, even if i set number of workers to default (1 worker), i still get 429 error, so i think, that my code won't work as i wanted at all.
I thought of making rate limit with redis, so every time in every thread and process bot will check the time of last 3 requests, but still i am not sure, that this is the right way, and i am not sure, how to write this.
Would be glad, if someone suggests any ideas or even examples of code (external api does not provide any x-rate-limit header)
Wrote this function, using redis to count requests (based on this https://www.binpress.com/tutorial/introduction-to-rate-limiting-with-redis/155 tutorial)
import redis
r_db = redis.Redis(port=port, db=db)
def limit_request(request_to_make, limit=3, per=1, request_name='test', **kwargs):
over_limit_lua_ = '''
local key_name = KEYS[1]
local limit = tonumber(ARGV[1])
local duration = ARGV[2]
local key = key_name .. '_num_of_requests'
local count = redis.call('INCR', key)
if tonumber(count) > limit then
local time_left = redis.call('PTTL', key)
return time_left
end
redis.call('EXPIRE', key, duration)
return -2
'''
if not hasattr(r_db, 'over_limit_lua'):
r_db.over_limit_lua = r_db.register_script(over_limit_lua_)
request_possibility = int(r_db.over_limit_lua(keys=request_name, args=[limit, per]))
if request_possibility > 0:
time.sleep(request_possibility / 1000.0)
return limit_request(request_to_make, limit, per, request_name, **kwargs)
else:
request_result = request_to_make(**kwargs)
return request_result
Related
I want to parallely send a GET request for the specified count say 100 times. How to achieve this using JMeter or Python ?
I tried bzm parallel executor but that doesn't workout.
import requests
import threading
totalRequests = 0
numberOfThreads = 10
threads = [0] * numberOfThreads
def worker(thread):
r = requests.get("url")
threads[thread] = 0 # free thread
while totalRequests < 100:
for thread in range(numberOfThreads):
if threads[thread] == 0:
threads[thread] = 1 # occupy thread
t = threading.Thread(target=worker, args=(thread,))
t.start()
totalRequests += 1
In JMeter:
Add Thread Group to your Test Plan and configure it like:
Add HTTP Request sampler as a child of the Thread Group and specify protocol, host, port, path and parameters:
if you're not certain regarding properly configuring the HTTP Request sampler - you can just record the request using your browser and JMeter's HTTP(S) Test Script Recorder or JMeter Chrome Extension
For Python the correct would be using Locust framework as I believe you're interested in metrics like response times, latencies and so on. The official website is down at the moment
so in the meantime you can check https://readthedocs.org/projects/locust/
I have a pyzmq Publisher which sends around 1000 messages per second. I am trying to start around 10 Subscribers in an asyncio event_loop.
It works but around 2.5 times slower than speed of the only one Subscriber.
What could possibly be wrong with the code?
import asyncio
import zmq
import json
from zmq.backend.cython.constants import NOBLOCK
from zmq.asyncio import Context, Poller
from loop_ import Loop
class Client:
REQUEST_TIMEOUT = 35000
SERVER_ENDPOINT = "tcp://localhost:6666"
def __init__(self, id_):
self.id = id_
def get_task(self):
return asyncio.create_task(self.client_coroutine())
async def client_coroutine(self):
context = Context.instance()
socket = context.socket(zmq.SUB)
socket.connect(self.SERVER_ENDPOINT)
socket.setsockopt(zmq.SUBSCRIBE, b'4')
poller = Poller()
poller.register(socket, zmq.POLLIN)
while True:
event = dict(await poller.poll(self.REQUEST_TIMEOUT))
if event.get(socket) == zmq.POLLIN:
reply = await socket.recv_multipart(flags=NOBLOCK)
if not reply:
break
else:
print(eval(json.loads(reply[1].decode('utf-8'))))
else:
print("No response from server, retrying...")
socket.setsockopt(zmq.LINGER, 0)
socket.close()
poller.unregister(socket)
async def tasks():
_tasks = [Client(id_).get_task() for id_ in range(10)]
done, pending = await asyncio.wait(_tasks, return_when=asyncio.FIRST_EXCEPTION)
loop = asyncio.get_event_loop()
loop.run_until_complete(tasks())
Q : What could possibly be wrong with the code?
Given the code is using the same localhost ( as seen from using the address ), the suspect number one is, that having 10x more work to process, the such workload will always stress the localhost's O/S and the CPU, won't it?
Next comes the choice of the transport-class. Given all the SUB-s are co-located on the same localhost as the PUB, there is all the L3-stack-based TCP/IP protocol work going wasted. To compare the relative costs ( the add-on effect of using the tcp:// transport-class for this hardware-singular messaging ), test the very same with using inproc:// transport-class, where none of the protocol-related TCP/IP-stack add-on processing will take place.
Last, but not least, my code will never mix different event-loops ( using ZeroMQ since v2.11, so someone may consider my a bit old-fashioned in avoiding relying on async-decorated capabilities available in recent py3.6+ )
My code will use an explicit, non-blocking, zero-waiting test for a presence of a message per-aSocketINSTANCE, as in aSocketINSTANCE.poll( zmq.POLLIN, 0 ) rather than using any "externally" added decoration, which may report the same, but via some additional (expensive and outside of my code domain of control) event-handling. All real-time, low-latency use-cases strive to bear as minimum latency/overheads as possible, so using explicit control will always win in my Projects, to any "modern" syntax-sugar sweetened tricks.
Anyway, enjoy the Zen-of-Zero
I am using sleekxmpp as the xmpp client for python. The requests are coming which I am further forwarding to other users/agents.
Now the use case is, if a user is not available we need to check the availability every 10 seconds and transfer to him when he is available. We need to send a message to customer only 5 times but check the availability for a long time.
I am using time.sleep() if the user is not available to check again in 10 seconds, but the problem is it is blocking the entire thread and no new requests are coming to the server.
send_msg_counter = 0
check_status = False
while not check_status:
check_status = requests.post(transfer_chat_url, data=data)
if send_msg_counter < 5:
send_msg("please wait", customer)
send_msg_counter += 1
time.sleep(10)
This is true that time.sleep(10) will block your active thread. You may actually find that using Python 3's async/await to be the way to go. Sadly I don't have much experience with those keywords yet, but another route might be to implement python's threading.
https://docs.python.org/3/library/threading.html
Here might be one way to implement this feature.
import threading
def poll_counter(customer, transfer_chat_url, data, send_count=5, interval=10):
send_msg_counter = 0
check_status = False
while not check_status:
check_status = requests.post(transfer_chat_url, data=data)
if send_msg_counter < send_count:
send_msg("please wait", customer)
send_msg_counter += 1
time.sleep(interval)
# If we're here, check status became true
return None
... pre-existing code ...
threading.Thread(target=poll_counter, args=(customer, transfer_chat_url, data)).start()
... proceed to handle other tasks while the thread runs in the background.
Now, I won't go into detail, but there are use cases where threading is a major mistake. This shouldn't be one of them, but here is a good read for you to understand those use cases.
https://realpython.com/python-gil/
Also, for more details on asyncio (async/await) here is a good resource.
https://docs.python.org/3/library/asyncio-task.html
Try implementing
delay = min(self.reconnect_delay * 2, self.reconnect_max_delay)
delay = random.normalvariate(delay, delay * 0.1)
log.debug('Waiting %s seconds before connecting.', delay)
elapsed = 0
try:
while elapsed < delay and not self.stop.is_set():
time.sleep(0.1)
elapsed += 0.1
except KeyboardInterrupt:
self.set_stop()
return False
except SystemExit:
self.set_stop()
return False
Source Link
I am working on my first Step Functions Activity Worker (EC2). Predictably, after 5 minutes of long polling with no activity from the Step Functions state machine, the client connection times out with the error:
botocore.exceptions.ReadTimeoutError: Read timeout on endpoint URL: "https://states.us-east-1.amazonaws.com/"
Would it be better to catch the error and retry the long poll (every 5 minutes when no activity is present), or try to terminate the call early and retry before the error? I've thought about using a different type of loop, but I want to maximize the value of long polling and not repeatedly request against the Step Functions API (although if that's the best way I'll do it).
Thank you,
Andrew
import boto3
import time
import json
region = 'us-east-1'
activity_arn = 'arn:aws:states:us-east-1:754112345676:activity:Process_Imagery'
while True:
client = boto3.client('stepfunctions', region_name=region)
response = client.get_activity_task(activityArn=activity_arn,
workerName='imagery_processor')
activity_token = response['taskToken']
input_params = json.loads(response['input'])
print("================")
print(input_params)
client.send_task_success(taskToken=activity_token, output='true')
I believe I answered my own question here. The AWS documentation states:
"The maximum time the service holds on to the request before responding is 60 seconds. If no task is available within 60 seconds, the poll returns a taskToken with a null string."
However, instead of string being returned, I believe the JSON response from StepFunctions has no 'taskToken' at all. This while loop works:
import boto3
import time
import json
from botocore.config import Config as BotoCoreConfig
region = 'us-east-1'
boto_config = BotoCoreConfig(read_timeout=70, region_name=region)
sf_client = boto3.client('stepfunctions', config=boto_config)
activity_arn = 'arn:aws:states:us-east-1:754185699999:activity:Process_Imagery'
while True:
response = sf_client.get_activity_task(activityArn=activity_arn,
workerName='imagery_processor')
if 'taskToken' not in response:
print('No Task Token')
# time.sleep(2)
else:
print(response['taskToken'])
print("===================")
activity_token = response['taskToken']
sf_client.send_task_success(taskToken=activity_token, output='true')
How can I run background tasks on App Engine?
You may use the Task Queue Python API.
GAE is very useful tool to build scalable web applications. Few of the limitations pointed out by many are no support for background tasks, lack of periodic tasks and strict limit on how much time each HTTP request takes, if a request exceeds that time limit the operation is terminated, which makes running time consuming tasks impossible.
How to run background task ?
In GAE the code is executed only when there is a HTTP request. There is a strict time limit (i think 10secs) on how long the code can take. So if there are no requests then code is not executed. One of the suggested work around was use an external box to send requests continuously, so kind of creating a background task. But for this we need an external box and now we dependent on one more element. The other alternative was sending 302 redirect response so that client re-sends the request, this also makes us dependent on external element which is client. What if that external box is GAE itself ? Everyone who has used functional language which does not support looping construct in the language is aware of the alternative ie recursion is the replacement to loop. So what if we complete part of the computation and do a HTTP GET on the same url with very short time out say 1 second ? This creates a loop(recursion) on php code running on apache.
<?php
$i = 0;
if(isset($_REQUEST["i"])){
$i= $_REQUEST["i"];
sleep(1);
}
$ch = curl_init("http://localhost".$_SERVER["PHP_SELF"]."?i=".($i+1));
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 1);
curl_exec($ch);
print "hello world\n";
?>
Some how this does not work on GAE. So what if we do HTTP GET on some other url say url2 which does HTTP GET on the first url ? This seem to work in GAE. Code for this looks like this.
class FirstUrl(webapp.RequestHandler):
def get(self):
self.response.out.write("ok")
time.sleep(2)
urlfetch.fetch("http://"+self.request.headers["HOST"]+'/url2')
class SecondUrl(webapp.RequestHandler):
def get(self):
self.response.out.write("ok")
time.sleep(2)
urlfetch.fetch("http://"+self.request.headers["HOST"]+'/url1')
application = webapp.WSGIApplication([('/url1', FirstUrl), ('/url2', SecondUrl)])
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
Since we found out a way to run background task, lets build abstractions for periodic task (timer) and a looping construct which spans across many HTTP requests (foreach).
Timer
Now building timer is straight forward. Basic idea is to have list of timers and the interval at which each should be called. Once we reach that interval call the callback function. We will use memcache to maintain the timer list. To find out when to call callback, we will store a key in memcache with interval as expiration time. We periodically (say 5secs) check if that key is present, if not present then call the callback and again set that key with interval.
def timer(func, interval):
timerlist = memcache.get('timer')
if(None == timerlist):
timerlist = []
timerlist.append({'func':func, 'interval':interval})
memcache.set('timer-'+func, '1', interval)
memcache.set('timer', timerlist)
def checktimers():
timerlist = memcache.get('timer')
if(None == timerlist):
return False
for current in timerlist:
if(None == memcache.get('timer-'+current['func'])):
#reset interval
memcache.set('timer-'+current['func'], '1', current['interval'])
#invoke callback function
try:
eval(current['func']+'()')
except:
pass
return True
return False
Foreach
This is needed when we want to do long taking computation say doing some operation on 1000 database rows or fetch 1000 urls etc. Basic idea is to maintain list of callbacks and arguments in memcache and each time invoke callback with the argument.
def foreach(func, args):
looplist = memcache.get('foreach')
if(None == looplist):
looplist = []
looplist.append({'func':func, 'args':args})
memcache.set('foreach', looplist)
def checkloops():
looplist = memcache.get('foreach')
if(None == looplist):
return False
if((len(looplist) > 0) and (len(looplist[0]['args']) > 0)):
arg = looplist[0]['args'].pop(0)
func = looplist[0]['func']
if(len(looplist[0]['args']) == 0):
looplist.pop(0)
if((len(looplist) > 0) and (len(looplist[0]['args']) > 0)):
memcache.set('foreach', looplist)
else:
memcache.delete('foreach')
try:
eval(func+'('+repr(arg)+')')
except:
pass
return True
else:
return False
# instead of
# foreach index in range(0, 1000):
# someoperaton(index)
# we will say
# foreach('someoperaton', range(0, 1000))
Now building a program which fetches list of urls every one hour is straight forward. Here is the code.
def getone(url):
try:
result = urlfetch.fetch(url)
if(result.status_code == 200):
memcache.set(url, '1', 60*60)
#process result.content
except :
pass
def getallurl():
#list of urls to be fetched
urllist = ['http://www.google.com/', 'http://www.cnn.com/', 'http://www.yahoo.com', 'http://news.google.com']
fetchlist = []
for url in urllist:
if (memcache.get(url) is None):
fetchlist.append(url)
#this is equivalent to
#for url in fetchlist: getone(url)
if(len(fetchlist) > 0):
foreach('getone', fetchlist)
#register the timer callback
timer('getallurl', 3*60)
complete code is here http://groups.google.com/group/httpmr-discuss/t/1648611a54c01aa
I have been running this code on appengine for few days without much problem.
Warning: We make heavy use of urlfetch. The limit on no of urlfetch per day is 160000. So be careful not to reach that limit.
You can find more about cron jobs in Python App Engine here.
Up and coming version of runtime will have some kind of periodic execution engine a'la cron. See this message on AppEngine group.
So, all the SDK pieces appear to work, but my testing indicates this isn't running on the production servers yet-- I set up an "every 1 minutes" cron that logs when it runs, and it hasn't been called yet
Hard to say when this will be available, though...
Using the Deferred Python Library is the easiest way of doing background task on Appengine using Python which is built on top of TaskQueue API.
from google.appengine.ext import deferred
def do_something_expensive(a, b, c=None):
logging.info("Doing something expensive!")
# Do your work here
# Somewhere else
deferred.defer(do_something_expensive, "Hello, world!", 42, c=True)
If you want to run background periodic tasks, see this question (AppEngine cron)
If your tasks are not periodic, see Task Queue Python API or Task Queue Java API
There is a cron facility built into app engine.
Please refer to:
https://developers.google.com/appengine/docs/python/config/cron?hl=en
Use the Task Queue - http://code.google.com/appengine/docs/java/taskqueue/overview.html