I try to collecting data from Twitter with Python and tweepy.
My code is :
import tweepy
consumer_key="..."
consumer_secret="..."
access_key = "..."
access_secret = "..."
class CustomStreamListener(tweepy.StreamListener):
def on_status(self, status):
print (status.text)
def on_error(self, status_code):
print >> sys.stderr, 'Encountered error with status code:', status_code
return True # Don't kill the stream
def on_timeout(self):
print >> sys.stderr, 'Timeout...'
return True # Don't kill the stream
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
sapi = tweepy.streaming.Stream(auth, CustomStreamListener())
sapi.filter(track=['capital'], async=True)
In my console, python return :
RT #gucamo74: #rubenuria eso es muy difícil. El Sevilla no tiene el halo protector arbitral de los equipos de la capital y del Barcelona.
RT #TonySantanaZA: #woznyjs On Macro scale, we failed 2 create Corporate stability, 4 investing Companies. Hence Capital flight,2 other mor… "Pour ne pas se faire rouler"...... #Capital
So it's fine but after few tweets he show me this message :
Exception in thread Thread-10:
Traceback (most recent call last):
File "//anaconda/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "//anaconda/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "//anaconda/lib/python3.5/site-packages/tweepy/streaming.py", line 286, in _run
raise
RuntimeError: No active exception to reraise
Do you have an idea why? I want that the steam doesn't stop before I ask him.
I found the solution on this post :
Unraised exception using Tweepy and MySQL
By checking tweepy/streaming.py at https://github.com/tweepy/tweepy/blob/master/tweepy/streaming.py it seems there is a bug in tweepy in the way exceptions are handled, spefically
if exception:
# call a handler first so that the exception can be logged.
self.listener.on_exception(exception)
raise
This raise should be raise exception
That's magic but It work...
Related
Hope you're fine.
I've followed a tutorial on how to create an auto-like Twitter bot using Tweepy. After many errors I've seen that Tweepy has changed some terms.
After making some changes like on except tweepy.TweepError to except tweepy.errors.TweepError. The code doesn't run properly. Was searching everywhere and could not find a solution to my problem.
I've also set on Twitter settings App to Read and Write.
Here is the code. It is pretty straightforward:
import time
auth = tweepy.OAuthHandler("CONSUMER_KEY","CONSUMER_SECRET")
auth.set_access_token("ACCESS_TOKEN","TOKEN_SECRET")
api = tweepy.API(auth, wait_on_rate_limit=True)
search = "#crypto OR #nft"
nrTweets = 500
for tweet in tweepy.Cursor(api.search_tweets, search).items(nrTweets):
try:
print("Tweet Liked")
tweet.favorite()
time.sleep(60)
except tweepy.errors.TweepError as e:
print(e.reason)
except StopIteration:
break
And here the error message. It just not that is not executing the program. It is also the except error is wrong. And I'm following the suggestions that I've seen on tutorial.
Tweet Liked
Traceback (most recent call last):
File "/home/zaesar/Escritorio/coding/python-twitter-bot/twitter.py", line 16, in <module>
tweet.favorite()
File "/home/zaesar/.local/lib/python3.9/site-packages/tweepy/models.py", line 371, in favorite
return self._api.create_favorite(self.id)
File "/home/zaesar/.local/lib/python3.9/site-packages/tweepy/api.py", line 46, in wrapper
return method(*args, **kwargs)
File "/home/zaesar/.local/lib/python3.9/site-packages/tweepy/api.py", line 925, in create_favorite
return self.request(
File "/home/zaesar/.local/lib/python3.9/site-packages/tweepy/api.py", line 257, in request
raise Unauthorized(resp)
tweepy.errors.Unauthorized: 401 Unauthorized
Read-only application cannot POST.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/zaesar/Escritorio/coding/python-twitter-bot/twitter.py", line 18, in <module>
except tweepy.errors.TweepError as e:
AttributeError: module 'tweepy.errors' has no attribute 'TweepError'
Any help will be really appreciated as I'm newbie.
Found the solution!
I made everything fine but after setting to Read and Write on the Twitter App. You need to regenerate the Keys and Tokens. Hope this helps to other people!
I'm using below code to streaming tweets and analyse them for making decisions. while running the below code I got an error. that error occurs twitter users those who had the friend list of more than 50.
import re
import tweepy
import sys
import time
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
non_bmp_map = dict.fromkeys(range(0x10000, sys.maxunicode + 1), 0xfffd)
users = tweepy.Cursor(api.friends, screen_name='#myuser').items()
while True:
try:
user = next(users)
except tweepy.TweepError:
time.sleep(60*15)
user = next(users)
except StopIteration:
break
for status in tweepy.Cursor(api.user_timeline,screen_name=user.screen_name,result_type='recent').items(5):
text=status._json['text'].translate(non_bmp_map)
print (user.screen_name + ' >>>>>> '+text)
while executing this script I have got an error as below.
Traceback (most recent call last):
File "D:sensitive2demo.py", line 31, in <module>
for status in tweepy.Cursor(api.user_timeline,screen_name=user.screen_name,result_type='recent').items(5):
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 49, in __next__
return self.next()
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 197, in next
self.current_page = self.page_iterator.next()
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 108, in next
data = self.method(max_id=self.max_id, parser=RawParser(), *self.args, **self.kargs)
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\binder.py", line 250, in _call
return method.execute()
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\binder.py", line 234, in execute
raise TweepError(error_msg, resp, api_code=api_error_code)
tweepy.error.TweepError: Twitter error response: status code = 401
I have googled a lot.but nothing worked. Can somebody help me to solve the problem?
401 is an http status code for 'Unauthorized'. I would suggest verifying your credentials.
I am currently using tweepy to gather data using Streaming API.
Here is my code and I ran this on Acaconda command prompt. When streaming starts, it returns tweets and then after giving few tweets it gives the following error:
Streaming Started ...
RT #ish10040: Crack Dealer Released Early From Prison By Obama Murders Woman And Her 2 Young Kids… Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\Jae Hee\Anaconda2\lib\threading.py", line 801, in __bootstrap_inner
self.run()
File "C:\Users\Jae Hee\Anaconda2\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Users\Jae Hee\Anaconda2\lib\site-packages\tweepy\streaming.py", line 294, in _run
raise exception
UnicodeEncodeError: 'cp949' codec can't encode character u'\xab' in position 31: illegal multibyte sequence
I believe that it has to do with encoding so I used chcp 65001 to deal with this issue but it does not give the solution!
Here is the code
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
class MyStreamListener(tweepy.StreamListener):
def on_status(self, status):
print(status.text)
def on_error(self, status_code):
#returning False in on_data disconnects the stream
if status_code == 420:
return False
def main():
myStreamListener = MyStreamListener()
myStream = tweepy.Stream(auth = api.auth, listener = myStreamListener)
print "Streaming Started ..."
try:
myStream.filter(track=['Obama'], async = True)
except:
print "error!"
myStream.disconnect()
if __name__ == '__main__':
main()
All text produced and accepted through the twitter API should be encoded as UTF-8, so your code should be using that codec to decode what's coming back.
See here: https://dev.twitter.com/overview/api/counting-characters
I am writing unit tests for a python program using the unittest framework. One of the functions of the program is connecting to an external REST API using the requests library. If there is a connection error or timeout, I would like the function to retry up to 3 times before failing. As such I have written a test which uses a mock (actually httpretty) to replace the external API and raises requests.ConnectionError twice before returning something.
class APITests(unittest.TestCase):
def mock_response(self, uri, body='OK', method=httpretty.GET,
status_code=200, error=None,
error_repeats=None):
"""
Function to register HTTPretty response
"""
responses = []
if error:
def callback(request, uri, headers):
raise error
if error_repeats:
responses += [httpretty.Response(body=callback)]*error_repeats
responses += [httpretty.Response(body=body,
status=status_code)]
else:
responses += [httpretty.Response(body=callback)]
else:
responses += [httpretty.Response(body=body, status=status_code)]
httpretty.register_uri(method, uri, responses=responses)
#httpretty.activate
def test_get_connection_error_then_success_doesnt_raise(self):
self.mock_response(
'http://irrelevant.com',
error=requests.ConnectionError,
error_repeats=2
)
api._get('http://irrelevant.com')
This works fine and the test passes when an exception is not raised no the third attempt but the two exceptions that are raised (intentionally) and caught and handled by the code are printed to the console, polluting the test output. Is there a clean way to stop this happening?
Further info
Here is the method(s) I am testing
def _get(self, url, retries=3):
while retries > 0:
try:
r = requests.get(url)
try:
r.raise_for_status()
return r
except requests.HTTPError as e:
self._handle_HTTP_error(e)
except (requests.ConnectionError, requests.Timeout) as e:
retries -= 1
if not retries:
self._handle_connection_error(e)
def _handle_connection_error(self, error):
raise requests.ConnectionError('Could not connect to API')
The console output is:
Exception in thread Thread-23:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/lukecaldwell/Documents/Repos/other/AidTrends/venv/lib/python2.7/site-packages/httpretty/core.py", line 637, in fill_filekind
headers
File "/Users/lukecaldwell/Documents/Repos/other/AidTrends/aidtrends/tests/test_OECD.py", line 131, in callback
raise error
ConnectionError
I launched this code via terminal via command python py/twi.py and it shows no reaction:
import oauth, tweepy
from time import sleep
message = "hello"
def init():
global api
#confident information
consumer_key = "***"
consumer_secret = "***"
callback_url = "https://twitter.com/Problem196"
access_key="***"
access_secret="***"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret, callback_url)
auth.set_access_token(access_key, access_secret)
api=tweepy.API(auth)
init()
api.update_status(message)
But it supposed to post tweet "hello" on page https://twitter.com/problem196 .
I have updated tweepy, it's fine. Why it's not posting? I have no idea. Please help.
UPD: After I put code print api.last_response.msg terminal showed me some errors:
artem#artem-VirtualBox:~$ python py/twi.py
Traceback (most recent call last):
File "py/twi.py", line 20, in <module>
api.update_status(message)
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.1-py2.7.egg/tweepy/binder.py", line 197, in _call
return method.execute()
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.1-py2.7.egg/tweepy/binder.py", line 154, in execute
raise TweepError('Failed to send request: %s' % e)
tweepy.error.TweepError: Failed to send request: [Errno -2] Name or service not known
artem#artem-VirtualBox:~$ python py/twi.py
Traceback (most recent call last):
File "py/twi.py", line 20, in <module>
api.update_status(message)
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.1-py2.7.egg/tweepy/binder.py", line 197, in _call
return method.execute()
File "/usr/local/lib/python2.7/dist-packages/tweepy-2.1-py2.7.egg/tweepy/binder.py", line 173, in execute
raise TweepError(error_msg, resp)
tweepy.error.TweepError: [{'message': 'Status is a duplicate', 'code': 187}]