We Keep Coding

How to delete all favorites using python? Can't delete, getting errors. Need solution

I've tried to make a python script that can delete / undo favorite all my twitter favorites for me. I have seen MATHEW INKSON's post to do the job. I don't need to delete my tweets, just want to clear the favorites. Besides that script is almost two years old and incompatible with latest python. So I've edited a little to run it with python 3.6.0 and my script look like this:
import tweepy
from datetime import datetime, timedelta
test_mode = False
verbose = False
delete_favs = True
days_to_keep = 7
consumer_key = 'my consumer key'
consumer_secret = 'my consumer secret'
access_token = 'my access token'
access_token_secret = 'my access token secret'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
cutoff_date = datetime.utcnow() - timedelta(days=days_to_keep)
if delete_favs:
print ("Retrieving favorited tweets")
favorites = tweepy.Cursor(api.favorites).items()
unfav_count = 0
kept_count = 0
for tweet in favorites:
if tweet.created_at < cutoff_date:
if verbose:
print ("Unfavoring %d: [%s] %s % (tweet.created_at,tweet.text)")
if not test_mode:
api.destroy_favorite(tweet.id)
unfav_count += 1
else:
kept_count += 1
print ("Unfavored %d tweets, ignored %d" % (unfav_count, kept_count))
But, everytime by running the script on my windows command I am getting this following error:
Traceback (most recent call last):
File "C:\Users\xyz\Desktop\New folder\Unfavorite.py", line 25, in <module>
for tweet in favorites:
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 49, in __next__
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 197, in next
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 108, in next
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\binder.py", line 250, in _call
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\binder.py", line 234, in execute
tweepy.error.TweepError: Twitter error response: status code = 429
I am using Python 3.6, my app's permissions are all correct. Everything is fine with my twitter app. I guess something is wrong with my script.
Please someone help fix my code. I've seen some other script too. Those didn't work out. Suggestions will be appreciated.
Thanks is advance.

As per Twitter response codes, Code 429 is Returned in when a request cannot be served due to the application’s rate limit having been exhausted for the resource. Which means your app has made too many requests and you have to look into API Rate limits.

tweepy.TweepError Rate Limit Exceeded

Question
I was writing a Twitter bot with Python an Tweepy. I successfully got the bot to work last night, however, after following 60 people, the bot started throwing an error that says [{u'message': u'Rate Limit Exceeded', u'code': 88}]. I understand that I am only allowed to do a certain amount of calls to the Twitter API and I have found this link that shows how many calls I can make on each of these functions. After reviewing my code, I have found that the error is being thrown where I am saying for follower in tweepy.Cursor(api.followers, me).items():. On the page that I found that says how many requests I get, it says that I get 15 requests for every 15 minutes for getting my followers. I have waited overnight and I retried the code this morning, however, it was still throwing the same error. I don't understand why Tweepy is throwing a rate limit exceeded error whenever I should still have requests left.
Code
Here is my code that's throwing the error.
#!/usr/bin/python
import tweepy, time, pprint
CONSUMER_KEY = ''
CONSUMER_SECRET = ''
ACCESS_KEY = ''
ACCESS_SECRET = ''
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
me = api.me()
pprint.pprint(api.rate_limit_status())
while True:
try:
for follower in tweepy.Cursor(api.followers, me).items():
api.create_friendship(id=follower.id)
for follower in tweepy.Cursor(api.friends, me).items():
for friend in tweepy.Cursor(api.friends, follower.id).items():
if friend.name != me.name:
api.create_friendship(id=friend.id)
except tweepy.TweepError, e:
print "TweepError raised, ignoring and continuing."
print e
continue
I have found the line that throws the error by typing in the interactive prompt
for follower in tweepy.Cursor(api.followers, me).items():
print follower
where it gives me the error
**Traceback (most recent call last):
File "<pyshell#31>", line 1, in <module>
for follower in api.followers(id=me.id):
File "C:\Users\Lane\AppData\Local\Enthought\Canopy\User\lib\site-packages\tweepy\binder.py", line 239, in _call
return method.execute()
File "C:\Users\Lane\AppData\Local\Enthought\Canopy\User\lib\site-packages\tweepy\binder.py", line 223, in execute
raise TweepError(error_msg, resp)
TweepError: [{u'message': u'Rate limit exceeded', u'code': 88}]**

API().followers method is actually GET followers/list, which is limited to 15 requests per window (before next epoch). Each call returns 20 users list by default, and if you are reaching limit exceed error, I'm sure authenticated user has more than 300 followers.
The solution is to increase the length of user's list received during each call, such that within 15 calls it can fetch all the users. Modify your code as following
for follower in tweepy.Cursor(api.followers, id = me.id, count = 50).items():
print follower
count denotes the number of users to be fetched per request.
Maximum value of count can be 200, it means within 15 minute, you can only fetch 200 x 15 = 3000 followers, which is enough in general scenarios.

"IncompleteRead" Error when retrieving Twitter Data using Python

While running this program to retrieve Twitter data using Python 2.7.8 :
#imports
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
#setting up the keys
consumer_key = '…………...'
consumer_secret = '………...'
access_token = '…………...'
access_secret = '……………..'
class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
#This is a basic listener that just prints received tweets to standard output
def on_data(self, data):
print (data)
return True
def on_error(self, status):
print (status)
#printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
stream = Stream(auth, TweetListener())
t = u"سوريا"
stream.filter(track=[t])
after running this program for 5 hours i got this Error message:
Traceback (most recent call last):
File "/Users/Mona/Desktop/twitter.py", line 32, in <module>
stream.filter(track=[t])
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 316, in filter
self._start(async)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 237, in _start
self._run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 173, in _run
self._read_loop(resp)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 225, in _read_loop
next_status_obj = resp.read( int(delimited_string) )
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 543, in read
return self._read_chunked(amt)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 612, in _read_chunked
value.append(self._safe_read(chunk_left))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 660, in _safe_read
raise IncompleteRead(''.join(s), amt)
IncompleteRead: IncompleteRead(0 bytes read, 976 more expected)
>>>
Actually i don't know what to do with this problem !!!

You should check to see if you're failing to process tweets quickly enough using the stall_warnings parameter.
stream.filter(track=[t], stall_warnings=True)
These messages are handled by Tweepy (check out implementation here) and will inform you if you're falling behind. Falling behind means that you're unable to process tweets as quickly as the Twitter API is sending them to you. From the Twitter docs:
Setting this parameter to the string true will cause periodic messages to be delivered if the client is in danger of being disconnected. These messages are only sent when the client is falling behind, and will occur at a maximum rate of about once every 5 minutes.
In theory, you should receive a disconnect message from the API in this situation. However, that is not always the case:
The streaming API will attempt to deliver a message indicating why a stream was closed. Note that if the disconnect was due to network issues or a client reading too slowly, it is possible that this message will not be received.
The IncompleteRead could also be due to a temporary network issue and may never happen again. If it happens reproducibly after about 5 hours though, falling behind is a pretty good bet.

I've just had this problem. The other answer is factually correct, in that it's almost certainly:
Your program isn't keeping up with the stream
you get a stall warning if that's the case.
In my case, I was reading the tweets into postgres for later analysis, across a fairly dense geographic area, as well as keywords (London, in fact, and about 100 keywords). It's quite possible that, even though you're just printing it, your local machine is doing a bunch of other things, and system processes get priority, so the tweets will back up until Twitter disconnects you. (This is typically manifests as an apparent memory leak - the program increases in size until it gets killed, or twitter disconnects - whichever is first.)
The thing that made sense here was to push off the processing to a queue. So, I used a redis and django-rq solution - it took about 3 hours to implement on dev and then my production server, including researching, installing, rejigging existing code, being stupid about my installation, testing, and misspelling things as I went.
Install redis on your machine
Start the redis server
Install Django-RQ (or just Install RQ if you're working solely in python)
Now, in your django directory (where appropriate - ymmv for straight python applications) run:
python manage.py rqworker &
You now have a queue! You can add jobs to that like by changing your handler like this:
(At top of file)
import django_rq
Then in your handler section:
def on_data(self, data):
django_rq.enqueue(print, data)
return True
As an aside - if you're interested in stuff emanating from Syria, rather than just mentioning Syria, then you could add to the filter like this:
stream.filter(track=[t], locations=[35.6626, 32.7930, 42.4302, 37.2182]
That's a very rough geobox centred on Syria, but which will pick up bits of Iraq/Turkey around the edges. Since this is an optional extra, it's worth pointing this out:
Bounding boxes do not act as filters for other filter parameters. For
example track=twitter&locations=-122.75,36.8,-121.75,37.8 would match
any tweets containing the term Twitter (even non-geo tweets) OR coming
from the San Francisco area.
From this answer, which helped me, and the twitter docs.
Edit: I see from your subsequent posts that you're still going down the road of using Twitter API, so hopefully you got this sorted anyway, but hopefully this will be useful for someone else! :)

This worked for me.
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
while True:
try:
stream.filter(track=['python', 'java'], stall_warnings=True)
except (ProtocolError, AttributeError):
continue

A solution is restarting the stream immediately after catching exception.
# imports
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
# setting up the keys
consumer_key = "XXXXX"
consumer_secret = "XXXXX"
access_token = "XXXXXX"
access_secret = "XXXXX"
# printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
# This is a basic listener that just prints received tweets to standard output
def on_data(self, data):
print(data)
return True
def on_exception(self, exception):
print('exception', exception)
start_stream()
def on_error(self, status):
print(status)
def start_stream():
stream = Stream(auth, TweetListener())
t = u"سوريا"
stream.filter(track=[t])
start_stream()

For me the back end application to which the URL is pointing is directly returning the string
I changed it to
return Response(response=original_message, status=200, content_type='application/text')
in the start I just returned text like
return original_message
I think this answer works only for my case

How can I retrieve all Tweets and attributes for a given user using Python?

I am attempting to retrieve data from Twitter, using Tweepy for a username typed at the command line. I'm wanting to extract quite a bit of data about the status and user,so have come up with the following:
Note that I am importing all the required modules ok and have oauth + keys (just not included it here) and filename is correct, just been changed:
# define user to get tweets for. accepts input from user
user = tweepy.api.get_user(input("Please enter the twitter username: "))
# Display basic details for twitter user name
print (" ")
print ("Basic information for", user.name)
print ("Screen Name:", user.screen_name)
print ("Name: ", user.name)
print ("Twitter Unique ID: ", user.id)
print ("Account created at: ", user.created_at)
timeline = api.user_timeline(screen_name=user, include_rts=True, count=100)
for tweet in timeline:
print ("ID:", tweet.id)
print ("User ID:", tweet.user.id)
print ("Text:", tweet.text)
print ("Created:", tweet.created_at)
print ("Geo:", tweet.geo)
print ("Contributors:", tweet.contributors)
print ("Coordinates:", tweet.coordinates)
print ("Favorited:", tweet.favorited)
print ("In reply to screen name:", tweet.in_reply_to_screen_name)
print ("In reply to status ID:", tweet.in_reply_to_status_id)
print ("In reply to status ID str:", tweet.in_reply_to_status_id_str)
print ("In reply to user ID:", tweet.in_reply_to_user_id)
print ("In reply to user ID str:", tweet.in_reply_to_user_id_str)
print ("Place:", tweet.place)
print ("Retweeted:", tweet.retweeted)
print ("Retweet count:", tweet.retweet_count)
print ("Source:", tweet.source)
print ("Truncated:", tweet.truncated)
I would like this eventually to iterate through all of a user's tweets (up to the 3200 limit). First things first though. So far though I have two problems, I get the following error message regarding retweets:
Please enter the twitter username: barackobamaTraceback (most recent call last):
File " usertimeline.py", line 64, in <module>
timeline = api.user_timeline(screen_name=user, count=100, page=1)
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 401
Traceback (most recent call last):
File "usertimeline.py", line 42, in <module>
user = tweepy.api.get_user(input("Please enter the twitter username: "))
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 404
Passing the username as a variable seems to be a problem also:
Traceback (most recent call last):
File " usertimleline.py", line 64, in <module>
timeline = api.user_timeline(screen_name=user, count=100, page=1)
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 401
I've isolated both these errors, i.e. they aren't working together.
Forgive my ignorance, I am not too hot with Twitter APIs but am learning pretty rapidly. Tweepy documentation really does suck and I've done loads of reading round on the net, just can't seem to get this fixed. If I can get this sorted, i'll be posting up some documentation.
I know how to transfer the data into an MySQL db once extracted (it will do that, rather than print to screen) and manipulate it so that I can do stuff with it, it is just getting it out that I am having the problems with. Does anyone have any ideas or is there another method I should be considering?
Any help really appreciated. Cheers
EDIT:
Following on from #Eric Olson's suggestion this morning; I did the following.
1) Created a completely brand new set of Oauth credentials to test.
2) Copied code across to a new script as follows:
Oauth
consumer_key = "(removed)"
consumer_secret = "(removed)"
access_key="88394805-(removed)"
access_secret="(removed)"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api=tweepy.API(auth)
# confirm account being used for OAuth
print ("API NAME IS: ", api.me().name)
api.update_status("Using Tweepy from the command line")
The first time i run the script, it works fine and updates my status and returns the API name as follows:
>>>
API NAME IS: Chris Howden
Then from that point on I get this:
Traceback (most recent call last):
File "C:/Users/Chris/Dropbox/Uni_2012-3/6CC995 - Independent Studies/Scripts/get Api name and update status.py", line 19, in <module>
api.update_status("Using Tweepy frm the command line")
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 403
The only reason I can see for it doing something like this is that it is rejecting the generated access token. I shouldn't need to renew the access token should I?

If you're open to trying another library, you could give rauth a shot. There's already a Twitter example but if you're feeling lazy and just want a working example, here's how I'd modify that demo script:
from rauth import OAuth1Service
# Get a real consumer key & secret from https://dev.twitter.com/apps/new
twitter = OAuth1Service(
name='twitter',
consumer_key='J8MoJG4bQ9gcmGh8H7XhMg',
consumer_secret='7WAscbSy65GmiVOvMU5EBYn5z80fhQkcFWSLMJJu4',
request_token_url='https://api.twitter.com/oauth/request_token',
access_token_url='https://api.twitter.com/oauth/access_token',
authorize_url='https://api.twitter.com/oauth/authorize',
base_url='https://api.twitter.com/1/')
request_token, request_token_secret = twitter.get_request_token()
authorize_url = twitter.get_authorize_url(request_token)
print 'Visit this URL in your browser: ' + authorize_url
pin = raw_input('Enter PIN from browser: ')
session = twitter.get_auth_session(request_token,
request_token_secret,
method='POST',
data={'oauth_verifier': pin})
params = {'screen_name': 'github', # User to pull Tweets from
'include_rts': 1, # Include retweets
'count': 10} # 10 tweets
r = session.get('statuses/user_timeline.json', params=params)
for i, tweet in enumerate(r.json(), 1):
handle = tweet['user']['screen_name'].encode('utf-8')
text = tweet['text'].encode('utf-8')
print '{0}. #{1} - {2}'.format(i, handle, text)
You can run this as-is, but be sure to update the credentials! These are meant for demo purposes only.
Full disclosure, I am the maintainer of rauth.

You're getting 401 response, which means "Unauthorized." (see HTTP status codes)
Your code looks good. Using api.user_timeline(screen_name="some_screen_name") works for me in the old example I have lying around.
I'm guessing you either need to authorize the app, or there is some problem with your OAuth setup.
Maybe you found this already, but here is the short code example that I started from: https://github.com/nloadholtes/tweepy/blob/nloadholtes-examples/examples/oauth.py

How to use refresh_access_token in the Yahoo Social Python SDK

I'm trying to use the Yahoo Social Python SDK to get a users contacts through oAuth. This is for a webapp running on App Engine. SO, I have everything set up to run through the oAuth dance, exchanging consumer keys and verifiers and all that jazz. I store the token and can reuse it to retrieve a users contacts until the token the expires an hour later. So, is there anyone out there who has used the Python SDK and can tell me what is wrong with this simple code:
import yahoo.application
CONSUMER_KEY = '####'
CONSUMER_SECRET = '##'
APPLICATION_ID = '##'
CALLBACK_URL = '##'
oauthapp = yahoo.application.OAuthApplication(CONSUMER_KEY, CONSUMER_SECRET, APPLICATION_ID, CALLBACK_URL)
oauthapp.token = yahoo.oauth.AccessToken.from_string(access_token) #access_token is legit string pulled from datastore
oauthapp.token = oauthapp.refresh_access_token(oauthapp.token)
contacts = oauthapp.getContacts()
Running this throws the following error:
'oauth_token'<br>
Traceback (most recent call last):<br>
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 513, in __call__<br>
handler.post(*groups)<br>
File "/base/data/home/apps/testproj/2.345270664321958961/scripteditor.py", line 1249, in post<br>
oauthapp.token = oauthapp.refresh_access_token(oauthapp.token)<br>
File "/base/data/home/apps/testproj/2.345270664321958961/yahoo/application.py", line 90, in refresh_access_token<br>
self.token = self.client.fetch_access_token(request)<br>
File "/base/data/home/apps/testproj/2.345270664321958961/yahoo/oauth.py", line 165, in fetch_access_token<br>
return AccessToken.from_string(self.connection.getresponse().read().strip())<br>
File "/base/data/home/apps/testproj/2.345270664321958961/yahoo/oauth.py", line 130, in from_string<br>
key = params['oauth_token'][0]<br>
KeyError: 'oauth_token'<br>
Basically, if I comment out the line with refresh_access_token, and the token has not expired, this code works and I get the users contacts. But with refresh_acces_token, it fails at that line. Can anyone give a hand?

Looks like something wrong with passing params. Try to debug oauth_token variable.

Solved. For reasons I can't understand, the above code now just works. It might have been a problem on yahoo's end, but I really can't be sure. It's been running fine for two weeks.