We Keep Coding

I made a Twitter bot that automatically tweets Direct Messages:

I recently started to learn python, i had 0 knowledge on pyhton, and in the last few weeks i've been studying python and the twitter api.
I decided to work on a simple twitter bot, that automatically posts whatever people send on my direct messages, and i maneged to do so.
Here's my code:
import tweepy
import json
import sys
import re
import time
print("acessing api...")
consumer_key = 'consumer_key'
consumer_secret = 'consumer_secret'
key = 'key'
secret = 'key_secret'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(key, secret)
api = tweepy.API(auth, wait_on_rate_limit = True)
print('api accessed')
print('colecting dms')
direct_message = api.list_direct_messages()
text = direct_message[0].message_create["message_data"]["text"]
def send_tweet():
print('sending tweet...')
api.update_status('/bot: ' + text)
while True:
send_tweet()
time.sleep(30)
The code works, but not 100%, i'm able to extract the text from the Dm and use it on the api.update_status("/bot: " + text) but when it loops i get the following error:
tweepy.error.TweepError: [{'code': 187, 'message': 'Status is a duplicate.'}]
i've looked it up, and tried many things, for example:
while True:
try:
send_tweet()
time.sleep(30)
except tweepy.TweepError as error:
if error.api_code == 187:
print('duplicate message')
else:
print('waiting for new message')
time.sleep(30)
All i want is a way to keep the code going and ignore messages and tweets that were already sent, also wait for new dm's
Another thing that kept happening was that when i tried testing the extraction of the text, i kept getting the same text from the same dm, instead of getting a nother newer one.

Direct Messages stay in the Inbox even after your read or reply. It is best to delete a DM after you process it (in your case tweet it) so you don't need to worry about it next time you fetch the messages.

How to delete all favorites using python? Can't delete, getting errors. Need solution

I've tried to make a python script that can delete / undo favorite all my twitter favorites for me. I have seen MATHEW INKSON's post to do the job. I don't need to delete my tweets, just want to clear the favorites. Besides that script is almost two years old and incompatible with latest python. So I've edited a little to run it with python 3.6.0 and my script look like this:
import tweepy
from datetime import datetime, timedelta
test_mode = False
verbose = False
delete_favs = True
days_to_keep = 7
consumer_key = 'my consumer key'
consumer_secret = 'my consumer secret'
access_token = 'my access token'
access_token_secret = 'my access token secret'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
cutoff_date = datetime.utcnow() - timedelta(days=days_to_keep)
if delete_favs:
print ("Retrieving favorited tweets")
favorites = tweepy.Cursor(api.favorites).items()
unfav_count = 0
kept_count = 0
for tweet in favorites:
if tweet.created_at < cutoff_date:
if verbose:
print ("Unfavoring %d: [%s] %s % (tweet.created_at,tweet.text)")
if not test_mode:
api.destroy_favorite(tweet.id)
unfav_count += 1
else:
kept_count += 1
print ("Unfavored %d tweets, ignored %d" % (unfav_count, kept_count))
But, everytime by running the script on my windows command I am getting this following error:
Traceback (most recent call last):
File "C:\Users\xyz\Desktop\New folder\Unfavorite.py", line 25, in <module>
for tweet in favorites:
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 49, in __next__
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 197, in next
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\cursor.py", line 108, in next
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\binder.py", line 250, in _call
File "C:\Users\xyz\AppData\Local\Programs\Python\Python36\lib\site-packages\tweepy-3.6.0-py3.6.egg\tweepy\binder.py", line 234, in execute
tweepy.error.TweepError: Twitter error response: status code = 429
I am using Python 3.6, my app's permissions are all correct. Everything is fine with my twitter app. I guess something is wrong with my script.
Please someone help fix my code. I've seen some other script too. Those didn't work out. Suggestions will be appreciated.
Thanks is advance.

As per Twitter response codes, Code 429 is Returned in when a request cannot be served due to the application’s rate limit having been exhausted for the resource. Which means your app has made too many requests and you have to look into API Rate limits.

How to break a while loop when reached limits on Twitter's API?

I am coding a program to get the followers from a given user, then use the follower's list to get their followers and so on. The code I have so far is as I show below:
import tweepy
import time
#insert your Twitter keys here
consumer_key ='key'
consumer_secret='secret'
access_token='accesstoken'
access_secret='accesssecret'
auth = tweepy.auth.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
list= open('/home/acrocephalus/GitHub/TweetViz/list.txt','w')
if(api.verify_credentials):
print '-------------------------\n*** You are logged in ***\n-------------------------'
#Set starting Twitter handle
username = ['moixera']
user = tweepy.Cursor(api.followers, screen_name=username).items()
#Set the number of levels to follow
depth=3
#Start extracting each level followers
while depth != 0:
for handle in username:
print '\n\n Getting followers from: #' + handle+'\n\n'
user = tweepy.Cursor(api.followers, screen_name=handle).items()
while True:
try:
u = next(user)
list.write(u.screen_name +'\n')
print u.screen_name
except:
time.sleep(15*60)
print 'We have exceeded the rate limit. Sleeping for 15 minutes'
u = next(user)
list.write(u.screen_name +'\n')
print u.screen_name
username = list.read().splitlines()
print 'Moving to next username'
depth = depth-1
list.close()
The problem is that it starts with the first user, gets her followers but doesn't continue with her followers list. I think that the problem is in the while loop. When it finishes getting the followers it jumps to the except part. The desired behaviour would be that when it has finished retrieving followers it jumps to the beginning of the for loop. The program should jumpt to the except part of the while loop when it reaches the Twitter's API hit limit and thus times out for 15 minutes. Can anyone help?
Cheers!
Dani

Use a for loop instead of the while loop:
user_list = open('/home/acrocephalus/GitHub/TweetViz/list.txt','w')
for user in tweepy.Cursor(api.followers, screen_name=handle).items():
user_list.write(user.screen_name +'\n')
print user.screen_name
N.B. don't use list as a variable name because it hides the list builtin.
I think that the API has some support for rate limiting, although I don't see it detailed in the documentation. You can enable it when initialising with tweepy.API(), see wait_on_rate_limit and wait_on_rate_limit_notify:
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)
A very quick glance at the source code suggests that the API will figure out an appropriate waiting period based on headers returned from Twitter, e.g. x-rate-limit-reset, but I have not used this API so I can't be sure whether it works.
There are other problems with your code, however, these go beyond your question.

Avoid Twitter API limitation with Tweepy

I saw in some question on Stack Exchange that the limitation can be a function of the number of requests per 15 minutes and depends also on the complexity of the algorithm, except that this is not a complex one.
So I use this code:
import tweepy
import sqlite3
import time
db = sqlite3.connect('data/MyDB.db')
# Get a cursor object
cursor = db.cursor()
cursor.execute('''CREATE TABLE IF NOT EXISTS MyTable(id INTEGER PRIMARY KEY, name TEXT, geo TEXT, image TEXT, source TEXT, timestamp TEXT, text TEXT, rt INTEGER)''')
db.commit()
consumer_key = ""
consumer_secret = ""
key = ""
secret = ""
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(key, secret)
api = tweepy.API(auth)
search = "#MyHashtag"
for tweet in tweepy.Cursor(api.search,
q=search,
include_entities=True).items():
while True:
try:
cursor.execute('''INSERT INTO MyTable(name, geo, image, source, timestamp, text, rt) VALUES(?,?,?,?,?,?,?)''',(tweet.user.screen_name, str(tweet.geo), tweet.user.profile_image_url, tweet.source, tweet.created_at, tweet.text, tweet.retweet_count))
except tweepy.TweepError:
time.sleep(60 * 15)
continue
break
db.commit()
db.close()
I always get the Twitter limitation error:
Traceback (most recent call last):
File "stream.py", line 25, in <module>
include_entities=True).items():
File "/usr/local/lib/python2.7/dist-packages/tweepy/cursor.py", line 153, in next
self.current_page = self.page_iterator.next()
File "/usr/local/lib/python2.7/dist-packages/tweepy/cursor.py", line 98, in next
data = self.method(max_id = max_id, *self.args, **self.kargs)
File "/usr/local/lib/python2.7/dist-packages/tweepy/binder.py", line 200, in _call
return method.execute()
File "/usr/local/lib/python2.7/dist-packages/tweepy/binder.py", line 176, in execute
raise TweepError(error_msg, resp)
tweepy.error.TweepError: [{'message': 'Rate limit exceeded', 'code': 88}]

For anyone who stumbles upon this on Google, tweepy 3.2+ has additional parameters for the tweepy.api class, in particular:
wait_on_rate_limit – Whether or not to automatically wait for rate limits to replenish
wait_on_rate_limit_notify – Whether or not to print a notification when Tweepy is waiting for rate limits to replenish
Setting these flags to True will delegate the waiting to the API instance, which is good enough for most simple use cases.

The problem is that your try: except: block is in the wrong place. Inserting data into the database will never raise a TweepError - it's iterating over Cursor.items() that will. I would suggest refactoring your code to call the next method of Cursor.items() in an infinite loop. That call should be placed in the try: except: block, as it can raise an error.
Here's (roughly) what the code should look like:
# above omitted for brevity
c = tweepy.Cursor(api.search,
q=search,
include_entities=True).items()
while True:
try:
tweet = c.next()
# Insert into db
except tweepy.TweepError:
time.sleep(60 * 15)
continue
except StopIteration:
break
This works because when Tweepy raises a TweepError, it hasn't updated any of the cursor data. The next time it makes the request, it will use the same parameters as the request which triggered the rate limit, effectively repeating it until it goes though.

Just replace
api = tweepy.API(auth)
with
api = tweepy.API(auth, wait_on_rate_limit=True)

If you want to avoid errors and respect the rate limit you can use the following function which takes your api object as an argument. It retrieves the number of remaining requests of the same type as the last request and waits until the rate limit has been reset if desired.
def test_rate_limit(api, wait=True, buffer=.1):
"""
Tests whether the rate limit of the last request has been reached.
:param api: The `tweepy` api instance.
:param wait: A flag indicating whether to wait for the rate limit reset
if the rate limit has been reached.
:param buffer: A buffer time in seconds that is added on to the waiting
time as an extra safety margin.
:return: True if it is ok to proceed with the next request. False otherwise.
"""
#Get the number of remaining requests
remaining = int(api.last_response.getheader('x-rate-limit-remaining'))
#Check if we have reached the limit
if remaining == 0:
limit = int(api.last_response.getheader('x-rate-limit-limit'))
reset = int(api.last_response.getheader('x-rate-limit-reset'))
#Parse the UTC time
reset = datetime.fromtimestamp(reset)
#Let the user know we have reached the rate limit
print "0 of {} requests remaining until {}.".format(limit, reset)
if wait:
#Determine the delay and sleep
delay = (reset - datetime.now()).total_seconds() + buffer
print "Sleeping for {}s...".format(delay)
sleep(delay)
#We have waited for the rate limit reset. OK to proceed.
return True
else:
#We have reached the rate limit. The user needs to handle the rate limit manually.
return False
#We have not reached the rate limit
return True

import tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
# will notify user on ratelimit and will wait by it self no need of sleep.
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)

I suggest you to use the new api v2 and use the Client obj with the flag wait_on_rate_limit=True the v1 will be deprecated asap
client = tweepy.Client(consumer_key=auth.consumer_key, consumer_secret=auth.consumer_secret, access_token_secret=auth.access_token_secret, access_token=auth.access_token,
bearer_token=twitter_bearer_token, wait_on_rate_limit=True)
It will be all automatic

How can I retrieve all Tweets and attributes for a given user using Python?

I am attempting to retrieve data from Twitter, using Tweepy for a username typed at the command line. I'm wanting to extract quite a bit of data about the status and user,so have come up with the following:
Note that I am importing all the required modules ok and have oauth + keys (just not included it here) and filename is correct, just been changed:
# define user to get tweets for. accepts input from user
user = tweepy.api.get_user(input("Please enter the twitter username: "))
# Display basic details for twitter user name
print (" ")
print ("Basic information for", user.name)
print ("Screen Name:", user.screen_name)
print ("Name: ", user.name)
print ("Twitter Unique ID: ", user.id)
print ("Account created at: ", user.created_at)
timeline = api.user_timeline(screen_name=user, include_rts=True, count=100)
for tweet in timeline:
print ("ID:", tweet.id)
print ("User ID:", tweet.user.id)
print ("Text:", tweet.text)
print ("Created:", tweet.created_at)
print ("Geo:", tweet.geo)
print ("Contributors:", tweet.contributors)
print ("Coordinates:", tweet.coordinates)
print ("Favorited:", tweet.favorited)
print ("In reply to screen name:", tweet.in_reply_to_screen_name)
print ("In reply to status ID:", tweet.in_reply_to_status_id)
print ("In reply to status ID str:", tweet.in_reply_to_status_id_str)
print ("In reply to user ID:", tweet.in_reply_to_user_id)
print ("In reply to user ID str:", tweet.in_reply_to_user_id_str)
print ("Place:", tweet.place)
print ("Retweeted:", tweet.retweeted)
print ("Retweet count:", tweet.retweet_count)
print ("Source:", tweet.source)
print ("Truncated:", tweet.truncated)
I would like this eventually to iterate through all of a user's tweets (up to the 3200 limit). First things first though. So far though I have two problems, I get the following error message regarding retweets:
Please enter the twitter username: barackobamaTraceback (most recent call last):
File " usertimeline.py", line 64, in <module>
timeline = api.user_timeline(screen_name=user, count=100, page=1)
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 401
Traceback (most recent call last):
File "usertimeline.py", line 42, in <module>
user = tweepy.api.get_user(input("Please enter the twitter username: "))
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 404
Passing the username as a variable seems to be a problem also:
Traceback (most recent call last):
File " usertimleline.py", line 64, in <module>
timeline = api.user_timeline(screen_name=user, count=100, page=1)
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 401
I've isolated both these errors, i.e. they aren't working together.
Forgive my ignorance, I am not too hot with Twitter APIs but am learning pretty rapidly. Tweepy documentation really does suck and I've done loads of reading round on the net, just can't seem to get this fixed. If I can get this sorted, i'll be posting up some documentation.
I know how to transfer the data into an MySQL db once extracted (it will do that, rather than print to screen) and manipulate it so that I can do stuff with it, it is just getting it out that I am having the problems with. Does anyone have any ideas or is there another method I should be considering?
Any help really appreciated. Cheers
EDIT:
Following on from #Eric Olson's suggestion this morning; I did the following.
1) Created a completely brand new set of Oauth credentials to test.
2) Copied code across to a new script as follows:
Oauth
consumer_key = "(removed)"
consumer_secret = "(removed)"
access_key="88394805-(removed)"
access_secret="(removed)"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api=tweepy.API(auth)
# confirm account being used for OAuth
print ("API NAME IS: ", api.me().name)
api.update_status("Using Tweepy from the command line")
The first time i run the script, it works fine and updates my status and returns the API name as follows:
>>>
API NAME IS: Chris Howden
Then from that point on I get this:
Traceback (most recent call last):
File "C:/Users/Chris/Dropbox/Uni_2012-3/6CC995 - Independent Studies/Scripts/get Api name and update status.py", line 19, in <module>
api.update_status("Using Tweepy frm the command line")
File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 403
The only reason I can see for it doing something like this is that it is rejecting the generated access token. I shouldn't need to renew the access token should I?

If you're open to trying another library, you could give rauth a shot. There's already a Twitter example but if you're feeling lazy and just want a working example, here's how I'd modify that demo script:
from rauth import OAuth1Service
# Get a real consumer key & secret from https://dev.twitter.com/apps/new
twitter = OAuth1Service(
name='twitter',
consumer_key='J8MoJG4bQ9gcmGh8H7XhMg',
consumer_secret='7WAscbSy65GmiVOvMU5EBYn5z80fhQkcFWSLMJJu4',
request_token_url='https://api.twitter.com/oauth/request_token',
access_token_url='https://api.twitter.com/oauth/access_token',
authorize_url='https://api.twitter.com/oauth/authorize',
base_url='https://api.twitter.com/1/')
request_token, request_token_secret = twitter.get_request_token()
authorize_url = twitter.get_authorize_url(request_token)
print 'Visit this URL in your browser: ' + authorize_url
pin = raw_input('Enter PIN from browser: ')
session = twitter.get_auth_session(request_token,
request_token_secret,
method='POST',
data={'oauth_verifier': pin})
params = {'screen_name': 'github', # User to pull Tweets from
'include_rts': 1, # Include retweets
'count': 10} # 10 tweets
r = session.get('statuses/user_timeline.json', params=params)
for i, tweet in enumerate(r.json(), 1):
handle = tweet['user']['screen_name'].encode('utf-8')
text = tweet['text'].encode('utf-8')
print '{0}. #{1} - {2}'.format(i, handle, text)
You can run this as-is, but be sure to update the credentials! These are meant for demo purposes only.
Full disclosure, I am the maintainer of rauth.

You're getting 401 response, which means "Unauthorized." (see HTTP status codes)
Your code looks good. Using api.user_timeline(screen_name="some_screen_name") works for me in the old example I have lying around.
I'm guessing you either need to authorize the app, or there is some problem with your OAuth setup.
Maybe you found this already, but here is the short code example that I started from: https://github.com/nloadholtes/tweepy/blob/nloadholtes-examples/examples/oauth.py