We Keep Coding

Twitter API, Tweepy, get_user_time_line bug

I am working on a Twitter bot based on Elon Musk tweets, retrieving his user_timeline() every 1s, I have tested it on other users tweets (users with lower followers) and it works just fine, however for Elon's after a few of correct retrievals it crashes, is that an API issue, or is it something related to Request Limits, please let me know if u have any idea how why that is happening cuz I've been struggling for the past few hours? Thank you for your response.

Sorting Twitter API pages in favorites by time you favorited them not when they were created

Okay, so basically I'm trying to write a program that would go through my twitter favorite history and find pieces of media from them and then download them. I actually have a working model in Python using Tweepy that works it's just it only gets the tweets in order of when they were created. See, the thing is I don't want to download my whole favorite history every time, just up to the point of the last mass download, so at first, I set it up to stop when the tweets reached a certain date. Every time I downloaded I would record the date, but then I realized that sometimes I favorite tweets that are from a while ago, before the cut off date, but is still new to my favorites history. So I decided to try something else, I would record the tweet ID of the first tweet in the list from the last time I downloaded and then set it to stop there, and this would work fine if the api.favorites() returned it in the order it appears on your profile, but instead it auto sorts them by date, so if I fav a post form 2010, I would have to cycle threw all the back to 2010 before it would appear on one of my returned pages. When I looked threw the docs I found a little bit on sorting, but nothing on sorting the Fav Hist by when you faved the tweet. And the thing is I know it stores the order that you liked the tweets in, since that's how it displays it on your profile, even if it is just a table that they append to every time, it still works. I know I'm writing this program in python, but I'm good enough with Java and JavaScript to understand the guts of the API, it's how I got this far. Anyway if you have some suggestions or now how to do it please let me know, any help is appreciated! If all else fails I'll try using Selenium to go through my Twitter Favs from a user perspective...

I had the same issue and I found that you can't do this with the regular Twitter API. They have an enterprise API, which requires payment to access, and that will give you the time a tweet was favorited but it doesn't look cheap so it's probably not worth it.

Most efficient way to check twitter friendship? (over 5000 check)

I'm facing problem like this. I used tweepy to collect +10000 tweets, i use nltk naive-bayes classification and filtered the tweets into +5000.
I want to generate a graph of user friendship from that classified 5000 tweet. The problem is that I am able to check it with tweepy.api.show_frienship(), but it takes so much and much time and sometime ended up with endless ratelimit error.
is there any way i can check the friendship more eficiently?

I don't know much about the limits with Tweepy, but you can always write a basic web scraper with urllib and BeautifulSoup to do so.
You could take a website such as www.doesfollow.com which accomplishes what you are trying to do. (not sure about request limits with this page, but there are dozens of other websites that do the same thing) This website is interesting because the url is super simple.
For example, in order to check if Google and Twitter are "friends" on Twitter, the link is simply www.doesfollow.com/google/twitter.
This would make it very easy for you to run through the users as you can just append the users to the url such as 'www.doesfollow.com/'+ user1 + '/' + user2
The results page of doesfollow has this tag if the users are friends on Twitter:
<div class="yup">yup</div>,
and this tag if the users are not friends on Twitter:
<div class="nope">nope</div>
So you could parse the page source code and search to find which of those tags exist to determine if the users are friends on Twitter.
This might not be the way that you wanted to approach the problem, but it's a possibility. I'm not entirely sure how to approach the graphing part of your question though. I'd have to look into that.

Tweepy not finding results that should be there

I am writing a script in Python, that uses tweepy to search for tweets with a given keyword. Here is the snippet:
for tweet in tweepy.Cursor(api.search, q=keyword, lang="en").items(10):
print tweet.id
I have everything authenticated properly and the code works most of the time. However, when I try to search for some keywords (examples below) it doesn't return anything.
The keywords that cause trouble are "digitalkidz" (a tech conference) and "newtrendbg" (a Bulgarian company). If you do a quick search on Twitter for either of those you will see that there are results. However, tweepy doesn't find anything. Again, it does work for pretty much any other keyword I use.
Do you have any ideas what might be the problem and how to fix it?
Thank you

I believe you're forgetting an important aspect of the twitter api, it's not exhaustive.
Taken from the api docs
Please note that Twitter’s search service and, by extension, the Search API is not meant to be an exhaustive source of Tweets. Not all Tweets will be indexed or made available via the search interface.
Regardless of whether you're using the streaming or rest api, you're going to have issues with this if you're looking for specific tweets.
Rest API
When looking for historical tweets, you unfortunately won't be able to obtain anything that is older than a week using api.search(). This is also shown in the docs.
Keep in mind that the search index has a 7-day limit. In other words, no tweets will be found for a date older than one week.
There are other ways of getting older tweets, this post details those options.
Streaming API
While it doesn't sound like you're using twitter's streaming API, it should be noted that this only gives a small sample of twitter's current tweet traffic (~1-2%).
Hopefully this is helpful. Let me know if you have any questions.

How do I make sure a twitter bot doesn't retweet the same tweet multiple times?

I'm writing a simple Twitter bot in Python and was wondering if anybody could answer and explain the question for me.
I'm able to make Tweets, but I haven't had the bot retweet anyone yet. I'm afraid of tweeting a user's tweet multiple times. I plan to have my bot just run based on Windows Scheduled Tasks, so when the script is run (for example) the 3rd time, how do I get it so the script/bot doesn't retweet a tweet again?
To clarify my question:
Say that someone tweeted at 5:59pm "#computer". Now my twitter bot is supposed to retweet anything containing #computer. Say that when the bot runs at 6:03pm it finds that tweet and retweets it. But then when the bot runs again at 6:09pm it retweets that same tweet again. How do I make sure that it doesn't retweet duplicates?
Should I create a separate text file and add in the IDs of the tweets and read through them every time the bot runs? I haven't been able to find any answers regarding this and don't know an efficient way of checking.

You should store somewhere the timestamp of the latest tweet processed, that way you won't go throught the same tweets twice, hence not retweeting a tweet twice.
This should also make tweet processing faster (because you only process each tweet once).

I wrote a twitter bot in python a few months ago and this link helped a lot. I also used this github repo which although is in Ruby, was quite helpful for logic flow. This repo uses a similar approach to what you mentioned, creating a local datastore of previous retweets to compare against each tweet.

This is how I did it. I grabbed the list of things to retweet and a list of my feed. I cut the lists down to only posts within the past 24 hours. Then for each item in retweetable I check to see if it's in my feed list. If not I post RT #user retweet content.
I also wrote a function to chop the str down to 140 chars (137 + '...')
E.G.
TO_RT = 'a post to post'
MYTWT = ('old post', 'other old post')
if TO_RT not in MYTWT
Tweet(TO_RT)

Twitter is set such that you can't retweet the same thing more than once. So if your bot gets such a tweet, it will be redirected to an Error 403 page by the API. You can test this policy by reducing the time between each run by the script to about a minute; this will generate the Error 403 link as the current feed of tweets remains unchanged.