I am trying to get the tweets from my saved searches using tweepy with python. I am not super great with Python, but the documentation seems pretty good, however, it is just returning empty lists. I know there are results for the searches because I can load them up in a browser. However, tweepy doesn't seem to see them.
import tweepy
CONSUMER_KEY = ''
CONSUMER_SECRET = ''
ACCESS_TOKEN = ''
ACCESS_TOKEN_SECRET = ''
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
saved_searches = api.saved_searches()
for search in saved_searches:
saved_search = api.get_saved_search(search.id)
results = api.search(q=str(saved_search.query))
for result in results:
print(result.text)
Can anyone help me figure out what I am missing here?
Related
I am trying to gather the tweets of a user navalny, from 01.11.2017 to 31.01.2018 using tweepy. I have ids of the first and last tweets that I need, so I tried the following code:
import tweepy
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
t = api.user_timeline(screen_name='navalny', since_id = 933000445307518976, max_id = 936533580481814529)
However, the returned value is an empty list.
What is the problem here?
Are there any restrictions on the history of tweets that I can get?
What are possible solutions?
Quick answer:
Using Tweepy you can only retrieve the last 3200 tweets from the Twitter REST API for a given user.
Unfortunately the tweets you are trying to access are older than this.
Detailed answer:
I did a check using the code below:
import tweepy
from tweepy import OAuthHandler
def tweet_check(user):
"""
Scrapes a users most recent tweets
"""
# API keys and initial configuration
consumer_key = ""
consumer_secret = ""
access_token = ""
access_secret = ""
# Configure authentication
authorisation = OAuthHandler(consumer_key, consumer_secret)
authorisation.set_access_token(access_token, access_secret)
api = tweepy.API(authorisation)
# Requests most recent tweets from a users timeline
tweets = api.user_timeline(screen_name=user, count=2,
max_id=936533580481814529)
for tweet in tweets:
tid = tweet.id
print(tid)
twitter_users = ["#navalny"]
for twitter_user in twitter_users:
tweet_check(twitter_user)
This test returns nothing before 936533580481814529
Using a seperate script I scraped all 3200 tweets, the max Twitter will let you scrape and the youngest tweet id I can find is 943856915536326662
Seems like you have run into Twitter's tweet scraping limit for user timelines here.
I'm currently using the GET statuses/user_timeline twitter API in python to retrieve tweets, It says in the docs This method can only return up to 3,200 of a user’s most recent Tweets however when i run it, it only returns 100-150.
How do i get it to return more tweets than it is already?
Thanks in advance
You'll have to write code to work with timelines. Twitter's Working with timelines documentation has a discussion of how and why Twitter timelines work the way they do.
Essentially, set your count to the maximum amount (200) and use since_id and max_id to manage reading each page. Alternatively, you can use an existing library to make the task much easier. Here's an example, using the tweepy library.
consumer_key = "<your consumer_key>"
consumer_secret = "<your consumer_secret>"
access_token = "<your access_token>"
access_token_secret = "<your access_token_secret>"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
for status in tweepy.Cursor(api.user_timeline, "JoeMayo").items():
print('status_id: {}, text: {}'.format(status.id, status.text.encode('utf-8')))
So I am a complete beginner to Tweepy, and I was trying to get started from their tutorial with the following code:
import tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
public_tweets = api.home_timeline()
for tweet in public_tweets:
print tweet.text
Before the OAuthHandler, I added the consumer_key, consumer_secret, etc as something like:
consumer_key = 'aslkjfdalskdjflsjlfsakdjflasjdkf'
consumer_secret = 'ldsjlfksajldjflasjdljflasjdf'
access_token = 'asldkfjasjdfalsjdflksajkfdlasd'
acess_token_secret = 'alskjdflksajdlfjsalfd'
Of course, with the proper values. however, when I run the whole example, I get nothing printed out. What am I doing wrong? I understand I am supposed to get a list of tweets back. Am I not supposed to pass the values in as a string? I tried looking for tweepy documentation regarding OAuthHandler but didn't really come up with anything.
Any help would be greatly appreciated, thanks!!
For anyone else that might have this problem, it was a very simple issue– I created a new account and didn't have anything in my timeline. Naturally, nothing was printing.
I have been trying to use either Tweepy or Twython with the Twitter API to search for a specific hashtag, extract usernames of users tweeting with the hashtage, and then see how many of those users follow one another. My ultimate goal is to then visualize the connections with NetworkX.
So far, I have been able to search for the hashtag and get a list of users tweeting with it. However, I can't figure out how to see who is following whom on that list. I finally got a friendship lookup to work, but then realized that that parameter only searches friends of the authenticated user (me).
Here is the latest version of the code:
from twython import Twython
import tweepy
# fill these in from Twitter API Dev
CONSUMER_KEY = ''
CONSUMER_SECRET = ''
ACCESS_KEY = ''
ACCESS_SECRET = ''
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
twitter = Twython(CONSUMER_KEY, CONSUMER_SECRET, ACCESS_KEY, ACCESS_SECRET)
# Search for hashtag, limit number of users
try:
search_results = twitter.search(q='energy', count=5)
except TwythonError as e:
print e
test5 = []
for tweet in search_results['statuses']:
if tweet['user']['screen_name'] not in test5:
test5.append((tweet['user']['screen_name']).encode('utf-8'))
print test5
# Lookup friendships
relationships = api.lookup_friendships(screen_names=test5[0:5])
for relationship in relationships:
if relationship.is_following:
print("User is following", relationship.screen_name)
Thanks!
With Tweepy, you can check if user_a follows user_b using the API.exists_friendship method. The code would look something like:
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
is_following = api.exists_friendship(user_a, user_b)
You can specify users by id or screenname.
Alternatively, you can fetch the entire list of followers using the API.followers_ids method:
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
user_b_followers = api.followers_ids(user_b)
is_following = user_a in user_b_followers
This approach would make more sense for large networks of users.
Keep in mind that, for either approach, you will only be able to see friendships that are visible to the authenticated user. This is a restriction put in place by Twitter for privacy reasons.
I'm trying to programmatically retweet various tweets with Python's python-twitter library. The code executes without error, but the RT never happens. Here's the code:
from twitter import Twitter, OAuth
# my actual keys are here
OAUTH_TOKEN = ""
OAUTH_SECRET = ""
CONSUMER_KEY = ""
CONSUMER_SECRET = ""
t = Twitter(auth=OAuth(OAUTH_TOKEN, OAUTH_SECRET,
CONSUMER_KEY, CONSUMER_SECRET))
result = t.statuses.retweets._id(_id=444320020122722304)
print(result)
The only output is an empty list. How can I get it to actually RT the tweet?
answer using tweepy:
import tweepy
CONSUMER_KEY = ''
CONSUMER_SECRET = ''
ACCESS_KEY = ''
ACCESS_SECRET = ''
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)
api.retweet(tweetID) # e.g. api.retweet(445959276855435264)
# or for use from command line:
# api.retweet(sys.argv[1])
hope that helps? I'm guessing my ACCESS key and secret are equivalent to your OAUTH token and secret..
All of the answers posted here were instrumental in finding the final code that works. Thank you all! The code that works with the python-twitter library is below.
from twitter import Twitter, OAuth
# my actual keys are here
OAUTH_TOKEN = ""
OAUTH_SECRET = ""
CONSUMER_KEY = ""
CONSUMER_SECRET = ""
t = Twitter(auth=OAuth(OAUTH_TOKEN, OAUTH_SECRET,
CONSUMER_KEY, CONSUMER_SECRET))
result = t.statuses.retweet(id=444320020122722304)
print(result)
You don't mention but I'm assuming you're using python-twitter library:
Try using (from the doc)
def PostRetweet(self, original_id, trim_user=False)
Check out Twython's retweet function under Core Interface here https://twython.readthedocs.org/en/latest/api.html and the associated Twitter API https://dev.twitter.com/docs/api/1.1/post/statuses/retweet/%3Aid documents.
Also this post on here Posting a retweet via twython gives 401 whereas I can easily access the timeline.