So I'm trying to get a timeline of a specific user. Here is the code:
import tweepy
consumer_key = 'numbers'
consumer_secret = 'numbers'
access_token = 'numbers'
access_token_secret = 'numbers'
user_list = [list of users]
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
for user in user_list:
for page in api.user_timeline(screen_name =user, count = 200):
print page
I've tried using the old documentation. When I run it I get the "Sorry, that page doesn't exist. Code 34"
I found the answer reading through the code documentation change screen_name to user_name
for user in users:
for page in api.user_timeline(user_id =user, count = 200):
print page
Related
import tweepy
import time
cosnsuemr_key = ''
cosnsuemr_secret = ''
access_token = ''
access_token_secret = ''
auth = tweepy.OAuthHandler(cosnsuemr_key,cosnsuemr_secret )
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth, wait_on_rate_limit=True)
user = api.me()
The new Tweepy v4.0.0 removed API.me(). You can use API.verify_credentials() instead.
user = api.verify_credentials()
print(user.screen_name)
I am trying to gather the tweets of a user navalny, from 01.11.2017 to 31.01.2018 using tweepy. I have ids of the first and last tweets that I need, so I tried the following code:
import tweepy
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
t = api.user_timeline(screen_name='navalny', since_id = 933000445307518976, max_id = 936533580481814529)
However, the returned value is an empty list.
What is the problem here?
Are there any restrictions on the history of tweets that I can get?
What are possible solutions?
Quick answer:
Using Tweepy you can only retrieve the last 3200 tweets from the Twitter REST API for a given user.
Unfortunately the tweets you are trying to access are older than this.
Detailed answer:
I did a check using the code below:
import tweepy
from tweepy import OAuthHandler
def tweet_check(user):
"""
Scrapes a users most recent tweets
"""
# API keys and initial configuration
consumer_key = ""
consumer_secret = ""
access_token = ""
access_secret = ""
# Configure authentication
authorisation = OAuthHandler(consumer_key, consumer_secret)
authorisation.set_access_token(access_token, access_secret)
api = tweepy.API(authorisation)
# Requests most recent tweets from a users timeline
tweets = api.user_timeline(screen_name=user, count=2,
max_id=936533580481814529)
for tweet in tweets:
tid = tweet.id
print(tid)
twitter_users = ["#navalny"]
for twitter_user in twitter_users:
tweet_check(twitter_user)
This test returns nothing before 936533580481814529
Using a seperate script I scraped all 3200 tweets, the max Twitter will let you scrape and the youngest tweet id I can find is 943856915536326662
Seems like you have run into Twitter's tweet scraping limit for user timelines here.
def get_tweets(api, input_query):
for tweet in tweepy.Cursor(api.search, q=input_query,lang="en").items():
yield tweet
if __name__ == "__version__":
input_query = sys.argv[1]
access_token = "REPLACE_YOUR_KEY_HERE"
access_token_secret = "REPLACE_YOUR_KEY_HERE"
consumer_key = "REPLACE_YOUR_KEY_HERE"
consumer_secret = "REPLACE_YOUR_KEY_HERE"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
tweets = get_tweets(api, input_query)
for tweet in tweets:
print(tweet.text)
I am trying to download data from Twitter using the command prompt. I have entered my keys (I just recreated them all), saved the script as "print_tweets" and am entering "python print_tweets.py subject" into the command prompt but nothing is happening, no error message or anything.
I thought the problem might have to do with the path environment, but I created another program that prints out "hello world" and this executed without issue using the command prompt.
Can anyone see any obvious errors with my code above? Does this work for you?
I've even tried changing "version" to "main" but this gives me an error message:
if name == "version":
It seems you are running the script in an ipython interpreter, which won't be receiving any command line arguments. Try this:
import tweepy
def get_tweets(api, input_query):
for tweet in tweepy.Cursor(api.search, q=input_query,lang="en").items():
yield tweet
input_query = "springbreak" # Change this string to the topic you want to search tweets
access_token = "REPLACE_YOUR_KEY_HERE"
access_token_secret = "REPLACE_YOUR_KEY_HERE"
consumer_key = "REPLACE_YOUR_KEY_HERE"
consumer_secret = "REPLACE_YOUR_KEY_HERE"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
tweets = get_tweets(api, input_query)
for tweet in tweets:
print(tweet.text)
I have been trying to use either Tweepy or Twython with the Twitter API to search for a specific hashtag, extract usernames of users tweeting with the hashtage, and then see how many of those users follow one another. My ultimate goal is to then visualize the connections with NetworkX.
So far, I have been able to search for the hashtag and get a list of users tweeting with it. However, I can't figure out how to see who is following whom on that list. I finally got a friendship lookup to work, but then realized that that parameter only searches friends of the authenticated user (me).
Here is the latest version of the code:
from twython import Twython
import tweepy
# fill these in from Twitter API Dev
CONSUMER_KEY = ''
CONSUMER_SECRET = ''
ACCESS_KEY = ''
ACCESS_SECRET = ''
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
twitter = Twython(CONSUMER_KEY, CONSUMER_SECRET, ACCESS_KEY, ACCESS_SECRET)
# Search for hashtag, limit number of users
try:
search_results = twitter.search(q='energy', count=5)
except TwythonError as e:
print e
test5 = []
for tweet in search_results['statuses']:
if tweet['user']['screen_name'] not in test5:
test5.append((tweet['user']['screen_name']).encode('utf-8'))
print test5
# Lookup friendships
relationships = api.lookup_friendships(screen_names=test5[0:5])
for relationship in relationships:
if relationship.is_following:
print("User is following", relationship.screen_name)
Thanks!
With Tweepy, you can check if user_a follows user_b using the API.exists_friendship method. The code would look something like:
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
is_following = api.exists_friendship(user_a, user_b)
You can specify users by id or screenname.
Alternatively, you can fetch the entire list of followers using the API.followers_ids method:
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)
user_b_followers = api.followers_ids(user_b)
is_following = user_a in user_b_followers
This approach would make more sense for large networks of users.
Keep in mind that, for either approach, you will only be able to see friendships that are visible to the authenticated user. This is a restriction put in place by Twitter for privacy reasons.
The code below was provided to another user who was scraping the "friends" (not followers) list of a specific Twitter user. For some reason, I get an error when using "api.lookup_users". The error states "Too many terms specified in query". Ideally, I would like to scrape the followers and output a csv with the screen names (not ids). I would like their descriptions as well, but can do this in a separate step unless there is a suggestion for pulling both pieces of information. Below is the code that I am using:
import time
import tweepy
import csv
#Twitter API credentials
consumer_key = ""
consumer_secret = ""
access_key = ""
access_secret = ""
auth = tweepy.auth.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)
ids = []
for page in tweepy.Cursor(api.friends, screen_name="").pages():
ids.extend(page)
time.sleep(60)
print(len(ids))
users = api.lookup_users(user_ids=ids) #iterates through the list of users and prints them
for u in users:
print(u.screen_name)
From the error you get, it seems that you are putting too many ids at once in the api.lookup_users request. Try splitting you list of ids in smaller parts and make a request for each part.
import time
import tweepy
import csv
#Twitter API credentials
consumer_key = ""
consumer_secret = ""
access_key = ""
access_secret = ""
auth = tweepy.auth.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)
ids = []
for page in tweepy.Cursor(api.friends, screen_name="").pages():
ids.extend(page)
time.sleep(60)
print(len(ids))
idChunks = [ids[i:i + 300] for i in range(0, len(ids), 300)]
users = []
for idChunk in idChunks:
try:
users.append(api.lookup_users(user_ids=idChunk))
except tweepy.error.RateLimitError:
print("RATE EXCEDED. SLEEPING FOR 16 MINUTES")
time.sleep(16*60)
users.append(api.lookup_users(user_ids=idChunk))
for u in users:
print(u.screen_name)
print(u.description)
This code has not been tested, and does not writes the CSV, but it should help you getting past that error you were having. The size of 300 for the chunks is completely arbitrary, adjust it if it is too small (or too big).