I'm currently using the GET statuses/user_timeline twitter API in python to retrieve tweets, It says in the docs This method can only return up to 3,200 of a user’s most recent Tweets however when i run it, it only returns 100-150.
How do i get it to return more tweets than it is already?
Thanks in advance
You'll have to write code to work with timelines. Twitter's Working with timelines documentation has a discussion of how and why Twitter timelines work the way they do.
Essentially, set your count to the maximum amount (200) and use since_id and max_id to manage reading each page. Alternatively, you can use an existing library to make the task much easier. Here's an example, using the tweepy library.
consumer_key = "<your consumer_key>"
consumer_secret = "<your consumer_secret>"
access_token = "<your access_token>"
access_token_secret = "<your access_token_secret>"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
for status in tweepy.Cursor(api.user_timeline, "JoeMayo").items():
print('status_id: {}, text: {}'.format(status.id, status.text.encode('utf-8')))
Related
I'm currently trying to learn the Twitter API within Python. My code is this:
import tweepy
consumer_key = "Consumer Key"
consumer_secret = "Consumer Secret"
access_token = "Access Token"
access_token_secret = "Access Token Secret"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_acess_token(access_token, access_token_secret)
auth.secure = True
api = tweepy.API(auth)
tweet = "This tweet was made from a program"
api.update_status(status=tweet)
However this is the error that the code is giving me:
Forbidden: 403 Forbidden
453 - You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve
Process finished with exit code 1
Do I really need to apply for further access just to tweet one thing? Thanks
Twitter has a newer (v2) API. You should use tweepy.Client to be able to use the new endpoints without signing up.
So i currently trying to mine tweets from Twitter account(s), but i wanted to exclude the retweets so i can get 200 of Tweets only data for my project. Currently I have a working code to mine the data feed, but still have Re-Tweets included. I have founded that to exclude Re-Tweets you need to put
-RT in the code but i simply do not know where since i am pretty new to programming.
(Currently using Twitter API for Python (Tweepy) with Python 3.6 using Spyder.)
import tweepy
from tweepy import OAuthHandler
import pandas as pd
consumer_key = 'consumer_key'
consumer_secret = 'consumer_secret'
access_token = 'access_token'
access_secret = 'access_secret'
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
api = tweepy.API(auth)
screen_name='screen_name'
tweets = api.user_timeline(screen_name, count=200)
save=['']*len(tweets)
for i in range(len(tweets)):
save[i]=tweets[i].text
print(tweets[i].text)
data = pd.DataFrame(save)
data.to_csv("results.csv")
Can anyone help me, preferrably with complete section for the code to remove the Retweets. Thank you very much
Faced the same issue back when i was using tweepy to retrieve tweets from twitter, what worked for me was that i used the twitter's api with inbuilt request i.e. http requests.
To exclude retweets you could pass -RT operator in query parameter .
Documentation to this api .
Change this line in your code:
tweets = api.user_timeline(screen_name, count=200)
to the following:
tweets = api.user_timeline(screen_name, count=200, include_rts=False)
This Twitter doc may be helpful: https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html
I am trying to gather the tweets of a user navalny, from 01.11.2017 to 31.01.2018 using tweepy. I have ids of the first and last tweets that I need, so I tried the following code:
import tweepy
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
t = api.user_timeline(screen_name='navalny', since_id = 933000445307518976, max_id = 936533580481814529)
However, the returned value is an empty list.
What is the problem here?
Are there any restrictions on the history of tweets that I can get?
What are possible solutions?
Quick answer:
Using Tweepy you can only retrieve the last 3200 tweets from the Twitter REST API for a given user.
Unfortunately the tweets you are trying to access are older than this.
Detailed answer:
I did a check using the code below:
import tweepy
from tweepy import OAuthHandler
def tweet_check(user):
"""
Scrapes a users most recent tweets
"""
# API keys and initial configuration
consumer_key = ""
consumer_secret = ""
access_token = ""
access_secret = ""
# Configure authentication
authorisation = OAuthHandler(consumer_key, consumer_secret)
authorisation.set_access_token(access_token, access_secret)
api = tweepy.API(authorisation)
# Requests most recent tweets from a users timeline
tweets = api.user_timeline(screen_name=user, count=2,
max_id=936533580481814529)
for tweet in tweets:
tid = tweet.id
print(tid)
twitter_users = ["#navalny"]
for twitter_user in twitter_users:
tweet_check(twitter_user)
This test returns nothing before 936533580481814529
Using a seperate script I scraped all 3200 tweets, the max Twitter will let you scrape and the youngest tweet id I can find is 943856915536326662
Seems like you have run into Twitter's tweet scraping limit for user timelines here.
It seem tweepy's search function only accesses the latest tweets. Is there a way to switch it so it searches the top tweets of a given query? Is there a workaround to retrieve the popular tweets if search cannot do this? I'm running OS X and python 3.6.
Thanks in advance.
Along with your query topic, pass a result_type='popular' parameter to tweepy's search function. Although the tweepy documentation does not list this in its parameters, it is an available parameter in the Twitter Dev Docs.
popular_tweets = api.search(q='python', result_type='popular') # e.g. "python"
popular : return only the most popular results in the response.
This should work (Add your Keys)
import tweepy
consumer_key =""
consumer_secret =""
access_token =""
access_token_secret =""
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
tweets = api.search(q='keyword', result_type='popular')
I'm trying to write a simple twitter bot using python and tweepy. The code is as follows:
import tweepy
CONSUMER_KEY = 'xxxxxxxxxxxxxxxxx'
CONSUMER_SECRET = 'xxxxxxxxxxxxxxxxxxxxx'
ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxx'
ACCESS_SECRET = 'xxxxxxxxxxxxxxxxx'
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)
api.update_status('hi')
I get the following error:
TweepError: Twitter error response: status code = 400
I would like some info on how to avoid this error
This may be a bug that emerged in the 3.2.0 release of Tweepy. See this issue opened on GitHub. In that issue, TylerGlaiel notes that api.update_status fails if called thus:
api.update_status('Test')
But works if called like so:
api.update_status(status='Test')
I have also found this works. I imagine a fix will be out before long.
This sentence: api.update_status(status='Test') let you know if your API Key and Token is working well. So, if you see:
TweepError: Twitter error response: status code = 400
Possibly means that you need to regenerate your credentiales for any reason. Regenerate your API Key, API Secret Key and your Tokens and will work.