I am trying to fetch all tweets using tweepy. I cannot find a document to show how to fetch all tweets rather than filter data using some keywords.
I tried "stream.filter()" without parameters, but it still does not work.
Could anyone give some help?
Thanks.
Related
noofitem = 1000
tweets = tweepy.Cursor(api.search,q=['#iphone11, -filter:retweets'],since='2019-11-14',lang='en',tweet_mode='extended',retweeted=False).items(noofitem)
i = [tweet.full_text for tweet in tweets] #Tweet text
I am trying to get about 1000 tweets using tweepy. But the max tweets I get are around 600. Changing the date does not work. Any modification or other workarounds will be helpful. Thanks.
Please note that Twitter’s search service and, by extension, the
Search API is not meant to be an exhaustive source of Tweets. Not all
Tweets will be indexed or made available via the search interface.
Please refer to this link for more information: http://docs.tweepy.org/en/latest/api.html#help-methods
Probably you will need to set up a Stream to get the amount of data you need.
Suppose that a twitter user has sent a tweet which has a specific tweet id. What is the best way to get n tweets from same user exactly before that specific tweet id using tweepy?
Any help will be appreciated.
Read the official Twitter documentation :
https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html
And use the parameter q with from:#user and count=10 where 10 is your "n" andmax_id=id where "id" is your tweet id.
And read the officiel tweepy documentation for the seach API.
We can fix bugs for you but won't code for you.
I am writing a script in Python, that uses tweepy to search for tweets with a given keyword. Here is the snippet:
for tweet in tweepy.Cursor(api.search, q=keyword, lang="en").items(10):
print tweet.id
I have everything authenticated properly and the code works most of the time. However, when I try to search for some keywords (examples below) it doesn't return anything.
The keywords that cause trouble are "digitalkidz" (a tech conference) and "newtrendbg" (a Bulgarian company). If you do a quick search on Twitter for either of those you will see that there are results. However, tweepy doesn't find anything. Again, it does work for pretty much any other keyword I use.
Do you have any ideas what might be the problem and how to fix it?
Thank you
I believe you're forgetting an important aspect of the twitter api, it's not exhaustive.
Taken from the api docs
Please note that Twitter’s search service and, by extension, the Search API is not meant to be an exhaustive source of Tweets. Not all Tweets will be indexed or made available via the search interface.
Regardless of whether you're using the streaming or rest api, you're going to have issues with this if you're looking for specific tweets.
Rest API
When looking for historical tweets, you unfortunately won't be able to obtain anything that is older than a week using api.search(). This is also shown in the docs.
Keep in mind that the search index has a 7-day limit. In other words, no tweets will be found for a date older than one week.
There are other ways of getting older tweets, this post details those options.
Streaming API
While it doesn't sound like you're using twitter's streaming API, it should be noted that this only gives a small sample of twitter's current tweet traffic (~1-2%).
Hopefully this is helpful. Let me know if you have any questions.
Is there any better way to get tweets from twitter then crawling twitter.com and mutating URLs?
If there is, how can I get latest tweets with current hash-tag?
Thank you!
Did you try Twitter REST API? Particulary, you can use search tweets endpoint. There are some limitations though, enforced by Twitter.
You can use one of many available python libraries.
For example, some sample code for tweepy can be found here.
I have the following questions about tweepy python module
1.I am trying to retrieve all tweets for a specific location. I am able to do this by using tweepy python module (streaming API), but I get only those tweets whose geo locations are enabled, which means I would loose rest of the tweeter’s tweet who have not enabled their geo location. Is there a better way to retrieve all the tweets, given a location?
2.I use Stream.Sample method to retrieve all the tweets, Can someone tell me about the parameters used in sample method? I see count, and async as parameters. Now what should we specify here?
3.What does firehose method in tweepy.Stream do?
Any help is much appreciated
If tweepy doesn't have a feature you need, you can always access Twitter directly with an HTTP request. The full Twitter REST API is described here: https://dev.twitter.com/docs/api
The ones that seem relevant to your interest are:
GET trends/:woeid which looks up tweets by woeid, a Yahoo Identifier for collecting information about a given place/landmark/etc.
GET geo/id/:place_id which only mines geotagged tweets.
There is documentation of all the information available for a GET request but the IP address is not among the available fields: https://dev.twitter.com/docs/api/1/get/search .
Lastly, Twitter has a location search FAQ that may be of interest.