I am trying to get a list of IDs from a list of usernames I have. Is there any method that tweepy provides that lets me do lookup user IDs using their username?
Twitter API has the resource https://dev.twitter.com/rest/reference/get/users/lookup for such requirements. It can return user objects for at most 100 users at a time.
You can use this in Tweepy like:
user_objects = api.lookup_users(screen_names=list_of_at_most_100_screen_names)
user_ids = [user.id_str for user in user_objects]
screen_name = unames['username']
enter code here
#my username df
#0 briankrebs
#1 Dejan_Kosutic
#2 msftsecresponse
#3 PrivacyProf
#4 runasand
data = []
def return_twitterid(screen_name):
print("The screen name is: " + screen_name)
twitterid = client.get_user(username=screen_name)
id = twitterid.data.id
return id
for s in range(len(screen_name)):
u_id = return_twitterid(screen_name[s])
data.append(u_id)
print(data)
For anyone who landed here from Google, this is a code snippet that is basically a username to id converter. It supports twitter api V2.
# the usernames variable is a list containing all the usernames you want to convert
usernames = ["POTUS", "VP"]
users = client.get_users(usernames=usernames)
for user in users.data:
print(user.id)
Related
I find very cool library called Instagram Private Api and Im trying get all people subscribed on my Instagram for educational purposes.
But I can't get more than 100 units :(
Can somebody help me understand how to fix it?
from random import randint
from time import sleep
user_id = api.username_info('target')['user']['pk']
# Create a list of followers' usernames
usernames = []
next_max_id = followers.get('next_max_id')
while next_max_id:
delay = randint(20,40)
print("Sleep " + str(delay) + "s")
sleep(delay)
# Get a list of the user's followers
followers = api.user_followers(user_id, rank_token=api.generate_uuid(),)
next_max_id = followers.get('next_max_id')
for follower in followers['users']:
usernames.append(follower['username'])
# Print the list of followers' usernames
print(len(usernames))```
From the documentation: you can pass max_id to offset the selection of users.
You can also look at the provided example.
I am trying to scrape tweets from twitter using twython and I want to use enterprise search api for this because I want to define fromDate and toDate parameters.
I couldn't find any way to do it though, and when I try to cursor tweets from this date, It only returns the tweets about 14 days ago from now.
twitter = Twython(consumer_token, access_token=ACCESS_TOKEN)
# Search parameters
def search_query(QUERY_TO_BE_SEARCHED):
"""
QUERY_TO_BE_SEARCHED : text you want to search for
"""
df_dict=[]
results = twitter.cursor(twitter.search, q=QUERY_TO_BE_SEARCHED,fromDate='2019071200',toDate='2019071400',count=100)
for q in results:
retweet_count = q['retweet_count']
favs_count = q['favorite_count']
date_created = q['created_at']
text = q['text']
hashtags = q['entities']['hashtags']
user_name = '#'+str(q['user']['screen_name'])
user_mentions = []
if(len(q['entities']['user_mentions'])!=0):
for n in q['entities']['user_mentions']:
user_mentions.append(n['screen_name']) # Mentioned profile names in the tweet
temp_dict = {'User ID':user_name,'Date':date_created,'Text':text,'Favorites':favs_count,'RTs':retweet_count,
'Hashtags':hashtags,'Mentions':user_mentions}
df_dict.append(temp_dict)
return pd.DataFrame(df_dict)
that is my code, can you help me improve this ?
I want to look up all the friends (meaning the twitter users one is following) of a sample of friends of one twitter account, to see what other friends they have in common. The problem is that I don't know how to handle protected accounts, and I keep running into this error:
tweepy.error.TweepError: Not authorized.
This is the code I have:
...
screen_name = ----
file_name = "followers_data/follower_ids-" + screen_name + ".txt"
with open(file_name) as file:
ids = file.readlines()
num_samples = 30
ids = [x.strip() for x in ids]
friends = [[] for i in range(num_samples)]
for i in range(0, num_samples):
id = random.choice(ids)
for friend in tweepy.Cursor(api.friends_ids, id).items():
print(friend)
friends[i].append(friend)
I have a list of all friends from one account screen_name, from which I load the friend ids. I then want to sample a few of those and look up their friends.
I have also tried something like this:
def limit_handled(cursor, name):
try:
yield cursor.next()
except tweepy.TweepError:
print("Something went wrong... ", name)
pass
for i in range(0, num_samples):
id = random.choice(ids)
items = tweepy.Cursor(api.friends_ids, id).items()
for friend in limit_handled(items, id):
print(friend)
friends[i].append(friend)
But then it seems like only one friend per sample friend is stored before moving on to the next sample. I'm pretty new to Python and Tweepy so if anything looks weird, please let me know.
First of all, a couple of comments on naming. The names file and id are protected, so you should avoid using them to name variables - I have changes these.
Secondly, when you initialise your tweepy API, it's clever enough to deal with rate limits if you use wait_on_rate_limit=True and will inform you when it's delayed due to rate limits if you use wait_on_rate_limit_notify=True.
You also lose some information when you set friends = [[] for i in range(num_samples)], as you then won't be able to associate the friends you find with the account they relate to. You can instead use a dictionary, which will associate each ID used with the friends found, allowing for better processing.
My corrected code is as follows:
import tweepy
import random
consumer_key = '...'
consumer_secret = '...'
access_token = '...'
access_token_secret = '...'
# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
# Creation of the actual interface, using authentication. Use rate limits.
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)
screen_name = '----'
file_name = "followers_data/follower_ids-" + screen_name + ".txt"
with open(file_name) as f:
ids = [x.strip() for x in f.readlines()]
num_samples = 30
friends = dict()
# Initialise i
i = 0
# We want to check that i is less than our number of samples, but we also need to make
# sure there are IDs left to choose from.
while i <= num_samples and ids:
current_id = random.choice(ids)
# remove the ID we're testing from the list, so we don't pick it again.
ids.remove(current_id)
try:
# try to get friends, and add them to our dictionary value if we can
# use .get() to cope with the first loop.
for page in tweepy.Cursor(api.friends_ids, current_id).pages():
friends[current_id] = friends.get(current_id, []) + page
i += 1
except tweepy.TweepError:
# we get a tweep error when we can't view a user - skip them and move onto the next.
# don't increment i as we want to replace this user with someone else.
print 'Could not view user {}, skipping...'.format(current_id)
The output is a dictionary, friends, with keys of user IDs and items of the friends for each user.
how do I derrive a plaintext username from a user Id number with tweepy?
Here is the CORRECTED code that I am using:
ids = []
userid = "someOne"
for page in tweepy.Cursor(api.followers_ids, screen_name=userid).pages():
ids.extend(page)
time.sleep(60)
print len(ids), "following have been gathered from", userid
users = api.lookup_users(user_ids=ids)#ieterates through the list of users and prints them
for u in users:
print u.screen_name
The last line I have commented is the one giving me an issue. Please advise. Thank you all!
You are iterating ids and i already contains element of ids. Try to pass i to lookup_users:
for i in ids:
print screen_name(api.lookup_users(i))
# print screen_name(api.lookup_users(ids[i]))
update, try this way:
users = api.lookup_users(user_ids=ids)
for u in users:
print u.screen_name
I have a simple one-to-many structure like this:
class User(db.Model):
userEmail = db.StringProperty()
class Comment(db.Model):
user = db.ReferenceProperty(User, collection_name="comments")
comment = db.StringProperty()
date = db.DateTimeProperty()
I fetch a user from by his email:
q = User.all() # prepare User table for querying
q.filter("userEmail =", "az#example.com") # apply filter, email lookup
results = q.fetch(1) # execute the query, apply limit 1
the_user = results[0] # the results is a list of objects, grab the first one
this_users_comments = the_user.comments # get the user's comments
How can I order the user's comments by date, and limit it to 10 comments?
You will want to use the key keyword argument of the built-in sorted function, and use the "date" property as the key:
import operator
sorted_comments = sorted(this_users_comments, key=operator.attrgetter("date"))
# The comments will probably be sorted with earlier comments at the front of the list
# If you want ten most recent, also add the following line:
# sorted_comments.reverse()
ten_comments = sorted_comments[:10]
That query fetches the user. You need to do another query for the comments:
this_users_comments.order('date').limit(10)
for comment in this_users_comments:
...