I am making an Instagram bot and I store the names of the users that the bot has followed in file.txt.
unique_photos = len(pic_hrefs) # TODO Let this run once and check whether this block of code works or not
followers_list = [] # Contains the names of the people you followed
for pic_href in pic_hrefs:
driver.get(pic_href)
sleep(2)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
try:
# Like this picture
driver.find_element_by_xpath("//*[#aria-label='Like']").click()
print("Picture liked") # TODO After checking delete this line
follow_button = driver.find_element_by_class_name('bY2yH')
# Follow the user if not followed already
if follow_button.text == "•\n" + "Follow":
follow_button.click()
followed = driver.find_element_by_class_name('e1e1d')
followers_list.append(followed.text)
with open("file.txt", 'a') as file:
file.write(",".join(followers_list))
file.write(",")
else:
continue
for second in reversed(range(0, 3)):
print_same_line("#" + tag + ': unique photos left: ' + str(unique_photos)
+ " | Sleeping " + str(second))
sleep(1)
except Exception:
sleep(2)
unique_photos -= 1
This is the final result in the file.txt:
kr.dramas_,kr.dramas_,marcelly.lds,kr.dramas_,marcelly.lds,espn
It's clear that the problem is that as I append the whole followers_list (which contains all the usernames of the people the bot followed) the names repeat. So I need a way to only append the new names.
And I know that I can just change the code to 'w' to create a whole new file every time but that creates a problem because after I stop the bot and if I don't unfollow the users from that list and start the bot again I will lose all the names from the file, which I don't want.
So I need suggestions so that after the bot is stopped the file.txt looks like this:
kr.dramas_,marcelly.lds,espn,
I would suggest that once you've followed everyone, you can read all of the names from the file into a list/set and then add names that aren't in the list/set into it. Then simply overwrite the old file.
followers_list = [] # will be populated with follower names
with open("file.txt", 'r') as file:
file_names = file.readline().split(",")
for follower in followers_list:
if follower not in file_names:
file_names.append(follower)
with open("file.txt", 'w') as file:
file.write(",".join(file_names))
Related
So basically I want to create a variable that changes after every iteration of a for loop to be the same as the search term that is used in the for loop in question, is that possible? I explained better in the code I think.
with open ('lista1.txt','r') as file_1:
reader_0 = file_1.readlines() # Reads a list of searchterms,
# the first search term of this list is "gt-710".
for search in reader_0:
file_0 = search.replace("\n","") +".txt"
file_1 = str(file_0.strip())
try: #if the file named the same as the searchterm exists, read its contents
file = open(file_1,"r")
search = file.readlines() # How do I create a variable that
# changes names? for example I want the
# content of file readlines be saved in
# a variable called the same as the
# searchterm in this ase I want it to
# be gt-710 = file.readlines()...in the
# next iteration I want it to be
# next_search_term_in_the_list =
# file.readlines()..an so on...
print(str(search) + "I actually tried")
except: #if not, create it
file = open(file_1,"w")
file.write("hello")
print("I didnt")
file.close()
This is impossible in Python, but you can do something similar. Enter stage left, the DICTIONARY! A dictionary is like a list, but you set your own keys. Make it like this:
my_dict = {}
You can add to the dictionary like so:
my_dict["key"] = "value"
A way you could implement this into your code could be as follows:
the_dict = {}
with open ('lista1.txt','r') as file_1:
[...]
file = open(file_1,"r")
file_contents = file.readlines()
the_dict[search] = file_contents
print(str(file_contents) + "I actually tried")
[...]
I am finally getting the hang of Python and have started using it on a daily basis at work. However, the learning curve is still steep and I have hit a roadblock in trying something new with a code I found here for scraping members from telegram channels.
Currently in lines 38-44 we can select a group from the list and it will scrape the user data into members.csv .
EDIT: Resolved the CSV naming issue:
print('Saving In file...')
print(target_group.title)
filename = target_group.title
with open(("{}.csv".format(filename)),"w",encoding='UTF-8') as f:
Instead of relying on input, I would like to create a for loop which would iterate through every group in the list.
print('Choose a group to scrape members from:')
i=0
for g in groups:
print(str(i) + '- ' + g.title)
i+=1
g_index = input("Enter a Number: ")
target_group=groups[int(g_index)]
The problem is that I am not sure exactly how to replace this part of the code with a for loop.
Although, just changing it into a for loop would make it merely overwrite the same members.csv file with each iteration, I plan on changing that so that it outputs into unique files.
So circling back to my question. How do I make this single program iteration loop through all of the groups, or just select all of them.
Thanks for the help !
Couldn't test this, but something like this maybe? This creates a new .csv file for each group.
for chat in chats:
try:
if chat.megagroup == True:
groups.append(chat)
except:
continue
for current_group in groups:
print(f"Fetching members for group \"{current_group.title}\"...")
all_participants = client.get_participants(current_group, aggressive=True)
current_file_name = f"members_{current_group.title}.csv"
print(f"Saving in file \"{current_file_name}\"...")
with open(current_file_name, "w+", encoding="UTF-8") as file:
writer = csv.writer(file, delimiter=",", lineterminator="\n")
writer.writerow(["username", "user id", "access hash", "name", "group", "group id"])
for user in all_participants:
username = user.username if user.username else ""
first_name = user.first_name.strip() if user.first_name else ""
last_name = user.last_name.strip() if user.last_name else ""
name = f"{first_name} {last_name}"
row = [username, user.id, user.access_hash, name, current_group.title, current_group.id]
writer.writerow(row)
print(f"Finished writing to file \"{current_file_name}\".")
print("Members scraped successfully.")
Ended up figuring out the issue:
On naming the CSV file: Used the title attribute to name the file and replacement within the string.
g_index = chat_num
target_group=groups[int(g_index)]
filename = target_group.title
print('Fetching Members from {} ...'.format(filename))
all_participants = []
all_participants = client.get_participants(target_group, aggressive=True)
print('Saving In file...')
with open(("{}.csv".format(filename)),"w",encoding='UTF-8') as f:
On creating a for loop for the sequence: The original code (posted in the question) did not include a for loop. My version of a workaround was to create a function from everything and then iterate through a an indexed list that was equal to the amount of instances detected. In the end looking like this:
chat_list_index = list(range(len(chats)))
for x in chat_list_index:
try:
get(x)
except:
print("No more groups.", end = " ")
pass
pass
print("Done")
Overall, this might not be the best solution to accomplish what I sought out to, however its good enough for me now, and I have learned a lot. Maybe someone in the future finds this beneficial. Full code available here: (https://github.com/ivanstruk/telegram-member-scraper/).
Cheers !
I'm creating a Contact list/book program which can create new contacts for you. Save them in a 'txt' file. List all contacts, and delete existing contacts. Well sort of. In my delete function there is an error which happens and I can't quite tell why?. There isn't a error prompted on the shell when running. It's meant to ask the user which contact they want to delete, find what the user said in the 'txt' file. Then delete it. It can find it easily, however it just doesn't delete the string at all.
I have tried other methods including if/else statements, other online code (copied) - nothing works.
import os, time, random, sys, pyautogui
#function for creating a new contact.
def new_contact():
name = str(input("Clients name?\n:"))
name = name + " -"
info = str(input("Info about the client?\n:"))
#starts formatting clients name and info for injection into file.
total = "\n\n"
total = total + name
total = total + " "
total = total + info
total = total + "\n"
#Injects info into file.
with open("DATA.txt", "a") as file:
file.write(str(total))
file.close
main()
#function for listing ALL contacts made.
def list():
file = open("DATA.txt", "r")
read = file.read()
file.close
#detects whether there are any contacts at all. If there are none the only str in the file is "Clients:"
if read == "Clients:":
op = str(input("You havn't made any contacts yet..\nDo you wish to make one?\n:"))
if op == "y":
new_contact()
else:
main()
else:
print (read)
os.system('pause')
main()
#Function for deleting contact
def delete_contact():
file = open("DATA.txt", "r")
read = file.read()
file.close
#detects whether there are any contacts at all. If there are none the only str in the file is "Clients:"
if read == "Clients:":
op = str(input("You havn't made any contacts yet..\nDo you wish to make one?\n:"))
if op == "y":
new_contact()
else:
main()
else:
#tries to delete whatever was inputted by the user.
file = open("DATA.txt", "r")
read = file.read()
file.close
print (read, "\n")
op = input("copy the Clinets name and information you wish to delete\n:")
with open("DATA.txt") as f:
reptext=f.read().replace((op), '')
with open("FileName", "w") as f:
f.write(reptext)
main()
#Main Menu Basically.
def main():
list_contacts = str(input("List contacts? - L\n\n\nDo you want to make a new contact - N\n\n\nDo you want to delete a contact? - D\n:"))
if list_contacts in ("L", "l"):
list()
elif list_contacts in ("N", "n"):
new_contact()
elif list_contacts in ("D", "d"):
delete_contact()
else:
main()
main()
It is expected to delete everything the user inputs from the txt file. No errors show up on shell/console, it's as if the program thinks it's done it, but it hasn't. The content in the txt file contains:
Clients:
Erich - Developer
Bob - Test subject
In your delete function, instead of opening DATA.txt, you open "FileName"
When using “with”, a file handle doesn't need to be closed. Also, file.close() is a function, you didnt call the function, just its address.
In addition, in the delete function, you opened “fileName” instead of “DATA.txt”
I have a function that read from a JSON file and display the content in a QtextEdit using Pyqt5.
Problem is that when i tried to parse the content in the TextEdit the last record of the File Name is displayed.
while if i print to the console all the records of the File Name are printed as it should.
at the end i need to display the result as the comment print
def displayReport(self,item):
searchRes=os.path.join(os.getcwd(),"search_result")
path = os.listdir(searchRes)
data =[]
try:
for file in path:
rpjson = json.load(open(os.path.join(searchRes,item)))
for js in rpjson:
fileName = js["File Name"]
srchwRD = js["Searched Word"]
nbrOfOccur = str(js["Number Of Occurence"])
result = [fileName + srchwRD + nbrOfOccur]
print("this is file name {}".format(fileName))
data.append(result)
#print("****************" + "\n" + "File Name: " +
#js["File Name"] + "\n" + "Searched Word: " +
#js["Searched Word"] + "\n" + "Number Of Occurence: " +
#str(js["Number Of Occurence"]))
except Exception as e:
print("can't read JSON because {}".format(e))
self.textEdit.setHtml("\n".join (data))
You are not "adding" to the textedit, instead you are replacing its content on each iteration of the loop - only the last content sticks.
Change
self.textEdit.setHtml(str(fileName)) # replaces the whole content of the qtestedit
to addding to its current content instead of replacing it.
Possbible solutions
See this SO post: How to append text to QPlainTextEdit without adding newline, and keep scroll at the bottom? for how to achieve this. (maybe you should add a newline between file names - you can adapt the solutions).
Maybe better way to do it:
Have a look at the API, append might also be a valid choice:QTextEdit.append(...)
collect all the text you need into a normal list of strings and set the QTextEdit only once after you visited all files. This should be faster performane wise as well as Qt does not update its element multiple times and has to process triggers and signals on it:
def displayReport(self,item): # code untested, no mvce provided, may need small fixup(s)
foundText = []
searchRes=os.path.join(os.getcwd(),"search_result")
path = os.listdir(searchRes)
try:
for file in path:
rpjson = json.load(open(os.path.join(searchRes,item)))
for js in rpjson:
fileName = js["File Name"]
print("this is file name {}".format(fileName))
foundText.append(fileName) # add to list, join later, faster then
# add to the immutable string multiple times
except Exception as e:
print("can't read JSON because {}".format(e))
self.textEdit.setHtml('\n'.join(foundText)) # set text only once after processing
I would probably go for the last option (collecting into list + set once) as it minimizes Signals and Triggers.
Ive got a txt file like:
first.txt
Johnny^plumber^NY;Anna^doctor^Washington;Kate^admin^Florida
then i've got one many output3*.txt files in folder which the data is saving all the time:
haha plumber blabla;
other one could be like:
haha doctor blabla;haha admin blabla
if there is no word "exit" in the output3*.txt files - its waiting for few seconds and then searching those words (plumber doctor admin) between haha and blabla in every file which didnt had "exit" inside and removing those words from the first txt file.
file_names3 = glob.glob(pathtemp+"/output3*.txt")
abort_after = 1 * 5
start = time.time()
while True:
if not file_names3:
break
delta = time.time() - start
if delta >= abort_after:
with open(path+"/"+statuses, "a") as statuses:
statuses.write("-----------------\n ERRORS:\n\n-----------------\n")
for file_name in file_names3:
statuses.write("%s" % file_name + " - file not done: ")
with open(file_name, 'r') as prenotf:
reader=prenotf.read()
for "haha" in reader:
finding=reader[reader.find("haha")+5:reader.find("blabla")]
statuses.write(finding)
break
time.sleep(3)
for file_name in file_names3:
with open(file_name, "r") as zz:
if "exit" in zz.read(): #<<<--- test data
file_names3.remove(file_name)
print ("\n ############# List of files still Waiting to be done:\n")
print (file_names3)
Im stuck in searching for those words between haha and blabla.
Thanks for any help.
When you alter an object while you're iterating through it, you foul up the inherent location pointer. This pointer is absolute. If you delete 10 characters from the file, the rest of the file shifts up, but the pointer doesn't change. This effective skips the next 10 characters.
Your logic comes in two parts, then:
Write to a second file while you parse the first. Once you're done, you can move the new file to the old name.
Maintain an active flag. Turn it off when you hit haha and back on when you hit blabla.
It looks something like this:
temp_file = open("tempfile.txt", 'w')
active = True
for line in <your input>:
if "haha" in line:
active = True
elif "blabla" in line:
active = False
elif active
temp_file.write(line)
Can you work that into your program's current logic?