Hi i try to request some json, all is ok, but sometimes request goes wrong, may be site stack or closing connection for me, cause i am requesting json every 15 min.
Code here:
def request_coins(config):
coins = None
print(str(datetime.now()) + ' - REQUESTING Coins')
while coins is None:
try:
coins = (requests.get(url=str(config['UrlPath']['url']) + str(config['UrlPath']['userrates'])))
except:
print(str(datetime.now()) + " - Site didn't respond. Reconnecting in 10 sec")
time.sleep(10)
if coins is not None:
coins = coins.json()['coins']
print(str(datetime.now()) + ' - Coins received correctly')
return coins
Problems is that try - catch don't work in my case. Request stack in print(str(datetime.now()) + ' - REQUESTING Coins') and i don't recieve any exception and message to cmd
print(str(datetime.now()) + " - Site didn't respond. Reconnecting in 10 sec")
As I understood code stacks at try:
coins = (requests.get(url=str(config['UrlPath']['url']) +
str(config['UrlPath']['userrates'])))
and don't go even to except:
Any idea how to fix?
Just need to add timeout to get request
Related
I'm asking the user for an email, and then sending it to an email verification api, which I then get certain bits of info from. I'm getting a KeyError: 'username' and I have no idea why I'm getting that error. It's also annoying to test since they ratelimit after ~5 attempts
import json, requests, sys
emailInput = ""
def printHelp():
print("Proper usage is: python test.py [email]")
if len(sys.argv) < 2:
printHelp()
sys.exit()
elif len(sys.argv) == 2 and sys.argv[1] == "--help":
printHelp()
sys.exit()
elif len(sys.argv) == 2 and sys.argv[1] != "--help":
emailInput = str(sys.argv[1])
url = 'https://api.trumail.io/v2/lookups/json?email=' + str(emailInput)
res = requests.get(url)
res.raise_for_status()
resultText = res.text
emailInfo = json.loads(resultText)
print("\nEmail Analyzer 1.0\n\nInformation for email: " + sys.argv[1])
print("=====================================================")
print("Username: " + str(emailInfo["username"]))
print("Domain: " + str(emailInfo["domain"]))
print("Valid Format: " + str(emailInfo["validFormat"]))
print("Deliverable: " + str(emailInfo["deliverable"]))
print("Full Inbox: " + str(emailInfo["fullInbox"]))
print("Host Exists: " + str(emailInfo["hostExists"]))
print("Catch All: " + str(emailInfo["catchAll"]))
print("Disposable: " + str(emailInfo["disposable"]))
print("Free: " + str(emailInfo["free"]))
The reason is because a user enters an email that might seem valid - i.e. it's a valid email address with an # symbol etc. - but the email likely does not exist or is not in use.
For example, I ran your script with the following dummy input:
emailInput = 'acdefg#gmail.com'
After I added a print(emailInfo) statement for debugging purposes, this is what I found to be the output from the server:
{'Message': 'No response received from mail server'}
Therefore, your goal here will be to validate the server output. That is, in the case of a valid email that does not exist, you will receive an HTTP 200 (OK) response from the server with a Message field alone populated in the JSON response object. The task here will be to correctly detect the presence of this key, and then run a separate logic other than the happy path, which was already being handled above.
Your error is coming from the fact that emailInfo does not have a key username. Perhaps use emailInfo.get("username", default_value), where default_value is any value you would like if there is no username.
The line with the error is print("Username: " + str(emailInfo["username"]))
I have written an Instagram Liking bot with python. The Liking function is:
Liking photos
unique_photos = len(pic_hrefs)
for pic_href in pic_hrefs:
driver.get(pic_href)
time.sleep(2)
try:
time.sleep(random.randint(4, 15))
like_button = lambda: driver.find_element_by_class_name("_8-yf5 ").click()
like_button().click()
for second in reversed(range(0, random.randint(18, 28))):
print_same_line("#" + hashtag + ': unique photos left: ' + str(unique_photos)
+ " | Sleeping " + str(second))
time.sleep(2)
except Exception as e:
time.sleep(2)
unique_photos -= 1
I want to add now a Likecount with alike limit. I also want to check if I already liked a picture and skip that one. How can I do that?
-I found the problem!- In function SendMessage I was using UserID (with capital letters) instead of userid (which was the actual parameter passed to each thread). So Python printed the UserID of the for cycle instead of the "individual" userid passed to the different functions. It was only a logging problem, the program sent messages correctly.
I have a for that loops through the elements of a user's list. Each iteration, I would like to start a separate background thread to send a message to that user. By saying "send a message" I mean a simple POST request made using the requests Python lib. At the end of the thread, an output on the console is written. Every 24 requests (so every 24 threads) the app needs to stop for about a second.
Success = 0
Bounces = 0
def SendMessage(botid, token, userid, messageid, tag):
global Success
global Bounces
try:
payload = {...}
r = requests.post("...", params=payload, headers=head, timeout=2)
#problem with request?
pjson = json.loads(r.text)
if r.status_code != 200:
log(str(r.status_code) + " " + pjson["result"] + " UserID: " + UserID + "; URL: " + "..." + BotID + "/users/" + UserID + "/send; Params: " + str(payload))
Bounces += 1
return
Success += 1
return
except requests.exceptions.Timeout:
#wait for connection to be available again!
while not conn_available():
print("... Waiting for a new connection...")
time.sleep(10)
log("Request timed out. UserID: " + UserID + "; URL: " + "..." + BotID + "/users/" + UserID + "/send; Params: " + str(payload))
Bounces += 1
except requests.exceptions.ConnectionError:
log("Unable to connect. UserID: " + UserID + "; URL: " + "..." + BotID + "/users/" + UserID + "/send; Params: " + str(payload))
Bounces += 1
except requests.exceptions.HTTPError:
log("Invalid request. UserID: " + UserID + "; URL: " + "..." + BotID + "/users/" + UserID + "/send; Params: " + str(payload))
Bounces += 1
except requests.exceptions.RequestException:
log("Invalid request. UserID: " + UserID + "; URL: " + "..." + BotID + "/users/" + UserID + "/send; Params: " + str(payload))
Bounces += 1
while True:
newMsgsReq = ""
try:
#Check for new messages
newMsgsReq = requests.get("...", timeout=2)
if newMsgsReq.text == "false":
#exit sub
time.sleep(2)
continue
except requests.exceptions.HTTPError as errh:
log("Request has failed: There was an error in the request: [" + str(errh) + "]")
time.sleep(2)
continue
except requests.exceptions.ConnectionError as errc:
log("Request has failed: check internet connection & retry: [" + str(errc) + "]")
time.sleep(2)
continue
except requests.exceptions.Timeout as errt:
log("Request has failed: check internet connection & retry: [" + str(errt) + "]")
time.sleep(2)
continue
except requests.exceptions.RequestException as err:
log("Request has failed: There was an error in the request: [" + str(err) + "]")
time.sleep(2)
continue
#we have a message!!!
#Extract BotID, Token, MessageID
msgInf = newMsgsReq.text.split("|")
MessageID = msgInf[0]
BotID = msgInf[1]
Token = msgInf[2]
Tag = msgInf[3]
del msgInf[0:4]
suc("New message found: " + str(MessageID))
suc("Total recipients: " + str(len(msgInf)))
#Begin send!
Cycles = 0
TotCycles = 0
#Loop through msgInf
for UserID in msgInf:
#Create the thread.
process = threading.Thread(target=SendMessage, args=[BotID, Token, UserID, MessageID, Tag])
process.start()
TotCycles += 1
pb.print_progress_bar(TotCycles)
Cycles += 1
if Cycles == 24:
time.sleep(1)
Cycles = 0
suc("Message " + str(MessageID) + " sent successfully (" + str(Success) + " success, " + str(Bounces) + " bounces")
Success = 0
Bounces = 0
time.sleep(3)
Let's say my user list is:
{1, 2, 3, 4, ..., 24, 25, ...}. I expect my application to output:
1. Message 1 sent successfully...
2. Message 2 sent successfully...
...
24. Message 24 sent successfully.
Instead, I am getting this output:
1. Message 1 sent successfully.
2. Message 1 sent successfully.
...
24. Message 1 sent successfully.
So all the 24 outputs are related to the first of the 24 ids. It seems like the for loop does not proceed...
This prints the incremented counter without any trouble so I think you may need to provide all of the code and some sample input.
import threading
import time
def SendMessage(userid):
print(userid)
while True:
cycles = 1
for user_id in [1, 2, 3]:
process = threading.Thread(target=SendMessage, args=[user_id])
process.start()
cycles += 1
if cycles == 24:
time.sleep(1)
cycles = 0
time.sleep(3)
Run it on repl.it
I've been having a lot of trouble multiprocessing - I've literally been trying for hours and can't get it right. Here's my code, commented the best I could do.
Linked all my code as I don't know what's causing it exactly.
Line 74 it says, on p.start()
The most relevant part of code is the bottom of the question.
Here are my imports
import urllib
import socket
import multiprocessing as mp
import queue
import requests
Header used for higher chance of success upon connecting to a website
headers={'User-agent' : 'Mozilla/5.0'}
Main function takes four parameters - queue, the URL List, the Output file, and the list of vulnerable URLs.
def mainFunction(q, URLList, Output, vulnURLS):
This list is used to check if the page source has any of the errors in the list after adding a string query to the end of the url (')
queries = ['SQL syntax', 'mysql_fetch', 'mysql_num_rows', 'mySQL Error', 'mySQL_connect()', 'UNION SELECT', 'MySQL server version']
This puts the URL in the correct format before testing for injection points.
URLReplace = [("['", ""),("']",""), ("\n", ""), ("https://","http://"), ("\s", "%20"), ("\s", "%20")]
URL = ''.join(str(URLList))
for URL in URLList:
if (z < len(URLReplace)):
URL = URL.replace(URLReplace[z])
z = z + 1
URL = (URL + "'")
This is the try request, where it attempts to connect and scrapes the HTML off of the webpage.
try:
req = requests.get(URL, timeout=2)
htmlObject = urllib.request.urlopen(URL)
This iterates through the list to check for any possible vulnerabilities. Also returns 404/400 messages.
if (y < len(queries)):
if queries[x] in htmlObject:
print ("\t [+] " + URL)
vulnURLS.append(URL)
Output.open()
for VURLS in vulnURLS:
Output.write(VURLS + '\n')
Output.close()
y = y + 1
else:
print ("\t [-] " + URL)
except urllib.error.HTTPError as e:
if e.code == 404:
print("\t [-] Page not found.")
if e.code == 400:
print ("\t [+] " + URL)
except urllib.error.URLError as e:
print("\t [-] URL Timed Out")
except socket.timeout as e:
print("\t [-] URL Timed Out")
except socket.error as e:
print("\t [-] Error in URL")
Here's the important part, where I use the Queue & multiprocessor.
if __name__=='__main__':
q = mp.Queue()
URLList = [i.strip().split() for i in open('sites.txt').readlines()]
Output = open('output.txt', 'r')
vulnURLS = []
p = mp.Process(target=mainFunction, args=(q, URLList, Output, vulnURLS))
p.start()
q.put(mainFunction(URLList))
q.close()
q.join_thread()
p.join()
Please help me out with this problem, I've been stuck on it for hours and am getting very frustrated that I cannot follow the solution. Every module I look at I follow to a T and get this same error.
I have tried multi-threading, but it is extremely slow and unstable when compared to multiprocessing.
Change to the following:
p = mp.Process(target=mainFunction, args=(q, Output))
p.start()
for url in URLList:
q.put(url)
I have been working on a TwitchTV Python chat bot for a while now, but I'm still getting to grips with Python.
It may seem simple, but this has confused me so I decided I would ask:
I'm currently pulling messages from Twitch Chat using data = irc.recv
What I want to do is use the data pulled and turn it into a string, so that I can then check for capitals in the messages using str.isupper()
I've already tried a few ways;
data = irc.recv (4096)
msg = data()
capsTime = "30s"
str = msg
if str.isupper():
message("[-] Woah! Hold back on the caps! (Timeout " + capsTime + ")")
message("/timeout " + user + capsTime)
# variable "user" already defined
This is just one, that unfortunately didn't work.
EDIT:
This is my new code, It runs without error messages, but It doesn't function as I want it to;
while True:
data = irc.recv (4096)
user = data.split(':')[1]
user = user.split('!')[0]
caps = data.split(':')[0]
capsmsg = str(data)
print data
if data.find('PING') != -1:
irc.send('PONG ' + data.split()[1] + '\r\n')
if capsmsg.isupper():
message("[-] Woah! Hold back on the caps, " + user + "! (Timeout 30s)")
message("/timeout " + user + " 30s")
EDIT 2:
Expected output:
If a message is found in ALL caps, it will print this message and time the user out:
message("[-] Woah! Hold back on the caps, " + user + "! (Timeout 30s)")
Current output:
The bot does not pick the message up or run the scripted code.
Try this:
data = irc.recv (4096)
# msg = data()
capsTime = "30s"
mystr = repr(data)
if mystr.isupper():
message("[-] Woah! Hold back on the caps! (Timeout " + capsTime + ")")
message("/timeout " + user + capsTime)
# variable "user" already defined
Don't use reserved keyword.