We Keep Coding

The Python Request module does not function when including a proxy

I have recently tried the python request module and it seems to work fine up until the point when I include a proxy in the command. I am using the Burp Suite proxy, when I run the code the program gets stuck on the line of code with the request module.
import requests
import sys
import urllib3
#input = "https://0a0100660376e8efc04b1a7600880072.web-security-academy.net/"
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
proxies = {'http': 'http://127.0.0.1:8080', 'https': 'https://127.0.0.1:8080'}
def exploit_sqli_column_number(URL):
path = "filter?category=Tech+gifts"
for i in range(1,51):
sql_payload = "'+order+by+%s--" %i
r = requests.get(url + path + sql_payload, verify = False, proxies = proxies)
res = r.text
if "Internal Server Error" in res:
return i - 1
return False
if __name__ == "__main__":
try:
url = sys.argv[1]
except IndexError:
print("[-] Usage: %s <url>" % sys.argv[0])
print("[-] Example: %s www.example.com" % sys.argv[0])
sys.exit(-1)
print("[+] Figuring out number of columns.")
num_col = exploit_sqli_column_number(URL)
if num_col:
print("[+] The number of columns is " + str(num_col)
+ ".")
else:
print("[-] The SQL Injection was not successful.")
I have tried other scripts where I just make the request without using the proxy and it works just fine, I have also checked the IP address and the Port, so there should be no issues with that.
Thank you for help in advance.

This code works for me:
import requests
import sys
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
proxies = {'http': 'http://127.0.0.1:8085', 'https': 'https://127.0.0.1:8085'}
r = requests.get('https://www.google.com', verify = False, proxies = proxies)
print(r)
I'd make sure you set the correct ports in Burp Suite under Proxy -> Options, and make sure you turn off intercept. If your code is just hanging and not giving any error then the issues is you have not turned off intercept. I would try using a port other than the default 8080 for your proxy.

How to use a proxy while opening browsers in python

Basically what I want to do is for me to be able to connect to a proxy while I am able to browse the internet
Here is the code I tried
import webbrowser
import socks, socket, requests
from multiprocessing import Process
def proxy():
while True:
socks.set_default_proxy(socks.SOCKS5, "184.32.91.92", 2901)
socket.socket = socks.socksocket
if __name__ == '__main__':
proxy_process = Process(target=proxy).start()
r = requests.get("http://icanhazip.com")
print(r.content) # stil gives me my actual IP address
webbrowser.open("http://icanhazip.com", new=2) # opening the webbrowser
So i tried to process the proxy to keep the connection alive but even when I open the browser it still gives me my actual IP

Looks like a timing issue. Your __main__ process probably reaches the line:
r = requests.get("http://icanhazip.com")
before proxy_process has done it's job.

Get incoming URL from listening socket/HTTPServer

Good day Stackoverflow,
This morning I've ran into a problem of which I can't seem to find a working answer. I'm trying to get the full URL (what shows up in the address bar) via either a HTTPServer or simple socket-ing that I get from a server that redirects me to localhost (Which has nothing behind it (no webserver, no pages, nothing), except the listening code below.) with the token and scope variables (as seen in the URL in question below). My desired result would be these variables to be saved so I can work with them:
http://localhost/#token=aai789as&scope=book%3Aedit+chat%3Aedit
I have tried the following with some progress but not the desired result:
from http.server import SimpleHTTPRequestHandler, HTTPServer
from urllib.parse import parse_qs
class MyHandler(SimpleHTTPRequestHandler):
def do_GET(self):
qs = {}
path = self.path
if '?' in path:
path, tmp = path.split('?', 1)
qs = urlparse.parse_qs(tmp)
print(self.path)
print (path, qs)
def log_request(self, code=None, size=None):
print('Request')
def log_message(self, format, *args):
print('Message')
if __name__ == "__main__":
try:
server = HTTPServer(('localhost', 80), MyHandler)
print('Started http server')
server.serve_forever()
except KeyboardInterrupt:
print('^C received, shutting down server')
server.socket.close()
The above snippet loads, but doesn't actually print anything of use. In fact, it prints just blank statements. But it does detect a connection being made.
So does this:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("localhost", 80))
s.listen(1)
conn, addr = s.accept()
d = conn.recv(4096)
conn.close()
print(d)
But this DOES return more than blank statements, yet it's hardly enough to get the variables from the URL:
b'\x07\n'
b'GET / HTTP/1.1\r\nHost: localhost\r\nUser-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en,en-US;q=0.7,nl;q=0.3\r\nAccept-Encoding: gzip, deflate\r\nDNT: 1\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\n\r\n'
I don't know what I'm supposed to be doing and since I don't know what exactly I'm looking for; searching through the documentation has taken up the better half of my day. As such I turn to the ever helpful Stackoverflow in the hopes of finding better knowledge than I possess.
Thank you for your time,
- Brent

http://localhost/#token=aai789as&scope=book%3Aedit+chat%3Aedit
This kind of URL is only transferred in part to the server. The # and everything after is only known to the browser and can be accessed as location.hash from withing Javascript. It will not be transferred to the server, i.e. all the server will see is http://localhost/.
b'GET / HTTP/1.1\r\nHost: localhost\r\n ...'
This part provides everything from the URL which the server will know. The localhost in the Host header specifies the hostname and the GET / specifies the path / - which makes together 'http://' + 'localhost' + '/' i.e. http://localhost/.
For more information see Is the anchor part of a URL being sent to a web server?.

Check internet availability before sending the data to database

I am new to Python 2.7. I am writing a program where I need to check the availability of internet for my Wifi (sometimes the internet disconnects) before I proceed to send the data to the database using the Internet. The send data to database will be skipped if there is no internet connection. How can I do that. Is this the correct way that I doing this?
import urllib
#Perhaps check internet availability first
try:
import httplib
except:
import http.client as httplib
def have_internet():
conn = httplib.HTTPConnection("www.google.com", timeout=5)
try:
conn.request("HEAD", "/")
conn.close()
return True
except:
conn.close()
return False
#send data to database
data = {'date':date_mmddyyyy,'time':time_hhmmss,'airtemperature':temperature_c,'humidity':humidity_c, 'watertemperature':watertemp_c, 'phsolution':pHvalue_c, 'waterlevel':distance_c, 'CO2 concentration':CO2_c, 'TDS value':tds_c}
result = firebase.put('Region 1', 'Parameter Reading', {'date':date_mmddyyyy,'time':time_hhmmss,'airtemperature':temperature_c,'humidity':humidity_c, 'watertemperature':watertemp_c, 'phsolution':pHvalue_c, 'waterlevel':distance_c, 'CO2 concentration':CO2_c, 'TDS value':tds_c})
result2 = requests.post(firebase_url + '/' + reading_location + '/History/Parameter Reading.json', data=json.dumps(data))
print 'All parameter records are inserted.\nResult Code = ' + str(result2.status_code) + ',' + result2.text

I've used the requests module for this.
In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.
So you could do the following:
import requests
def is_connected():
try:
r = requests.get("http://google.com", timeout=5)
return True
except requests.exceptions.ConnectionError:
return False
Note that it may raise other exceptions, but this should be enough to start.

As suggested by #FranciscoCouzo, you can just try the connect and see what happens. But suppose you want a smaller sanity check before even delving into the database portion of your code. If you know the port number of your database server (1433 for instance) you can try a connect and then reset that connection. You still have to deal with loosing your wifi connection as you work, but this is a light weight way to know its okay to start.
import socket
import struct
def is_alive(host, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.settimeout(5)
s.connect((host, port))
s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
struct.pack("ii", 1, 0))
s.close()
except OSError:
return False
finally:
s.close()
return True
print(is_alive("example.com", 1344))

How to change Tor identity in Python?

I have the following script:
import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
socket.socket = socks.socksocket
import urllib2
print(urllib2.urlopen("http://www.ifconfig.me/ip").read())
which uses tor and SocksiPy
Now I want to change tor identity with each request, for example:
for i in range(0, 10):
#somehow change tor identity
print(urllib2.urlopen("http://www.ifconfig.me/ip").read())
How can I do this?

Tor wrote a new TOR control library in Python, stem. It can be found on PyPI. They provide some nice tutorials how to work with it, one of them explains how to change your identity:
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
Make sure your config is correct.

Today, I have searched a lot about this question, and finally managed to answer myself. But before I need to say that pirvoxy and tor should be configured correctly. First script, then a little bit about configuration:
import urllib2
from TorCtl import TorCtl
proxy_support = urllib2.ProxyHandler({"http" : "127.0.0.1:8118"})
opener = urllib2.build_opener(proxy_support)
def newId():
conn = TorCtl.connect(controlAddr="127.0.0.1", controlPort=9051, passphrase="your_password")
conn.send_signal("NEWNYM")
for i in range(0, 10):
print "case "+str(i+1)
newId()
proxy_support = urllib2.ProxyHandler({"http" : "127.0.0.1:8118"})
urllib2.install_opener(opener)
print(urllib2.urlopen("http://www.ifconfig.me/ip").read())
Above script gets new IP and checks it from ifconfig.me web site. About configuration:
We need Privoxy. to use TOR with HTTP connections, privoxy should work with tor. We can do it by adding thi to /etc/privoxy/config file:
forward-socks5 / localhost:9050 . #dot is important at the end
then we configure ControlPort in /etc/tor/torrc file. We need just uncomment this line:
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:872860B76453A77D60CA2BB8C1A7042072093276A3D701AD684053EC4C
then we just restart tor:
/etc/init.d/tor restart

Another simple solution, no external libraries required, works for both IPv4 and IPv6:
import socket
try:
tor_c = socket.create_connection((TOR_CTRL_HOST, TOR_CTRL_PORT))
tor_c.send('AUTHENTICATE "{}"\r\nSIGNAL NEWNYM\r\n'.format(TOR_CTRL_PWD))
response = tor_c.recv(1024)
if response != '250 OK\r\n250 OK\r\n':
sys.stderr.write('Unexpected response from Tor control port: {}\n'.format(response))
except Exception, e:
sys.stderr.write('Error connecting to Tor control port: {}\n'.format(repr(e)))

This is a video where im using STEM, SockSipy, Tor 100% working :)
#!/usr/bin/python
import socks
import socket
import time
from stem.control import Controller
from stem import Signal
import urllib2
import sys
def info():
print "[*] Welcome to Chart-Cheat Script"
print "[*] This script works with running TOR only"
print "[*] usage is chartcheat.py domain"
print "[*] argument domain must be in format www.example.com"
print "[*] Example: chartcheat.py www.example.com"
return
if len(sys.argv)==2:
info();
counter = 0
url = str(sys.argv[1]);
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
socket.socket = socks.socksocket
#visiting url in infinite loop
while True:
urllib2.urlopen("http://"+url)
counter=counter+1
print "Page " + url + " visited = " + str(counter)
#wait till next identity will be available
controller.signal(Signal.NEWNYM)
time.sleep(controller.get_newnym_wait())
else:
info();
In case you are running python3, urllib package in python3 will be the same as urllib2 package in python2.

You can enable tor control server by uncommenting few lines in
/etc/tor/torrc
And use stem library to send NEWNYM signal to change circuit.
controller.signal(Signal.NEWNYM)
You can read tutorial here.

you can write something like this :
def renew_connection():
with Controller.from_port(port=9051) as controller:
controller.authenticate(password='password')
controller.signal(Signal.NEWNYM)
controller.close()
def request_tor(url, headers):
renew_connection()
session = requests.session()
session.proxies = {}
session.proxies['http'] = 'socks5h://localhost:9050'
print((session.get(url)).text)

The following could work:
for i in range(0, 10):
#somehow change tor identity
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050+i)
socket.socket = socks.socksocket
print(urllib2.urlopen("http://www.ifconfig.me/ip").read())
You basically set set the proxy prior to making each connection. I am asuming that you have different proxies for different IPs since you have not stated how you intend to change the IP