I have some troubles getting the picture on my ip camera on python. I have an axis camera, I almost do the work on the rtsp link and cv2 video capture but when the hours go by I got an h264 error (here I asked for that problem).
So I decided to use a get request to get the picture, but now I got 401, error. Here is my code:
import requests
from requests.auth import HTTPBasicAuth
r = requests.get("http://xxx.xxx.xxx.xxx/jpg/image.jpg", auth=HTTPBasicAuth('xxx', 'xxx'))
print(r.status_code)
I also tried with out the HTTPBasicAuth but the same, I don't know how to get a good auth here.
Any help?
There is nothing wrong with your code. I have done the same code and works fine on my side. I would suggest you to verify the credentials that you have provided as a 401 response code is received when you provide wrong password or username.
Additionally, don't forget to pass the stream=True parameter inside the requests.get parameter otherwise the process will never successfully return anything even if the credentials actually work.
import requests
from requests.auth import HTTPBasicAuth
r = requests.get("http://xxx.xxx.xxx.xxx/jpg/image.jpg", auth=HTTPBasicAuth('xxx', 'xxx'), stream=True)
for streamDataChunks in r:
process_raw_image_data(streamDataChunks)
Related
I am trying to web-scrape data from https://www.mcmaster.com. They have provided me with a .pfx file and a passphrase. When making a GET request on Postman using their .json file, I input my website login/password and upload the .pfx certificate with its passphrase and everything works fine. Now I am trying to do this same thing but in Python, but am a bit unsure.
Here is my current Python code, I am unsure where I would put the website email/password login and how to successfully do a GET request.
import requests_pkcs12
from requests_pkcs12 import get
r = get('https://api.mcmaster.com/v1/login', pkcs12_filename='Schallert.pfx', pkcs12_password='mcmasterAPI#1901')
response = requests_pkcs12.get(r)
print(response.text)
Here is how I have it setup in Postman (Website email/pw login)
.PFX Certificate page
Postman has a built in feature where it will convert requests into code. You can do it like so:
On the far right click the Code Snippet Button (</>)
Once you are on that page, there is two available python options
Then all you need to do is copy the code into a Python file and add all your customizations (Should be already optimized)
One thing I’ll warn you about though is the URL. Postman doesn’t add
http:// or https:// to the URL, which means Python will throw a No
Scheme Supplied error.
Available Packages for Auto Conversion:
Requests
http.client
Meaning you will have to use a different package instead of requests_pkcs12
After a quick web search, it looks like you need to create a temporary certificate as a .pem file, which is then passed to the request.
from contextlib import contextmanager
from pathlib import Path
from tempfile import NamedTemporaryFile
import requests
from cryptography.hazmat.primitives.serialization import Encoding, PrivateFormat, NoEncryption
from cryptography.hazmat.primitives.serialization.pkcs12 import load_key_and_certificates
#contextmanager
def pfx_to_pem(pfx_path, pfx_password):
pfx = Path(pfx_path).read_bytes()
private_key, main_cert, add_certs = load_key_and_certificates(pfx, pfx_password.encode('utf-8'), None)
with NamedTemporaryFile(suffix='.pem') as t_pem:
with open(t_pem.name, 'wb') as pem_file:
pem_file.write(private_key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption()))
pem_file.write(main_cert.public_bytes(Encoding.PEM))
for ca in add_certs:
pem_file.write(ca.public_bytes(Encoding.PEM))
yield t_pem.name
with pfx_to_pem('your pfx file path', 'your passphrase') as cert:
requests.get(url, cert=cert, data=payload)
The package requests_pkcs12 is a wrapper written above the requests package. So, all the parameters that accept requests will accept by the requests_pkcs12.
Here is the source code proof for that. https://github.com/m-click/requests_pkcs12/blob/master/requests_pkcs12.py#L156
Also, from your screenshots, I understood that you are using POST not GET.
import json
from requests_pkcs12 import post
url = "https://api.mcmaster.com/v1/login"
payload = {'Username': 'yourusername',
'Password': 'yourpassword'}
resp = post(url, pkcs12_filename='Schallert.pfx', pkcs12_password='mcmasterAPI#1901', data=json.dumps(payload))
print(resp)
Footnote: I hope the password mcmasterAPI#1901 it's a fake one. if not please don't share any credentials in the platform.
I`m trying to get access to an area with a Network response
I`m using the following code, but it does not work.
import websocket
import json
from pprint import pprint
ws = websocket.WebSocket()
ws.connect("ws://localhost:9222/devtools/page/5EC90A588BEC2DA0229988D28BA67495")
ws.send(json.dumps({"method": "Network.getResponseBody", "id": })) # don`t know where i can find it
response = ws.recv()
pprint(response)
And I`m not sure if I do the right things.
So, does anybody know how to do this?
P.S. I know, I can make a direct request to API web-source and get JSON Object, but I need to make it with Chrome DevTools Protocol
Ok, I finally found it!
U just need to use the following method: "Network.getResponseBody"
I'm trying to use the Hasura API to get the contents of my database. The appropriate endpoint is v1alpha1/pg_dump.
I've tried doing the following in Python:
import requests
api_url = 'http://localhost:9695/v1alpha1/pg_dump'
header = {'Content-Type': 'application/json',
'x-hasura-admin-secret': 'MY_SECRET',
'X-Hasura-Role': 'admin'}
r = requests.post(url=api_url, headers=header)
If I do requests.get, I get information back (html code, although nothing particularly useful). However, if I do requests.post (which is required by Hasura: https://hasura.io/docs/1.0/graphql/core/api-reference/pgdump.html), I get a 404 error. I don't understand why. It's not an authentication error, but a page not found error.
Have I built my url incorrectly? Is there something I'm missing? The port is correct (and if I change it in the code, it gives me a different error telling me the port is invalid/closed). I'm not sure what else to change.
So, I have tried in my own Digital Ocean 1 click deployment environment. I have not secured it so I am not providing any headers. It works fine as follows:
import requests
import json
r = requests.post('http://address_of_hasura/v1alpha1/pg_dump',
data = json.dumps({
'opts' : ['-O', '-x', '--schema-only', '--schema', 'public'],
'clean_output': True
}) )
print r.text
If you have used the HASURA_GRAPHQL_ENABLED_APIS env variable and not included pgdump, that could be a reason it would be disabled.
Scope:
I am currently trying to write a Web scraper for this specific page. I have a pretty strong "Web Crawling" background using C#, but this httplib is beating me off.
Problem:
When trying to make a Http Get request for the page specified above I get a "Moved Permanently", that points to the very same URL. I can make a request using the requests lib, but I want to make it work using httplib so I can understand what I am doing wrong.
Code Sample:
I am completely new to Python, so any wrong language guideline or syntax is C#'s fault.
import httplib
# Wrapper for a "HTTP GET" Request
class HttpClient(object):
def HttpGet(self, url, host):
connection = httplib.HTTPConnection(host)
connection.request('GET', url)
return connection.getresponse().read()
# Using "HttpClient" class
httpclient = httpClient()
# This is the full URL I need to make a get request for : https://420101.com/strain-database
httpResponseText = httpclient.HttpGet('www.420101.com','/strain-database')
print httpResponseText
I really want to make it work using the httplib library, instead of requests or any other fancy one because I feel like I am missing something really small here.
The problem i've had too little or too much caffeine in my system.
To get a https, I needed the HTTPSConnection class.
Also, there is no 'www' in the address I wanted to GET. So, it shouldn't be included in the host.
Both of the wrong addresses redirect me to the correct one, with the 301 error code. If I were using requests or a more full featured module, it would have automatically followed the redirect.
My Validation:
c = httplib.HTTPSConnection('420101.com')
c.request("GET", "/strain-database")
r = c.getresponse()
print r.status, r.reason
200 OK
I am trying to generate the digest authorization header for use in a python test case. Because of the way the code base works, it is important that I am able to get the header as a string.
This header looks something like this
Authorization: Digest username="the_user", realm="my_realm", nonce="1389832695:d3c620a9e645420228c5c7da7d228f8c", uri="/some/uri", response="b9770bd8f1cf594dade72fe9abbb2f31"
I think my best bets are to use either urllib2 or the requests library.
With urllib2, I have gotten this far:
au=urllib2.HTTPDigestAuthHandler()
au.add_password("my_realm", "http://example.com/", "the_user", "the_password")
but I can't get the header out of that.
With requests, I have gotten this far:
requests.HTTPDigestAuth("the_user", "the_password")
But I when I try to use that, in a request, I am getting errors about setting the realm which I can't figure out how to do
If you're prepared to contort yourself around it, you can get the requests.auth.HTTPDigestAuth class to give you the right answer by doing something like this:
from requests.auth import HTTPDigestAuth
chal = {'realm': 'my_realm', 'nonce': '1389832695:d3c620a9e645420228c5c7da7d228f8c'}
a = HTTPDigestAuth('the_user', password)
a.chal = chal
print a.build_digest_header('GET', '/some/uri')
If I use 'the_password' as the user's password, that gives me this result:
Digest username="the_user", realm="my_realm", nonce="1389832695:d3c620a9e645420228c5c7da7d228f8c", uri="/some/uri", response="0b34daf411f3d9739538c7e7ee845e92"
When I tried #Lukasa answer I got an error:
'_thread._local' object has no attribute 'chal'
So I solved it in a slightly dirty way but it works:
from requests.auth import HTTPDigestAuth
chal = {'realm': 'my_realm',
'nonce': '1389832695:d3c620a9e645420228c5c7da7d228f8c'}
a = HTTPDigestAuth("username", "password")
a.init_per_thread_state()
a._thread_local.chal = chal
print(a.build_digest_header('GET', '/some/uri'))