We Keep Coding

Getting response 444 when making request to a webpage

I am trying to make a request to a webpage using the code below and i am getting response 444.
Is there anything i can to about it?
import requests
url = "https://www.pseudosite.com/"
response = requests.get(url)
print(response) # <Response [444]>
http.dev website says the following:
When the 444 No Response status code is generated, the server returns
no information to the client and closes the HTTP Connection. This
error message can be found in the nginx logs and will not be sent to
the client. It is useful for dealing with malicious HTTP requests,
such as one that includes an illegal Host header.
I am trying to webscrape that website using python, but I am blocked at first step.

I believe that you need to add headers to view this webpage. If you open devtools on your browser, then you should see a 'get' request; if you now click on the 'Headers' tab, you can create a dictionary with the data.

View headers and body of a POST request made by Python Script

In my application, I have my API that is in localhost:8000/api/v0.1/save_with_post.
I've also made a Python Script in order to do a Post Request on such Api.
### My script
import requests
url = 'localhost:8000/api/v0.1/save_with_post'
myobj = {'key': 'value'}
x = requests.post(url, data = myobj)
Is it possible to view headers and body of the request in Chrome rather than debugging my application code?

You want Postman.
With Postman you can either generate a request to your service from Postman itself, or set up Postman as a proxy so you can see the requests that your API client is generating and the responses from the server.

If you want to view the response headers from the post request, have you tried:
>>> x.headers
Or you could just add headers yourself to your POST request as so:
h = {"Content-Type": "application/xml", ("etc")}
x = requests.post(url, data = myobj, headers = h)

well, I don't know if there is a way you could view the request in Chrome DevTools directly (I don't think there is) however, I know there are two alternatives for seeing the request body and response:
1 - use selenium with a chrome webdriver
this will allow you to run chrome automated by python. then you can open a test page and run javascript in it to do your post request,
see this for more info on how to do this:
1 https://selenium-python.readthedocs.io/getting-started.html
2 Getting the return value of Javascript code in Selenium
you will need to use Selenium-requests library to use requests library with selenium
https://pypi.org/project/selenium-requests/3
2 - use Wireshark
this program will allow you to see all the traffic that is going on your network card and therefore you will be able to monitor all the requests going back and forth. however, Wireshark will throw all the traffic that you network card send or receives it may be hard to see the specific request you want

Generating a Cookie in Python Requests

I'm relatively new to Python so excuse any errors or misconceptions I may have. I've done hours and hours of research and have hit a stopping point.
I'm using the Requests library to pull data from a website that requires a login. I was initially successful logging in through through a session.post,(payload)/session.get. I had a [200] response. Once I tried to view the JSON data that was beyond the login, I hit a [403] response. Long story short, I can make it work by logging in through a browser and inspecting the web elements to find the current session cookie and then defining the headers in requests to pass along that exact cookie with session.get
My questions is...is it possible to set/generate/find this cookie through python after logging in? After logging in and out a few times, I can see that some of the components of the cookie remain the same but others do not. The website I'm using is garmin connect.
Any and all help is appreciated.

If your issue is about login purposes, then you can use a session object. It stores the corresponding cookies so you can make requests, and it generally handles the cookies for you. Here is an example:
s = requests.Session()
# all cookies received will be stored in the session object
s.post('http://www...',data=payload)
s.get('http://www...')
Furthermore, with the requests library, you can get a cookie from a response, like this:
url = 'http://example.com/some/cookie/setting/url'
r = requests.get(url)
r.cookies
But you can also give cookie back to the server on subsequent requests, like this:
url = 'http://httpbin.org/cookies'
cookies = dict(cookies_are='working')
r = requests.get(url, cookies=cookies)
I hope this helps!
Reference: How to use cookies in Python Requests

Authentication using API Request

I'm currently writing a short script using the Request API and I need to access my e-mail box in order to retrieve some information.
So I wrote this code in order to authenticate:
#! /usr/bin/python3
import requests
payload = {'horde_user': '',
'horde_pass': '',
'login_post': '1'}
r = requests.post('http://webmail.technion.ac.il/login.php', data=payload)
print(r.status_code)
print(r.text)
Obviously I replaced horde_user by my username and horde_pass by my password in the payload dictionary.
But the output is the source code of the initial page, while I expected it to be the content of my mail box.
And the status code is 200 while it is supposed to be 302. Could somebody explain to me what I'm doing wrong.
Thanks a lot for helping!

Try requesting the page first and then pulling out the session ids from that first request to be used in the login attempt. I bet it is failing because you're sending old session ids and the website is rejecting them and sending you back to the login page with a new session id. The session ids would be Horde and horde_secret_key in the cookies.

Cannot upload a file in python + selenium. Keep getting [{"error":true,"error_msg":"Authentication Failed"}]

I am badly stuck up and I'm not able to proceed any further. Trust me, I have looked all over the web to get a CONCRETE solution but all in vain!
I have an application justcloud.com where I need to upload a file and verify if it's uploaded.
After I login to this application and reach the page where I have the button to select a file to be uploaded, here is my code:
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
import urllib2
register_openers()
fileToUpload = {'files[]':open('/home/pinku/Desktop/Test_Upload.odt', 'rb')}
datagen, headers = multipart_encode(fileToUpload)
url = "https://upload.backupgrid.net/add"
request = urllib2.Request(url, datagen, headers)
print urllib2.urlopen(request).read()
The error I keep getting every single time is [{"error":true,"error_msg":"Authentication Failed"}]
I know I just need to simulate the actual file upload process which is nothing but an HTTP POST request to the server which also includes some authentication that I need to overcome.
My assumption is that may be cookies can help me in resolving this authentication issue but I am not sure and I do not know how to include it in my python code. Any sample code will be helpful.
I request anyone reading this to help me.
This is not my first time on stackoverflow when I have posted this question but I have not received much help. I am still giving it a shot. Thanks anyways...

The error message is telling you exactly what the problem is. You need to be logged into that website before you can POST a file to that URL.
You need to get a logged in session cookie somehow. Typically when you POST to a login form URL you will receive an http response that includes this cookie. Then every time you make another request you keep sending that cookie with each request so that the site is aware you are authenticated.
How would you login to backupgrid? Find the URL for that, login, grab the cookie from the response and then include it with your request.
This stackoverflow answer provides a good explanation of how to add cookies to the request.