We Keep Coding

How do I reach cookie information in python requests?

I am trying to write a small script that will allow me to see information related to the cookies set by my website.
I want to know if it has secure or httpOnly flags set on them. But so far I wasn't able to do it, I only figured out how to get cookie names and values. Here is my current code:
r = requests.post('url', data=data, headers=headers)
for (name, cookie) in r.cookies.items():
print name, cookie
So far this works fine, but I want to get information related to the cookies, not the value itself. Cookie meta-data if you will.
How can I achieve that?

You can extract the information from each cookie individually:
import requests
r = requests.post('http://www.about.com')
for cookie in r.cookies:
print(cookie.__dict__)
print(cookie.secure)
This is because r.cookies is an instance of RequestsCookieJar which extends from CookieJar (Python 2: cookielib.CookieJar, Python 3: http.cookiejar.CookieJar). A CookieJar has Cookie objects.
References:
cookielib: https://docs.python.org/2.7/library/cookielib.html
cookielib.Cookie.secure: https://docs.python.org/2.7/library/cookielib.html#cookielib.Cookie.secure
https://stackoverflow.com/a/27523891/295246
Update:
I have not found a way to retrieve the httponly value from a Cookie object. In Python 3, you can define a Morsel object via a dictionary, and it considers httponly to be a standard attribute of a cookie (https://docs.python.org/3/library/http.cookies.html), but I couldn't find any reference to httponly in the defining specification RFC2109 (https://www.ietf.org/rfc/rfc2109.txt).
That said, if httponly is in fact a non-standard attribute, then you can use the following to check if a cookie has it:
cookie.has_nonstandard_attr('httponly')

Under Python 3, I was not able to retrieve the httpOnly flag from the following:
cookie.get_nonstandard_attr('httpOnly')
and
cookie.has_nonstandard_attr('httpOnly')
returned False even if the httpOnly flag was included with the cookie.
This didn't work with any of the variations of httponly, HttpOnly, etc. either.
Using #HEADLESS_0NE's post, I found you can retrieve the flag by looking at the _rest field in cookie.__dict__. If httpOnly is included in the cookie,
cookie.__dict__['_rest']
will return something like this:
{'HttpOnly': None, ...}
Thus, here is a small helper function to check if a cookie has the httpOnly flag.
def has_http_only(cookie):
extra_args = cookie.__dict__.get('_rest')
if extra_args:
for key in extra_args.keys():
if key.lower() == 'httponly':
return True
return False
The secure flag is automatically added to the cookie object and can be retrieved using cookie.secure.

How do I clear cache with Python Requests?

Does the requests package of Python cache data by default?
For example,
import requests
resp = requests.get('https://some website')
Will the response be cached? If so, how do I clear it?

Add a 'Cache-Control: no-cache' header:
self.request = requests.get('http://google.com',
headers={'Cache-Control': 'no-cache'})
See https://stackoverflow.com/a/55613686/469045 for complete answer.

Late answer, but python requests doesn't cache requests, you should use the Cache-Control and Pragma headers instead, i.e.:
import requests
h = {
...
"Cache-Control": "no-cache",
"Pragma": "no-cache"
}
r = requests.get("url", headers=h)
...
HTTP/Headers
Cache-Control
The Cache-Control general-header field is used to specify directives for caching mechanisms in both requests and responses. Caching directives are unidirectional, meaning that a given directive in a request is not implying that the same directive is to be given in the response.
Pragma
Implementation-specific header that may have various effects anywhere
along the request-response chain. Used for backwards compatibility
with HTTP/1.0 caches where the Cache-Control header is not yet
present.
Directive
no-cache
Forces caches to submit the request to the origin server for
validation before releasing a cached copy.
Note on Pragma:
Pragma is not specified for HTTP responses and is therefore not a
reliable replacement for the general HTTP/1.1 Cache-Control header,
although it does behave the same as Cache-Control: no-cache, if the
Cache-Control header field is omitted in a request. Use Pragma only
for backwards compatibility with HTTP/1.0 clients.

Python-requests doesn't have any caching features.
However, if you need them you can look at requests-cache, although I never used it.

Requests does not do caching by default. You can easily plug it in by using something like CacheControl.

I was getting outdated version of a website and I thought about requests cache too, but adding no-cache parameter to headers didn't change anything. It appears that the cookie I was passing, was causing the server to present outdated site.

How to extract JSON data from a response containing a header and body?

this is my first question posed to Stack Overflow, because typically I can find the solutions to my problem here, but for this particular situation, I cannot. I am writing a Python plugin for my compiler that outputs REST calls in various languages for interaction with an API. I am authenticating with the socket and ssl modules by sending a username and password in the request body in JSON form. Upon successful authentication, the API returns a response in the following format with important response data in the body:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Date: Tue, 05 Feb 2013 03:36:18 GMT
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST,OPTIONS,GET
Access-Control-Allow-Headers: Content-Type
Server: Restlet-Framework/2.0m5
Content-Type: text/plain;charset=ISO-8859-1
Content-Length: 94
{"authentication-token":"<token>","authentication-secret":"<secret>"}
This is probably a very elementary question for Pythonistas, given its powerful tools for String manipulation. But alas, I am a new programmer who started with Java. I would like to know what would be the best way to parse this entire response to obtain the "<token>" and "<secret>"? Should I use a search for a "{" and dump the substring into a json object? My intuition is telling me to try and use the re module, but I cannot seem to figure out how it would be used in this situation, since the pattern of the token and secret are obviously not predictable. Because I have opted to authenticate with a low-level module set, this response is one big String obtained by constructing the header and appending JSON data to it in the body, then executing the request and obtaining the response with the following code:
#Socket configuration and connection execution
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn = ssl.wrap_socket(sock, ca_certs = pem_file)
conn.connect((host, port))
conn.send(req)
response = conn.recv()
print(response)
The print statement outputs the first code sample. Any help or insight would be greatly appreciated!

HTTP headers are split from the rest of the body by a \r\n\r\n sequence. Do something like:
import json
...
(headers, js) = response.split("\r\n\r\n")
data = json.loads(js)
token = data["authentication-token"]
secret = data["authentication-secret"]
You'll probably want to check the response, etc, and various libraries (e.g. requests) can do all of this a whole lot easier for you.

setting and reading cookies in mod_python (apache)

I've seen plenty of things explaining how to read and write cookies, however, I have no clue about how to do it in mod_python in apache. I tried putting this at the start of my HTML code, but it says to put it in the HTTP header. How do I do that?
Also, how do I retrieve them? I was originally looking mainly at this site:
http://webpython.codepoint.net/cgi_set_the_cookie
My code currently starts like this (and it displays as part of the HTML)
Content-Type: text/html
Set-Cookie: test=1
<html>
<head>

mod_python is not CGI, and provides it's own way to set and read cookies:
from mod_python import Cookie, apache
import time
def handler(req):
# read a cookie
spam_cookie = get_cookie(req, 'spam')
# set a cookie
egg_cookie = Cookie.Cookie('eggs', 'spam')
egg_cookie.expires = time.time() + 300
Cookie.add_cookie(req, egg_cookie)
req.write('<html><head></head><body>There's a cookie</body></html>')
return apache.OK
You'll find more documentation here : http://www.modpython.org/live/current/doc-html/pyapi-cookie.html

Really weird Cookie header behaviour? - Cookies

I'm using Firefox 3.6.8 for these tests.
I'm setting a cookie within the response headers of my web app using:
Set-Cookie: session=7878dfdsfjsdf89sd89f8df9
This does not seem to override the session Cookie.
When a request is performed instead Firefox even sends duplicate cookies:
Cookie: session=7d75cd8f55895cbccb0d31ee07c7afc0;
session=671e8448a5cebda0442005a186cf69a3;
4cb6f2d75c9ffc8916cb55bcbaafecd8
What is going on??
Any ideas would be great!! =)
This is quite disastrous in my case... if someone could explain what's going on it would really help me out!

If you don't specify the path or domain for a cookie when setting it, it defaults to the current path and current hostname. If you then go ahead and try setting the same cookie name from a URL with a different path or hostname, it will add a new cookie instead of replacing the old one.
I suspect what you want to do is just set a cookie with a global path for your site and for your entire domain. So something like this:
Set-Cookie: session=7878dfdsfjsdf89sd89f8df9; path=/; domain=.mysite.com

You can delete the previous cookie using the response object.
response.delete_cookie(cookie_key)
The set of cookies is available via the request object in the request.COOKIES dictionary, and you can obtain the key from there.
Since you're using Django, here's how you might do this in the view function:
def my_view(request):
# do some work and create a response object
response = HttpResponse(some_content)
# first delete any previously set cookie named "session"
if 'session' in request.COOKIES:
response.delete_cookie('session')
# set the new cookie
response.set_cookie('session', <cookie value goes here>')
return response