We Keep Coding

Scripted requests to CloudFlare protected resource. Python

Firstly, I need to send requests to the API of the resource. The resource is protected by CloudFlare detecting system.
I cannot send any request. The response is 403 Forbidden.
No data, no cookies.
Firstly, the API request requires cookies in headers. I found the request that was sent to accept all cookies.
After that I copied the bash of request, switched off all headers and sent request through Insomnia. It returned me completed cookies.
When I send the identical request in code it returned me 403 Forbidden status and non-completed cookies.
After that I decided to check the difference between browser, Insomnia and Python requests. I used Fiddler4. Python request returned 200 OK when I send the request in Python and everything returns me completed cookies. But if I switch Fiddler off I have 403 status.
Result of sending Python request with Fiddler:
[200 OK]
__Host-ariregweb=g4q8icpEmmIdHnQmFSNsfHdNhLLsgISKE7XRzphrLiscb4pSIzrpRLm9aiLCHqXa; Domain=ariregister.rik.ee; HttpOnly; Path=/; SameSite=lax; Secure; __cf_bm=GIYOJNShc1kY79bPk7GG1U6T.jV6K4BG8DLQoc70NT8-1671428340-0-AcxNKFuuhvkmblvy/q4WPGEPezvLQUL8/k6NeylOmaX5awlf1L7eOWnc55DGMsyPzpv5YKUDL6w100KlzzjsJVE=; Domain=ariregister.rik.ee; expires=Mon, 19-Dec-22 06:09:00 GMT; HttpOnly; Path=/; SameSite=None; Secure; _cfuvid=0EIfdSb4ltpOgYzKuMthWVvN0x5L3kn_uCfhyvBEk34-1671428340155-0-604800000; Domain=ariregister.rik.ee; HttpOnly; Path=/; SameSite=None; Secure
Result of sending Python request without Fiddler:
[403 Forbidden]
__cf_bm=0h8pprOoE6nOtjrZt6MYx6l7_4DIxIPn1_BL_e7Um2s-1671430063-0-AbvFUsYR8fOTT0NnrO1B4lJVTziYD+x2pnPLx1IyGjsgC29mqBNk+9iXhw2b1ewJiL2Cyi/iaTUilt6uPIbrSnw=; Domain=ariregister.rik.ee; expires=Mon, 19-Dec-22 06:37:43 GMT; HttpOnly; Path=/; SameSite=None; Secure
Maybe someone had such kind of problem or has some ideas how to bypass it. Thanks a lot.

Same cookie received twice, how to handle in the client?

I am trying to write a client to some web app. At one point it sends the following Set-cookie:
JSESSIONID=1BDC39CBF91C299C3330963D1EEFE399; Path=; HttpOnly; Secure, JSESSIONID=E6FFF3B159AFB9575D47662FC70DC161; Path=/; Secure; HttpOnly, XSESSIONID=b163abe6-bd6c-4381-9f68-01eeaee15a6c; Path=/; Secure; HttpOnly
It is unclear to me what is the meaning of having the same cookie name twice and what I am supposed to send back to the server?
I checked it with the following python code:
import sys
if sys.version_info.major == 2:
from Cookie import SimpleCookie
else:
from http.cookies import SimpleCookie
set_cookie = "JSESSIONID=1BDC39CBF91C299C3330963D1EEFE399; Path=; HttpOnly; Secure, JSESSIONID=E6FFF3B159AFB9575D47662FC70DC161; Path=/; Secure; HttpOnly, XSESSIONID=b163abe6-bd6c-4381-9f68-01eeaee15a6c; Path=/; Secure; HttpOnly"
cookies = SimpleCookie()
cookies.load(set_cookie)
for name in cookies.keys():
print("{} = {}".format(name, cookies[name].value))
for field in ['secure', 'httponly', 'path']:
print(" {}: {}".format(field, cookies[name][field]))
The python 2 code shows only one of the duplicate keys. The python 3 version does not recognize this at all.
$ /usr/bin/python cookie.py
XSESSIONID = b163abe6-bd6c-4381-9f68-01eeaee15a6c
secure: True
httponly: True
path: /
JSESSIONID = E6FFF3B159AFB9575D47662FC70DC161
secure: True
httponly: True
path: /
$ python3 cookie.py
So I would like to understand what is the meaning of having the same key twice and what should be sent back to the server?
It would be also nice to understand why does the Python 3 library disregard the whole string. What do I need to do to fix it?

Accessing all cookies in the Flask test response

After I make a request with the Flask test client, I want to access the cookies that the server set. If I iterate over response.headers, I see multiple Set-Cookie headers, but if I do response.headers["Set-Cookie"], I only get one value. Additionally, the headers are unparsed strings that are hard to test.
response = client.get("/")
print(response.headers['Set-Cookie'])
'mycookie=value; Expires=Thu, 27-Jun-2019 13:42:19 GMT; Max-Age=1800; Path=/'
for item in response.headers:
print(item)
('Content-Type', 'application/javascript')
('Content-Length', '215')
('Set-Cookie', 'mycookie=value; Expires=Thu, 27-Jun-2019 13:42:19 GMT; Max-Age=1800; Path=/')
('Set-Cookie', 'mycookie2=another; Domain=.client.com; Expires=Sun, 04-Apr-2021 13:42:19 GMT; Max-Age=62208000; Path=/')
('Set-Cookie', 'mycookie3=something; Domain=.client.com; Expires=Thu, 04-Apr-2019 14:12:19 GMT; Max-Age=1800; Path=/')
Why does accessing the Set-Cookie header only give me one header? How can I access the cookies and their properties for testing?

response.headers is a MultiDict, which provides the getlist method to get all the values for a given key.
response.headers.getlist('Set-Cookie')
It might be more useful to examine the cookies the client has, rather than the specific raw Set-Cookie headers returned by a response. client.cookie_jar is a CookieJar instance, iterating over it yields Cookie instances. For example, to get the value of the cookie with the name "user_id":
client.post("/login")
cookie = next(
(cookie for cookie in client.cookie_jar if cookie.name == "user_id"),
None
)
assert cookie is not None
assert cookie.value == "4"

The previous answer guided me to a slightly alternate version depending on what you want to do with the cookie.
I tried using client.cookie_jar, but I was testing for a few "non-standard" attributes like HttpOnly and SameSite. The cookie returned from client.cookie_jar does not return them, so I instead inspect the Set-Cookie header:
from werkzeug.http import parse_cookie
cookies = response.headers.getlist('Set-Cookie')
cookie = next(
(cookie for cookie in cookies if expected_cookie_name in cookie),
None
)
assert cookie is not None
cookie_attrs = parse_cookie(cookie)
assert cookie_attrs[expected_cookie_name] == expected_cookie_value
assert 'Secure' in cookie_attrs
assert 'HttpOnly' in cookie_attrs
assert cookie_attrs['SameSite'] == 'Lax'

Python parsing set-cookie header

In PHP I send one cookie with secure and http only flags, and other without
setcookie("c2","value");
setcookie("c1","value", 0, "/", "", true, true);
It produces header
Set-Cookie: c2=value, c1=value; path=/; secure; httponly
In firebug I can see, this is OK (c1 secure flag is True, c2 is False)
I want to get which one of them is not using secure flag
My python code:
cookies = Cookie.SimpleCookie()
cookies.load(headers['set-cookie'])
print cookies
Output:
Set-Cookie: c1=value; Path=/\\r\\nSet-Cookie: c2=value
headers['set-cookie'] does contain original set-cookie header, it's ok
According to python documentation printing(handling as string) SimpleCookie instance should create set-cookie header. Point is, that something is missing after parsing original header.
Morsels also contains wrong values (secure and http only).
Is this some kind of misconfiguration or it's a bug in python library ?
Thanks :)

This might be a bit late but saw your question and thought you may still need help.
The code I use to read a cookie is:
import Cookie,os
def getCookieData():
try:
cookie = Cookie.SimpleCookie(os.environ["HTTP_COOKIE"])
session = cookie['usrSession'].value
return session
except (Cookie.CookieError, KeyError):
return None
My cookie string its something along the lines of:
Cookie: usrSession=12345
Hope this helps

python SimpleCookie distinguish error

When i use python SimpleCookie object to pick up cookie from http headers, some exception occurs:
cookiestr = "a_em=[BU]co|12345678-901234567[DG]; Expires=Sat, 31 Dec 2016 17:09:50 GMT; Domain=.somesite.com; Path=/"
C = Cookie.SimpleCookie()
C.load(cookiestr)
print C
the output is:
Set-Cookie: a_em=; Domain=.somesite.com; expires=Sat,; Path=/
the cookie value and the cookie expires time is error!
how should i solve this?

RFC format for expires should be:
Expires=Sat, 31-Dec-2016 17:09:50 GMT
The full string should be (note quotes)
cookiestr = 'a_em="[BU]co|12345678-901234567[DG]"; Expires=Sat, 31-Dec-2016 17:09:50 GMT; Domain=.somesite.com; Path=/'