In django_session table session_data is stored which is first pickled using pickle module of Python and then encoded in base64 by using base64 module of Python.
I got the decoded pickled session_data.
session_data from django_session table:
gAJ9cQEoVQ9fc2Vzc2lvbl9leHBpcnlxAksAVRJfYXV0aF91c2VyX2JhY2tlbmRxA1UpZGphbmdvLmNvbnRyaWIuYXV0aC5iYWNrZW5kcy5Nb2RlbEJhY2tlbmRxBFUNX2F1dGhfdXNlcl9pZHEFigECdS5iZmUwOWExOWI0YTZkN2M0NDc2MWVjZjQ5ZDU0YjNhZA==
after decoding it by base64.decode(session_data):
\x80\x02}q\x01(U\x0f_session_expiryq\x02K\x00U\x12_auth_user_backendq\x03U)django.contrib.auth.backends.ModelBackendq\x04U\r_auth_user_idq\x05\x8a\x01\x02u.bfe09a19b4a6d7c44761ecf49d54b3ad
I want to find out the value of auth_user_id from auth_user_idq\x05\x8a\x01\x02u.
I had trouble with Paulo's method (see my comment on his answer), so I ended up using this method from a scottbarnham.com blog post:
from django.contrib.sessions.models import Session
from django.contrib.auth.models import User
session_key = '8cae76c505f15432b48c8292a7dd0e54'
session = Session.objects.get(session_key=session_key)
uid = session.get_decoded().get('_auth_user_id')
user = User.objects.get(pk=uid)
print user.username, user.get_full_name(), user.email
NOTE: format changed since original answer, for 1.4 and above see the update below
import pickle
data = pickle.loads(base64.decode(session_data))
>>> print data
{'_auth_user_id': 2L, '_auth_user_backend': 'django.contrib.auth.backends.ModelBackend',
'_session_expiry': 0}
[update]
My base64.decode requires filename arguments, so then I tried base64.b64decode, but this returned "IndexError: list assignment index out of range".
I really don't know why I used the base64 module, I guess because the question featured it.
You can just use the str.decode method:
>>> pickle.loads(session_data.decode('base64'))
{'_auth_user_id': 2L, '_auth_user_backend': 'django.contrib.auth.backends.ModelBackend',
'_session_expiry': 0}
I found a work-around (see answer below), but I am curious why this doesn't work.
Loading pickled data from user sources (cookies) is a security risk, so the session_data format was changed since this question was answered (I should go after the specific issue in Django's bug tracker and link it here, but my pomodoro break is gone).
The format now (since Django 1.4) is "hash:json-object" where the first 40 byte hash is a crypto-signature and the rest is a JSON payload. For now you can ignore the hash (it allows checking if the data was not tampered by some cookie hacker).
>>> json.loads(session_data.decode('base64')[41:])
{u'_auth_user_backend': u'django.contrib.auth.backends.ModelBackend',
u'_auth_user_id': 1}
If you want to learn more about it and know how does encode or decode work, there are some relevant code.
By the way the version of Django that i use is 1.9.4.
django/contrib/sessions/backends/base.py
class SessionBase(object):
def _hash(self, value):
key_salt = "django.contrib.sessions" + self.__class__.__name__
return salted_hmac(key_salt, value).hexdigest()
def encode(self, session_dict):
"Returns the given session dictionary serialized and encoded as a string."
serialized = self.serializer().dumps(session_dict)
hash = self._hash(serialized)
return base64.b64encode(hash.encode() + b":" + serialized).decode('ascii')
def decode(self, session_data):
encoded_data = base64.b64decode(force_bytes(session_data))
try:
# could produce ValueError if there is no ':'
hash, serialized = encoded_data.split(b':', 1)
expected_hash = self._hash(serialized)
if not constant_time_compare(hash.decode(), expected_hash):
raise SuspiciousSession("Session data corrupted")
else:
return self.serializer().loads(serialized)
except Exception as e:
# ValueError, SuspiciousOperation, unpickling exceptions. If any of
# these happen, just return an empty dictionary (an empty session).
if isinstance(e, SuspiciousOperation):
logger = logging.getLogger('django.security.%s' %
e.__class__.__name__)
logger.warning(force_text(e))
return {}
django/contrib/sessions/serializer.py
class JSONSerializer(object):
"""
Simple wrapper around json to be used in signing.dumps and
signing.loads.
"""
def dumps(self, obj):
return json.dumps(obj, separators=(',', ':')).encode('latin-1')
def loads(self, data):
return json.loads(data.decode('latin-1'))
Let's focus on SessionBase's encode function.
Serialize the session dictionary to a json
create a hash salt
add the salt to serialized session , base64 the concatenation
So, decode is inverse.
We can simplify the decode function in the following code.
import json
import base64
session_data = 'YTUyYzY1MjUxNzE4MzMxZjNjODFiNjZmZmZmMzhhNmM2NWQzMTllMTp7ImNvdW50Ijo0fQ=='
encoded_data = base64.b64decode(session_data)
hash, serialized = encoded_data.split(b':', 1)
json.loads(serialized.decode('latin-1'))
And that what session.get_decoded() did.
from django.conf import settings
from django.contrib.auth.models import User
from django.utils.importlib import import_module
def get_user_from_sid(session_key):
django_session_engine = import_module(settings.SESSION_ENGINE)
session = django_session_engine.SessionStore(session_key)
uid = session.get('_auth_user_id')
return User.objects.get(id=uid)
I wanted to do this in pure Python with the latest version of DJango (2.05). This is what I did:
>>> import base64
>>> x = base64.b64decode('OWNkOGQxYjg4NzlkN2ZhOTc2NmU1ODY0NWMzZmQ4YjdhMzM4OTJhNjp7Im51bV92aXNpdHMiOjJ9')
>>> print(x)
b'9cd8d1b8879d7fa9766e58645c3fd8b7a33892a6:{"num_visits":2}'
>>> import json
>>> data = json.loads(x[41:])
>>> print(data)
{'num_visits': 2}
I just had to solve something like this on a Django install. I knew the ID (36) of the user and wanted to delete the session data for that specific user. I wanted to put this code out as a prototype to build from for finding a user in session data:
from django.contrib.sessions.models import Session
TARGET_USER = 36 # edit this to match target user.
TARGET_USER = str(TARGET_USER) # type found to be a string
for session in Session.objects.all():
raw_session= session.get_decoded()
uid = session.get_decoded().get('_auth_user_id')
if uid == TARGET_USER: # this could be a list also if multiple users
print(session)
# session.delete() # uncomment to delete session data associated with the user
Hope this helps anyone out there.
Related
hello i guess have problem with client and member config which config should i use as you can see i am inserting json as data when i call get_data it returns with no problem but when i try to use predicate-sql it gives me error "hazelcast.errors.HazelcastSerializationError: Exception from server: com.hazelcast.nio.serialization.HazelcastSerializationException: There is no suitable de-serializer for type -120. This exception is likely caused by differences in t
he serialization configuration between members or between clients and members."
#app.route('/insert_data/<database_name>/<collection_name>', methods=['POST'])
def insert_data(database_name, collection_name):
client = hazelcast.HazelcastClient(cluster_members=[
url
])
dbname_map = client.get_map(f"{database_name}-{collection_name}").blocking()
if request.json:
received_json_data = request.json
received_id = received_json_data["_id"]
del received_json_data["_id"]
dbname_map.put(received_id, received_json_data)
client.shutdown()
return jsonify()
else:
client.shutdown()
abort(400)
#app.route('/get_data/<database_name>/<collection_name>', methods=['GET'])
def get_all_data(database_name, collection_name):
client = hazelcast.HazelcastClient(cluster_members=[
url
])
dbname_map = client.get_map(f"{database_name}-{collection_name}").blocking()
entry_set = dbname_map.entry_set()
output = dict()
datas = []
for key, value in entry_set:
value['_id'] = key
output = value
datas.append(output)
client.shutdown()
return jsonify({"Result":datas})
#bp.route('/get_query/<database_name>/<collection_name>/<name>', methods=['GET'])
def get_query_result(database_name, collection_name,name):
client = hazelcast.HazelcastClient(cluster_members=[
url
])
predicate_map = client.get_map(f"{database_name}-{collection_name}").blocking()
predicate = and_(sql(f"name like {name}%"))
entry_set = predicate_map.values(predicate)
#entry_set = predicate_map.entry_set(predicate)
send_all_data = ""
for x in entry_set:
send_all_data += x.to_string()
send_all_data += "\n"
print(send_all_data)
# print("Retrieved %s values whose age is less than 30." % len(result))
# print("Entry is", result[0].to_string())
# value=predicate_map.get(70)
# print(value)
return jsonify()
i try to change hazelcast.xml according to hazelcast-full-example.xml but i can't start hazelcast after
the changes and do i really have to use serialization ? hazelcast version:4.1 python:3.9
This is most likely happening because you are putting entries of the type dictionary to the map, which is serialized by the pickle because you didn't specify a serializer for that and the client does not know how to handle that correctly, so it fallbacks to the default serializer. However, since pickle serialization is Python-specific, servers cannot deserialize it and throw such an exception.
There are possible solutions to that, see the https://hazelcast.readthedocs.io/en/stable/serialization.html chapter for details.
I think the most appropriate solution for your use case would be Portable serialization which does not require a configuration change or code on the server-side. See the https://hazelcast.readthedocs.io/en/stable/serialization.html#portable-serialization
BTW, client objects are quite heavyweight, so you shouldn't be creating them on demand like this. You can construct it once in your application and share and use it in your endpoints or business-logic code freely since it is thread-safe. The same applies to the map proxy you get from the client. It can also be re-used.
I have a requests.cookies.RequestCookieJar object which contains multiple cookies from different domain/path. How can I extract a cookies string for a particular domain/path following the rules mentioned in here?
For example
>>> r = requests.get("https://stackoverflow.com")
>>> print(r.cookies)
<RequestsCookieJar[<Cookie prov=4df137f9-848e-01c3-f01b-35ec61022540 for .stackoverflow.com/>]>
# the function I expect
>>> getCookies(r.cookies, "stackoverflow.com")
"prov=4df137f9-848e-01c3-f01b-35ec61022540"
>>> getCookies(r.cookies, "meta.stackoverflow.com")
"prov=4df137f9-848e-01c3-f01b-35ec61022540"
# meta.stackoverflow.com is also satisfied as it is subdomain of .stackoverflow.com
>>> getCookies(r.cookies, "google.com")
""
# r.cookies does not contains any cookie for google.com, so it return empty string
I think you need to work with a Python dictionary of the cookies. (See my comment above.)
def getCookies(cookie_jar, domain):
cookie_dict = cookie_jar.get_dict(domain=domain)
found = ['%s=%s' % (name, value) for (name, value) in cookie_dict.items()]
return ';'.join(found)
Your example:
>>> r = requests.get("https://stackoverflow.com")
>>> getCookies(r.cookies, ".stackoverflow.com")
"prov=4df137f9-848e-01c3-f01b-35ec61022540"
NEW ANSWER
Ok, so I still don't get exactly what it is you are trying to achieve.
If you want to extract the originating url from a requests.RequestCookieJar object (so that you could then check if there is a match with a given subdomain) that is (as far as I know) impossible.
However, you could off course do something like:
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import requests
import re
class getCookies():
def __init__(self, url):
self.cookiejar = requests.get(url).cookies
self.url = url
def check_domain(self, domain):
try:
base_domain = re.compile("(?<=\.).+\..+$").search(domain).group()
except AttributeError:
base_domain = domain
if base_domain in self.url:
print("\"prov=" + str(dict(self.cookiejar)["prov"]) + "\"")
else:
print("No cookies for " + domain + " in this jar!")
Then if you do:
new_instance = getCookies("https://stackoverflow.com")
You could then do:
new_instance.check_domain("meta.stackoverflow.com")
Which would give the output:
"prov=5d4fda78-d042-2ee9-9a85-f507df184094"
While:
new_instance.check_domain("google.com")
Would output:
"No cookies for google.com in this jar!"
Then, if you (if needed) fine-tune the regex & create a list of urls, you could first loop through the list to create many instances and save them in eg a list or dict. In a second loop you could check another list of urls to see if their cookies might be present in any of the instances.
OLD ANSWER
The docs you link to explain:
items()
Dict-like items() that returns a list of name-value
tuples from the jar. Allows client-code to call
dict(RequestsCookieJar) and get a vanilla python dict of key value
pairs.
I think what you are looking for is:
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import requests
def getCookies(url):
r = requests.get(url)
print("\"prov=" + str(dict(r.cookies)["prov"]) + "\"")
Now I can run it like this:
>>> getCookies("https://stackoverflow.com")
"prov=f7712c78-b489-ee5f-5e8f-93c85ca06475"
actually , when I just have the problem as you are. but when I access the Class Define
class RequestsCookieJar(cookielib.CookieJar, MutableMapping):
I found a func called def get_dict(self, domain=None, path=None):
you can simply write code like this
raw = "rawCookide"
print(len(cookie))
mycookie = SimpleCookie()
mycookie.load(raw)
UCookie={}
for key, morsel in mycookie.items():
UCookie[key] = morsel.value
The following code is not promised to be "forward compatible" because I am accessing attributes of classes that were intentionally hidden (kind of) by their authors; however, if you must get into the attributes of a cookie, take a look here:
import http.cookies
import requests
import json
import sys
import os
aresponse = requests.get('https://www.att.com')
requestscookiejar = aresponse.cookies
for cdomain,cooks in requestscookiejar._cookies.items():
for cpath, cookgrp in cooks.items():
for cname,cattribs in cookgrp.items():
print(cattribs.version)
print(cattribs.name)
print(cattribs.value)
print(cattribs.port)
print(cattribs.port_specified)
print(cattribs.domain)
print(cattribs.domain_specified)
print(cattribs.domain_initial_dot)
print(cattribs.path)
print(cattribs.path_specified)
print(cattribs.secure)
print(cattribs.expires)
print(cattribs.discard)
print(cattribs.comment)
print(cattribs.comment_url)
print(cattribs.rfc2109)
print(cattribs._rest)
When a person needs to access the simple attributes of cookies is it likely less complicated to go after the following way. This avoids the use of RequestsCookieJar. Here we construct a single SimpleCookie instance by reading from the headers attribute of a response object instead of the cookies attribute. The name SimpleCookie would seem to imply a single cookie but that isn't what a simple cookie is. Try it out:
import http.cookies
import requests
import json
import sys
import os
def parse_cookies(http_response):
cookie_grp = http.cookies.SimpleCookie()
for h,v in http_response.headers.items():
if 'set-cookie' in h.lower():
for cook in v.split(','):
cookie_grp.load(cook)
return cookie_grp
aresponse = requests.get('https://www.att.com')
cookies = parse_cookies(aresponse)
print(str(cookies))
You can get list of domains in ResponseCookieJar and then dump the cookies for each domain with the following code:
import requests
response = requests.get("https://stackoverflow.com")
cjar = response.cookies
for domain in cjar.list_domains():
print(f'Cookies for {domain}: {cjar.get_dict(domain=domain)}')
Outputs:
Cookies for domain .stackoverflow.com: {'prov': 'efe8c1b7-ddbd-4ad5-9060-89ea6c29479e'}
In this example, only one domain is listed. It would have multiple lines in output if there were cookies for multiple domains in the Jar.
For many usecases, the cookie jar can be serialized by simply ignoring domains by calling:
dCookies = cjar.get_dict()
We can easily extract cookies string for a particular domain/path using functions already available in requests lib.
import requests
from requests.models import Request
from requests.cookies import get_cookie_header
session = requests.session()
r1 = session.get("https://www.google.com")
r2 = session.get("https://stackoverflow.com")
cookie_header1 = get_cookie_header(session.cookies, Request(method="GET", url="https://www.google.com"))
# '1P_JAR=2022-02-19-18; NID=511=Hz9Mlgl7DtS4uhTqjGOEolNwzciYlUtspJYxQ0GWOfEm9u9x-_nJ1jpawixONmVuyua59DFBvpQZkPzNAeZdnJjwiB2ky4AEFYVV'
cookie_header2 = get_cookie_header(session.cookies, Request(method="GET", url="https://stackoverflow.com"))
# 'prov=883c41a4-603b-898c-1d14-26e30e3c8774'
Request is used to prepare a :class:PreparedRequest <PreparedRequest>, which is sent to the server.
What you need is get_dict() method
a_session = requests.Session()
a_session.get('https://google.com/')
session_cookies = a_session.cookies
cookies_dictionary = session_cookies.get_dict()
# Now just print it or convert to json
as_string = json.dumps(cookies_dictionary)
print(cookies_dictionary)
This is a duplicate to this question:
How to convert suds object to xml
But the question has not been answered: "totxt" is not an attribute on the Client class.
Unfortunately I lack of reputation to add comments. So I ask again:
Is there a way to convert a suds object to its xml?
I ask this because I already have a system that consumes wsdl files and sends data to a webservice. But now the customers want to alternatively store the XML as files (to import them later manually). So all I need are 2 methods for writing data: One writes to a webservice (implemented and tested), the other (not implemented yet) writes to files.
If only I could make something like this:
xml_as_string = My_suds_object.to_xml()
The following code is just an example and does not run. And it's not elegant. Doesn't matter. I hope you get the idea what I want to achieve:
I have the function "write_customer_obj_webservice" that works. Now I want to write the function "write_customer_obj_xml_file".
import suds
def get_customer_obj():
wsdl_url = r'file:C:/somepathhere/Customer.wsdl'
service_url = r'http://someiphere/Customer'
c = suds.client.Client(wsdl_url, location=service_url)
customer = c.factory.create("ns0:CustomerType")
return customer
def write_customer_obj_webservice(customer):
wsdl_url = r'file:C:/somepathhere/Customer.wsdl'
service_url = r'http://someiphere/Customer'
c = suds.client.Client(wsdl_url, location=service_url)
response = c.service.save(someparameters, None, None, customer)
return response
def write_customer_obj_xml_file(customer):
output_filename = r'C\temp\testxml'
# The following line is the problem. "to_xml" does not exist and I can't find a way to do it.
xml = customer.to_xml()
fo = open(output_filename, 'a')
try:
fo.write(xml)
except:
raise
else:
response = 'All ok'
finally:
fo.close()
return response
# Get the customer object always from the wsdl.
customer = get_customer_obj()
# Since customer is an object, setting it's attributes is very easy. There are very complex objects in this system.
customer.name = "Doe J."
customer.age = 42
# Write the new customer to a webservice or store it in a file for later proccessing
if later_processing:
response = write_customer_obj_xml_file(customer)
else:
response = write_customer_obj_webservice(customer)
I found a way that works for me. The trick is to create the Client with the option "nosend=True".
In the documentation it says:
nosend - Create the soap envelope but don't send. When specified, method invocation returns a RequestContext instead of sending it.
The RequestContext object has the attribute envelope. This is the XML as string.
Some pseudo code to illustrate:
c = suds.client.Client(url, nosend=True)
customer = c.factory.create("ns0:CustomerType")
customer.name = "Doe J."
customer.age = 42
response = c.service.save(someparameters, None, None, customer)
print response.envelope # This prints the XML string that would have been sent.
You have some issues in write_customer_obj_xml_file function:
Fix bad path:
output_filename = r'C:\temp\test.xml'
The following line is the problem. "to_xml" does not exist and I can't find a way to do it.
What's the type of customer? type(customer)?
xml = customer.to_xml() # to be continued...
Why mode='a'? ('a' => append, 'w' => create + write)
Use a with statement (file context manager).
with open(output_filename, 'w') as fo:
fo.write(xml)
Don't need to return a response string: use an exception manager. The exception to catch can be EnvironmentError.
Analyse
The following call:
customer = c.factory.create("ns0:CustomerType")
Construct a CustomerType on the fly, and return a CustomerType instance customer.
I think you can introspect your customer object, try the following:
vars(customer) # display the object attributes
help(customer) # display an extensive help about your instance
Another way is to try the WSDL URLs by hands, and see the XML results.
You may obtain the full description of your CustomerType object.
And then?
Then, with the attributes list, you can create your own XML. Use an XML template and fill it with the object attributes.
You may also found the magic function (to_xml) which do the job for you. But, not sure the XML format matches your need.
client = Client(url)
client.factory.create('somename')
# The last XML request by client
client.last_sent()
# The last XML response from Web Service
client.last_received()
I have written JSON validator in python using jsonschema module.
Its not validating the schema correctly. I am also using web based tool, http://jsonschemalint.com/ for validating.
I wanted to exactly have some thing similar. I am putting my code here please point out the things that I am mising.
from jsonschema import validate
import json
class jsonSchemaValidator(object):
def __init__(self, schema_file):
self.__schema_file = open(schema_file)
self.__json_schema_obj = json.load(self.__schema_file)
def validate(self, json_file):
json_data_obj = json.load(open(json_file))
try:
validate(json_data_obj, self.__json_schema_obj)
print 'The JSON is follows the schema'
except Exception, extraInfo:
print str(extraInfo)
data_file_path = 'C:\\Users\\LT-BPant\\Desktop\\Del\\Schema\\new schema\\sample_output\\'
schema_path = 'C:\\Users\\LT-BPant\\Desktop\\Del\\Schema\\new schema\\'
def main():
json_file = data_file_path + 'report.json'
schema = schema_path+ 'report_new.schema'
obj = jsonSchemaValidator(schema)
obj.validate(json_file)
main()
I have manually modified the json data but still I am getting JSON DATA follows the schema as oputput whereas the web based tool is correctly showing the difference.
How can I get a JSON Object in python from getting data via Google App Engine Datastore?
I've got model in datastore with following field:
id
key_name
object
userid
created
Now I want to get all objects for one user:
query = Model.all().filter('userid', user.user_id())
How can I create a JSON object from the query so that I can write it?
I want to get the data via AJAX call.
Not sure if you got the answer you were looking for, but did you mean how to parse the model (entry) data in the Query object directly into a JSON object? (At least that's what I've been searching for).
I wrote this to parse the entries from Query object into a list of JSON objects:
def gql_json_parser(query_obj):
result = []
for entry in query_obj:
result.append(dict([(p, unicode(getattr(entry, p))) for p in entry.properties()]))
return result
You can have your app respond to AJAX requests by encoding it with simplejson e.g.:
query_data = MyModel.all()
json_query_data = gql_json_parser(query_data)
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(simplejson.dumps(json_query_data))
Your app will return something like this:
[{'property1': 'value1', 'property2': 'value2'}, ...]
Let me know if this helps!
If I understood you correctly I have implemented a system that works something like this. It sounds like you want to store an arbitrary JSON object in a GAE datastore model. To do this you need to encode the JSON into a string of some sort on the way into the database and decode it from a string into a python datastructure on the way out. You will need to use a JSON coder/decoder to do this. I think the GAE infrastructure includes one. For example you could use a "wrapper class" to handle the encoding/decoding. Something along these lines...
class InnerClass(db.Model):
jsonText = db.TextProperty()
def parse(self):
return OuterClass(self)
class Wrapper:
def __init__(self, storage=None):
self.storage = storage
self.json = None
if storage is not None:
self.json = fromJsonString(storage.jsonText)
def put(self):
jsonText = ToJsonString(self.json)
if self.storage is None:
self.storage = InnerClass()
self.storage.jsonText = jsonText
self.storage.put()
Then always operate on parsed wrapper objects instead of the inner class
def getall():
all = db.GqlQuery("SELECT * FROM InnerClass")
for x in all:
yield x.parse()
(untested). See datastoreview.py for some model implementations that work like this.
I did the following to convert the google query object to json. I used the logic in jql_json_parser above as well except for the part where everything is converted to unicode. I want to preserve the data-types like integer, floats and null.
import json
class JSONEncoder(json.JSONEncoder):
def default(self, obj):
if hasattr(obj, 'isoformat'): #handles both date and datetime objects
return obj.isoformat()
else:
return json.JSONEncoder.default(self, obj)
class BaseResource(webapp2.RequestHandler):
def to_json(self, gql_object):
result = []
for item in gql_object:
result.append(dict([(p, getattr(item, p)) for p in item.properties()]))
return json.dumps(result, cls=JSONEncoder)
Now you can subclass BaseResource and call self.to_json on the gql_object