Hi I would like to save my python requests cookies into a text file as json like so:
{"Proxy": "Cookies"}
At the moment im using:
with open('data.txt') as f:
proxy_data = json.load(f)
r = requests.get(URL,proxies=proxies, cookies=proxy_data['127:000'])
proxy_data['127:000'] = dict(session.cookies)
with open('data.txt', 'w') as json_file:
json.dump(proxy_data, json_file)
My data.txt file looks like this after writing it:
{"127:000": "{\"__cf_bm\": \"628f4c77a5139148cf28961a73f4edffed52f8c1-1622925414-1800-AaLguAyS9Hfl8TXCaQYUjjZ5i3sDck15kn/vcPJCWZBxi+nM9xU0zd/GfyKJu6/3R+GQEFdk/DGHEV4mANvpG1UMvcsXA9G9l/AFNEnIknTv\", \"cf_clearance\": \"84bffd070fbafcb91edcfa82f8cc3ba5a60a618d-1622925414-0-150\", \"cf_chl_seq_b116a013f9e2c44\": \"d3ac4406fc56d30\"}"}
However I run into a number of errors here. How can I do the properly?
UPDATE
ERRORS I GET
Python\Python37\lib\site-packages\requests\cookies.py", line 524, in
cookiejar_from_dict
cookiejar.set_cookie(create_cookie(name, cookie_dict[name])) TypeError: string indices must be integers
Currently, the value of proxy_data["127:000"] is a string which you're passing into the cookies parameter.
with open("data.txt") as f:
proxy_data = json.load(f)
print(type(proxy_data["127:000"]))
# >> <class 'str'>
Two options:
Parse the value of proxy_data["127:000"] to a dictionary using json.loads
r = requests.get(URL, proxies=proxies, cookies=json.loads(proxy_data['127:000']))
Make sure to save the nested dicts correctly.
proxy_data = {"127:000": {"cookie_test": "value_test"}}
with open("data.txt", "w") as f:
f.write(json.dumps(proxy_data))
with open("data.txt", "r") as f:
proxy_data = json.loads(f.read())
print(type(proxy_data["127:000"]))
# >> <class 'dict'>
Related
I have a json file, and I need to read all of that json file content as String data. How can I read all the data and set a variable as a String for all of that content? Json file has blanks, new lines, special characters etc if it's neccesarry.
Thanks for your help!
import json
from ast import literal_eval
with open('<path_to_json_data>/json_data.txt') as f:
json_data = json.load(f) # dict object
print(json_data, type(json_data))
json_data_as_str = str(json_data) # dict-->str object
print(json_data_as_str, type(json_data_as_str))
data = literal_eval(json_data_as_str) # str-->dict object again
print(data, type(data))
Hope it helps
Simple as this example
import json
with open("path/to/json/filename.json", "r") as json_file:
data = json.load(json_file)
print(data)
dataStr = json.dumps(data)
print(dataStr)
use json.loads
import json
with open(file_name, "r") as fp:
as_string = str(json.loads(fp.read()))
I have a JSON file and i need to read it into dictionary or list without using and library.This is my file content.
{
"101":"Break and Enter Commercial",
"102":"Break and Enter Residential/Other",
"103":"Vehicle Collision or Pedestrian Struck (with Fatality)",
"104":"Vehicle Collision or Pedestrian Struck (with Injury)"
}
This is what i try
def read_codes(filename):
jsonData = {}
# empty list to append to it later
file = open(filename, "r")
for key in file:
print(key)
return jsonData
print(read_codes('codes.json'))
What about such way:
with open(file) as f:
your_dict = eval(f.read().replace('\n', ''))
You could open it as a text file. It would return you list, then filter the list as you require.
with open('file.json', 'r') as jsonFile:
json_obj = jsonFile.readlines()
json_obj = [(obj.rstrip()).lstrip()[:-1] for obj in json_obj[1:-1]]
print(json_obj)
am trying to write a loop that gets .json from an url via requests, then writes the .json to a .csv file. Then I need it to it over and over again until my list of names (.txt file) is finished(89 lines). I can't get it to go over the list, it just get the error:
AttributeError: module 'response' has no attribute 'append'
I canĀ“t find the issue, if I change 'response' to 'responses' I get also an error
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
OSError: [Errno 22] Invalid argument: "listan-['A..
I can't seem to find a loop fitting for my purpose. Since I am a total beginner of python I hope I can get some help here and learn more.
My code so far.
#Opens the file with pricelists
pricelists = []
with open('prislistor.txt', 'r') as f:
for i, line in enumerate(f):
pricelists.append(line.strip())
# build responses
responses = []
for pricelist in pricelists:
response.append(requests.get('https://api.example.com/3/prices/sublist/{}/'.format(pricelist), headers=headers))
#Format each response
fullData = []
for response in responses:
parsed = json.loads(response.text)
listan=(json.dumps(parsed, indent=4, sort_keys=True))
#Converts and creates a .csv file.
fullData.append(parsed['Prices'])
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
dw.writeheader()
for data in fullData:
dw = csv.DictWriter(outf, data[0].keys())
for row in data:
dw.writerow(row)
print ("The file list-{}.csv is created!".format(pricelists))
Can you make the below changes in the place where you are making the api call(import json library as well) and see?
import json
responses = []
for pricelist in pricelists:
response = requests.get('https://api.example.com/3/prices/sublist/{}/'.format(pricelist), headers=headers)
response_json = json.loads(response.text)
responses.append(response_json)
and the below code also should be in a loop which loops through items in pricelists
for pricelist in pricelists:
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
dw.writeheader()
for data in fullData:
dw = csv.DictWriter(outf, data[0].keys())
for row in data:
dw.writerow(row)
print ("The file list-{}.csv is created!".format(pricelists))
Finally got it working. Got a help from another questions I created here at the forum. #waynelpu
The misstake I did was to not put the code into a loop.
Here is the code that worked like a charm.
pricelists = []
with open('prislistor.txt', 'r') as f:
for i, line in enumerate(f): # from here on, a looping code block start with 8 spaces
pricelists = (line.strip())
# Keeps the indents
response = requests.get('https://api.example.se/3/prices/sublist/{}/'.format(pricelists), headers=headers)
#Formats it
parsed = json.loads(response.text)
listan=(json.dumps(parsed, indent=4, sort_keys=True))
#Converts and creates a .csv file.
data = parsed['Prices']
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
dw = csv.DictWriter(outf, data[0].keys())
dw.writeheader()
for row in data:
dw.writerow(row)
print ("The file list-{}.csv is created!".format(pricelists))
# codes here is outside the loop but still INSIDE the 'with' block, so you can still access f here
# codes here leaves all blocks
I want to take the json response data from a REST request and create an actual json file. I tried something like this, but it did not work. Essentially, it just prints the headers. Any suggestions?
params = {'f': 'json', 'where': '1=1', 'geometryType': 'esriGeometryPolygon', 'spatialRel': 'esriSpatialRelIntersects','outFields': '*', 'returnGeometry': 'true'}
r = requests.get('https://hazards.fema.gov/gis/nfhl/rest/services/CSLF/Prelim_CSLF/MapServer/3/query', params)
cslfJson = r.json()
path = r"C:/Workspace/Sandbox/ScratchTests/cslf.json"
file = open(path, 'w')
for line in cslfJson:
file.write(line + "\r\n")
file.close()
use the json module
my_data = json.loads(r.json())
# my_data is a dict mapping the JSON
with open(path, 'w') as f:
json.dump(my_data, f)
if you want, you can print in pretty way using the indent parameter
json.dump(my_data, f, indent=2)
I am trying to run this code but it creates error.
import json
import requests
import pprint
data = []
with open('data.txt') as o1:
for line in o1:
data.append(json.loads(line))
print(data)
print(" \n")
print(data)
url = 'http://xyz.abcdfx.in/devicedata'
body_json=json.dumps(data)
headers = {'Content-Type':'application/json'}
d = requests.post(url, data = body_json, headers=headers)
pprint.pprint(d.json())
it shows
Value Error: No json object could be Decoded
I am new to programming and not able to figure out what is the problem.
It seems like you are trying to parse the json file line by line, but the json objects may (and usually are) span more than one line. You need to have the entire file in order to parse it:
with open('data.txt') as o1:
data = json.loads(o1.read()) # read ALL the file and parse. no loops
print(data)
i solved my problem using this:
data =[]
with open('data.txt') as f:
for line in f:
data = json.loads(line)
print(data)
url = 'http://xyz.abcdfx.cn/devicedata'
body_json=json.dumps(data)
headers = {'Content-Type':'application/json'}
d = requests.post(url, data = body_json, headers=headers)
pprint.pprint(d.json())