Id like to know what it takes to remove a comma from the last line within a for loop in python. When I run the script it gives me the below output(after code section). I want to remove the comma at the end of fourth line "{"{#MACRO}":"queue4"}," Please can someone help?
By the way if there is a better way to construct the block please share the ideas. I'm a beginner and like to learn. :)
Code:
import json
import urllib
import string
Url= "http://guest:guest#localhost:55672/api/queues"
Response = urllib.urlopen(Url)
Data = Response.read()
def Qlist(Name):
Text = ''' {{"{{#MACRO}}":"{Name}"}},'''.format(Name=Name)
print Text
X_json = json.loads(Data)
print '''{
"data":['''
for i in X_json:
VV = i['name']
Qlist(VV)
print ''']
}'''
Below is the Output:
{
"data":[
{"{#MACRO}":"queue1"},
{"{#MACRO}":"queue2"},
{"{#MACRO}":"queue3"},
{"{#MACRO}":"queue4"},
]
}
Thanks so much
You can modify your loop as follows.
# Create and initialize a dictionary (Associative Array)
# data['data'] is an empty list.
# Variable name (data in this case) can be anything you want.
# 'data' is a key. notice the quotations around. it's not a variable.
# I used 'data' as the key, becasue you wanted your final output to include that part.
data = {"data": []}
for i in X_json:
# We are not calling the data dictionary here.
# We are accessing the empty list we have created inside the `data` dict (above) using data['data'] syntax.
# We can use the append function to add an item to a list.
# We create a new dictionary for every `name` item found in your json array and
# append that new dictionary to the data['data'] list.
data['data'].append({"{#MACRO}": i['name']})
print(json.dumps(data))
# or print json.dumps(data, indent=True)
Read more about json.dumps() here. You can read more about python's list and dictionary here
Don't print inside Qlist - instead return a value; then you can join all returned values using comma as separator:
def Qlist(Name):
Text = ''' {{"{{#MACRO}}":"{Name}"}}'''.format(Name=Name)
return Text
print '''{
"data":[''' +
',\n'.join([ Qlist(i['name']) for i in X_json ]) +
''']
}'''
And anyway, using json.dumps is likely a better idea.
Related
I have a .js containing data used to plot points on a map. In the file, I have a single line var statement which contains nested dictionaries.
var json_Project = {"type":"FeatureCollection","name":"project","crs":{"type":"name","properties":{"name":"urn:ogc:def:crs:OGC:1.3:CRS84"}},"features":[{"type":"Feature","properties":{"ID":"001","field1":"someText","field2":"someText"},"geometry":{"type":"Point","coordinates":[-1.14,60.15]}},{"type":"Feature","properties":{"ID":"002","field1":"someText","field2":"someText"},"geometry":{"type":"Point","coordinates":[-1.14,60.15]}}]}
One of these contains the "features" key which stores a list of dictionaries for each point (see below for readability).
var json_Project = {"type":"FeatureCollection","name":"project","crs":{"type":"name","properties":{"name":"urn:ogc:def:crs:OGC:1.3:CRS84"}},"features":[
{"type":"Feature","properties":{"ID":"001","field1":"someText","field2":"someText"},"geometry":{"type":"Point","coordinates":[-1.14,60.15]}},
{"type":"Feature","properties":{"ID":"002","field1":"someText","field2":"someText"},"geometry":{"type":"Point","coordinates":[-1.14,60.15]}}]}
The "properties" for the first point has "ID":"001", second point has "ID":"001" etc.
I would like to insert a new property for each point with "INFORMATION:" and get the value from another dictionary using the keys from "ID".
info_dict = {'001': 'This is Project 1',
'002': 'This is Project 2'}
Specifically, I would like this new property to be inserted before "field2" so that it looks like:
var json_Project = {"type":"FeatureCollection","name":"project","crs":{"type":"name","properties":{"name":"urn:ogc:def:crs:OGC:1.3:CRS84"}},"features":[
{"type":"Feature","properties":{"ID":"001","field1":"someText","INFORMATION":"This is Project 1","field2":"someText"},"geometry":{"type":"Point","coordinates":[-1.14,60.15]}},
{"type":"Feature","properties":{"ID":"002","field1":"someText","INFORMATION":"This is Project 2","field2":"someText"},"geometry":{"type":"Point","coordinates":[-1.14,60.15]}}]}
My futile attempt involved converting the one-liner to a string and then extract everything after the "features:" string as that is unique in the file. But not sure how to add the new key/value pair.
with open('project.js') as f:
contents = f.read().split()
contents_toString = ''.join(contents)
new_contents = re.findall('(?<="features":\[).*$', contents_toString)
EDIT
Thanks to #TenaciousB, I can read the geojson file and add in the "INFORMATION" property with the correct values from the dictionary:
with open('project.js') as f:
contents = f.read()
x = json.loads(contents)
for y in x['features']:
key = y['properties']['ID']
description = dictionary[key]
y['properties']['INFORMATION'] = description
I am still unsure how to:
Remove var json_Project = programmatically before being read;
Place the "INFORMATION" property before "field2";
Re-insert var json_Project = , save and ensure the single-line format is retained.
I have a variable that looks like this:
data = {"add_content": {"errata_ids": [advisory]},"content_view_version_environments": [{"content_view_version_id": version_id}]}
I need to add single quotes to this variable , i.e. if I will assign the variables:
advisory and version_id and add the single quotes to data variable like this:
data = '{"add_content": {"errata_ids": ["RHSA-2017:1390"]},"content_view_version_environments": [{"content_view_version_id": 160}]}'
I am able to post to the API
I have tried to add the single quotes in variety of ways:
new_data = "'" + str(data) + "'"
>>> new_data
'\'{\'add_content\': {\'errata_ids\': [\'"RHSA-2017:1390"\']}, \'content_view_version_environments\': [{\'content_view_version_id\': \'160\'}]}\''
or using:
'"%s"'%(data)
and a few more ways.
How can I add the single quotes to the outer to the data variable before and after the opening { and closing }?
This is exactly what JSON does:
import json
new_data = json.dumps(data)
If in contrary of previous answers and comments you are not trying to convert to a json string then use string.format around the variables:
data = {"add_content": {"errata_ids": '[{}]'.format(advisory)},"content_view_version_environments": [{"content_view_version_id": '{}'.format(version_id)}]}
So I am trying to modify some json file and I want to be consistent with its own style. I managed to handle order of keys, separators, etc, but I can't print empty list across few lines (see diff image above).
Here's the code snippet I have for this task.
info = json.load(f, object_pairs_hook=OrderedDict)
# make some changes in info
f.seek(0)
f.write(json.dumps(info, separators=(',', ': '), indent=4))
f.truncate()
I thought about a workaround using .replace("[],", "[\n\n\t],"), but it's kinda dirty (and incorrect for nested stuff). Any better way to do this? (or I am missing something in the json module?)
You can update the data by adding a special string to every empty list. Then you might end up with a file containing this:
"configs": [
"MAGIC_VALUE_NOBODY_USES"
],
Then just remove the lines that contain "MAGIC_VALUE_NOBODY_USES".
You can modify the json module on your own, simply change json/encoder.py.
The function _make_iterencode is the key for the output, modify the local function _iterencode_list,
def _iterencode_list(lst, _current_indent_level):
if not lst:
if _indent is not None:
yield '[' + '\n' + (' ' * (_indent * _current_indent_level)) + ']'
else:
yield '[]'
return
# ...
I am trying to put data from a text file into an array. below is the array i am trying to create.
[("major",r,w,w,s,w,w,w,s), ("relative minor",r,w,s,w,w,s,w,w),
("harmonic minor",r,w,s,w,w,s,w+s,s)]
But instead when i use the text file and load the data from it I get below as my output. it should output as above, i realise i have to split it but i dont really know how for this sort of set array. could anyone help me with this
['("major",r,w,w,s,w,w,w,s), ("relative minor",r,w,s,w,w,s,w,w),
("harmonic minor",r,w,s,w,w,s,w+s,s)']
below is my text file I am trying to load.
("major",r,w,w,s,w,w,w,s), ("relative minor",r,w,s,w,w,s,w,w), ("harmonic minor",r,w,s,w,w,s,w+s,s)
And this is how im loading it
file = open("slide.txt", "r")
scale = [file.readline()]
If you mean a list instead of an array:
with open(filename) as f:
list_name = f.readlines()
Some questions come to mind about what the rest of your implementation looks like and how you figure it all will work, but below is an example of how this could be done in a pretty straight forward way:
class W(object):
pass
class S(object):
pass
class WS(W, S):
pass
class R(object):
pass
def main():
# separate parts that should become tuples eventually
text = str()
with open("data", "r") as fh:
text = fh.read()
parts = text.split("),")
# remove unwanted characters and whitespace
cleaned = list()
for part in parts:
part = part.replace('(', '')
part = part.replace(')', '')
cleaned.append(part.strip())
# convert text parts into tuples with actual data types
list_of_tuples = list()
for part in cleaned:
t = construct_tuple(part)
list_of_tuples.append(t)
# now use the data for something
print list_of_tuples
def construct_tuple(data):
t = tuple()
content = data.split(',')
for item in content:
t = t + (get_type(item),)
return t
# there needs to be some way to decide what type/object should be used:
def get_type(id):
type_mapping = {
'"harmonic minor"': 'harmonic minor',
'"major"': 'major',
'"relative minor"': 'relative minor',
's': S(),
'w': W(),
'w+s': WS(),
'r': R()
}
return type_mapping.get(id)
if __name__ == "__main__":
main()
This code makes some assumptions:
there is a file data with the content:
("major",r,w,w,s,w,w,w,s), ("relative minor",r,w,s,w,w,s,w,w), ("harmonic minor",r,w,s,w,w,s,w+s,s)
you want a list of tuples which contains the values.
It's acceptable to have w+s represented by some data type, as it would be difficult to have something like w+s appear inside a tuple without it being evaluated when the tuple is created. Another way to do it would be to have w and s represented by data types that can be used with +.
So even if this works, it might be a good idea to think about the format of the text file (if you have control of that), and see if it can be changed into something which would allow you to use some parsing library in a simple way, e.g. see how it could be more easily represented as csv or even turn it into json.
I am importing data from Mongo into a CSV file. The import consists of "timestamp" and "text" for each JSON Document.
The documents:
{
name: ...,
size: ...,
timestamp: ISODate("2013-01-09T21:04:12Z"),
data: { text:..., place:...},
other: ...
}
The code:
with open(output, 'w') as fp:
for r in db.hello.find(fields=['text', 'timestamp']):
print >>fp, '"%s","%s"' % (r['text'], r['timestamp'].strftime('%H:%M:%S'))
I would like to remove duplicates (some Mongo docs have the same text), and I would like to keep the first instance (with regards to the time) intact. Is it possible to remove these dupes as I import?
Thanks for your help!
I would use a set to store the hashes of the data, and check for duplicates. Something like this:
import md5
hashes = set()
with open(output, 'w') as fp:
for r in db.hello.find(fields=['text', 'timestamp']):
digest = md5.new(r['text']).digest()
if digest in hashes:
# It's a duplicate!
continue
else:
hashes.add(digest)
print >>fp, '"%s","%s"' % (r['text'], r['timestamp'].strftime('%H:%M:%S'))
It's worth noting that you could use the text field directly, but for larger text fields storing just the hash is much more memory efficient.
You just need to maintain a map (dictionary) to maintain (text, timestamp) pairs. The 'text' is the key, so there won't be any duplicates. I will assume the order of reading is not guaranteed to return the oldest timestamp first. In that case you will have to make 2 passes-- once for reading and later one pass for writing.
textmap = {}
def insert(text, ts):
global textmap
if text in textmap:
textmap[text] = min(ts, textmap[text])
else:
textmap[text] = ts
for r in db.hello.find(fields=['text', 'timestamp']):
insert(r['text'], r['timestamp'])
for text in textmap:
print >>fp, text, textmap[text] # with whatever format desired.
At the end, you can also easily convert the dictionary into list of tuples, in case you want to sort the results using timestamp before printing, for example.
(See Sort a Python dictionary by value )