I am trying to convert a JSON file to an iCalendar file. My supervisor suggested using two functions convertTo(data) (which converts a JSON to a String) and convertFrom(data) (which converts a String to a JSON; I am not sure of the purpose of this function).
My current approach uses a lot of refactoring and multiple functions.
#returns a String
def __convert(data):
convStr = __convertTo(data)
convStr = __fields(convStr)
return convStr
#convert JSON to a String
def __convertTo(data):
str = "" + data
return str
#takes string arg (prev converted from JSON) to split it into useful info
def __fields(data)
#########
iCalStr = __iCalTemplate(title, dtStart, UID, remType, email)
return iCalStr
#
def __iCalTemplate(title, dtStart, UID, remType, email):
icsTempStr = "BEGIN:VEVENT\n
DTSTART:" + dtStart + "\nUID:" + UID + "\nDESCRIPTION:" + desc + "\nSUMMARY:" + title
if remType is not None
icsTempStr += "\nBEGIN:VALARM\nACTION:" + remType + "DESCRIPTION:This is an event reminder"
if remType is email
icsTempStr += "\nSUMMARY:Alarm notification\nATTENDEE:mailto:" + email
icsTempStr += "\nEND:VALARM"
return icsTempStr
Any hints or suggestions would be very helpful. I am fully aware that this code needs a LOT of work.
This isn't intended to be a complete answer, but as a longer tip.
There's a Python idiom that will be very helpful to you in building strings, especially potentially large ones. It's probably easier to see an example than explain:
>>> template = 'a value: {a}; b value: {b}'
>>> data = {'a': 'Spam', 'b': 'Eggs'}
>>> template.format(**data)
'a value: Spam; b value: Eggs'
This idiom has a number of advantages over string concatenation and could eliminate the need for a function altogether if you write the template correctly. Optional inserts could, for example, be given values of ''. Once you format your iCal template correctly, it's just a matter of retrieving the right data points from JSON... and if you name your template insert points the same as what you have in JSON, you might even be able to do that conversion in one step. With a bit of planning, your final answer could be something as simple as:
import json
template = 'full iCal template with {insert_point} spec goes here'
data = json.JSONDecoder().decode(your_json_data)
ical = template.format(**data)
To do a quick (and slightly different) interpreter example:
>>> import json
>>> decoder = json.JSONDecoder()
>>> json_example = '{"item_one" : "Spam", "item_two" : "Eggs"}'
>>> template = 'Item 1: {item_one}\nItem 2: {item_two}'
>>> print template.format(**decoder.decode(json_example))
Item 1: Spam
Item 2: Eggs
I ended up using a completely different, more efficient approach to accomplish this. In summary, my method traverses through a JSON, extracting each value from each field and manually places it in the appropriate place in an iCalendar template. It returns a string. Something like this...
def convert(self, json):
template = 'BEGIN:VEVENT\n'
template += 'DTSTART:%s\n' % json['event-start']
...
return template
Related
Im getting plaintext responses from an API like these:
So i would like to parse or pass those values to variables.
Example:
If the response is:
TD_OK
3213513
I would like to convert this to:
TD_Result = TD_OK
TD_Number = 3213513
I tried something like this, but did not work:
result = """
TD_EXISTS
23433395"""
result2 = []
for r in result:
result2.append(r)
TD_Result = result2[1]
TD_Number = result2[2]
print (TD_Result)
print (TD_Number)
Any idea about how to do that?
for r in result: -> for r in result.splitlines():
or
as #Matmarbon said, below will be better
result = """
TD_EXISTS
23433395
"""
td_result, td_number = result.split()
print(td_result)
print(td_number)
get rid of unnecessary dict
use Extended Iterable Unpacking
use snake_case to comply with its naming convention
You can do this using the split method as follows.
Also note that list indexes in Python start at zero instead of one.
result = """
TD_EXISTS
23433395"""
result2 = result.split()
TD_Result = result2[0]
TD_Number = result2[1]
print (TD_Result)
print (TD_Number)
I have a web app that searches through a few databases, some of the data saved is in uppercase and some a mix of upper and lowercase, however when searching the keyword I want it to ignore the case and just bring up results that match the word. for example I want to search "document_reference" without having to write the proper way it is saved which is "Document_Reference"
I was told to add case insensitivity in my index hwoever not sure what to do or add in there,
I tried this (found it in whoosh documentation)
class CaseSensitivizer(analysis.Filter):
def __call__(self, tokens):
for t in tokens:
yield t
if t.mode == "index":
low = t.text.lower()
if low != t.text:
t.text = low
yield t
this what my index and query parser looks like
def open_index(indexDirectory):
# open index and return a idex object
ix = index.open_dir(indexDirectory)
return ix
def search_index(srch, ix):
# Search the index and print results
# ix = open_index(indexDirectory)
results = ''
lst = []
qp = MultifieldParser(['Text', 'colname',
'tblname', 'Length', 'DataType', 'tag_name'],
schema=ix.schema, group=qparser.OrGroup)
# qp = QueryParser('Text', schema=ix.schema)
q = qp.parse(srch)
with ix.searcher() as s:
results = s.search(q, limit=None)
for r in results:
print('\n', r)
lst.append(r.fields())
if(DEBUG):
print('Search Results:\n', lst)
print('\nFinished in search.py')
return lst
currently it only ever gives results that exactly match what I typed in search bar, so If I type "document" but the source is actually stored as "DOCUMENT" I wouldnt get any results
I know this is an older issue but thought would reply in case somebody like me came here looking for a solution.
The CaseSensitivizer class needs to be used when you define your schema. This is how you would use it to create the schema from the quickstart example from the docs
>>> from whoosh.index import create_in
>>> from whoosh.fields import *
>>> from whoosh import analysis
>>> class CaseSensitivizer(analysis.Filter):
def __call__(self, tokens):
for t in tokens:
yield t
if t.mode == "index":
low = t.text.lower()
if low != t.text:
t.text = low
yield t
>>> myanalyzer = analysis.RegexTokenizer() | CaseSensitivizer()
>>> schema = Schema(title=TEXT(stored=True), path=ID(stored=True), content=TEXT(analyzer=myanalyzer))
Now you can use this schema to create your index and do what you were doing before to search. That worked for me.
instead of using lower() or upper(), you can use casefold() for string comparison.
A very good example given here.
In short, example is:
s1 = 'Apple'
s3 = 'aPPle'
s1.casefold() == s3.casefold()
returns True.
import requests
import json
import csv
# These our are demo API keys, you can use them!
#location = ""
api_key = 'simplyrets'
api_secret = 'simplyrets'
#api_url = 'https://api.simplyrets.com/properties?q=%s&limit=1' % (location)
api_url = 'https://api.simplyrets.com/properties'
response = requests.get(api_url, auth=(api_key, api_secret))
response.raise_for_status()
houseData = json.loads(response.text)
#different parameters we need to know
p = houseData['property']
roof = p["roof"]
cooling = p["cooling"]
style = p["style"]
area = p["area"]
bathsFull = p["bathsFull"]
bathsHalf = p["bathsHalf"]
This is a snippet of the code that I am working with to try and take the information from the JSON provided by the API and put them into variables that I can actually use.
I thought that when you loaded it with json.loads() it would become a dictionary.
Yet it is telling me that I cannot do p = houseData['property'] because "list indices must be integers, not str".
Am I wrong that houseData should be a dictionary?
There are hundreds of properties returned, all of which are in a list.
You'll need to specify which property you want, so for the first one:
p = houseData[0]['property']
From https://docs.python.org/2/library/json.html :
json.loads(s[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]])
Deserialize s (a str or unicode instance containing a JSON document) to a Python object using this conversion table.
If s is a str instance and is encoded with an ASCII based encoding other than UTF-8 (e.g. latin-1), then an appropriate encoding name must be specified. Encodings that are not ASCII based (such as UCS-2) are not allowed and should be decoded to unicode first.
The other arguments have the same meaning as in load().
If your JSON starts as an array at the outermost layer, it will be an array. If your JSON's outermost layer is an associative array, then please post your JSON and we can look into it a little further.
The problem is that json.loads() doesn't necessarily return a dictionary. If the outside container of the JSON is a list, then json.loads() will return a list, where the elements could be lists or dictionaries. Try iterating through the list returned by json.loads(). It's possible the dictionary you're looking for is simply json.loads()[0] or some other element.
There are 2 different types of JSON elements: nodes and arrays.
A node looks like:
node = {
foo = 7
bar = "Hello World!"
}
A array looks like this:
array = [ "one", "two", 3, 4, "5ive" ]
Your JSON element is probably a array. You can verify whether it's an array, dict, or other by using:
isinstance(json_element, dict)
isinstance(json_element, list)
Hope this helps!
There are some minor changes you should do:
Your API response is returning a list, so you have to iterate over it.
The requests library already supports converting to JSON so you don't have to worry about it.
import requests
# These our are demo API keys, you can use them!
#location = ""
api_key = 'simplyrets'
api_secret = 'simplyrets'
#api_url = 'https://api.simplyrets.com/properties?q=%s&limit=1' % (location)
api_url = 'https://api.simplyrets.com/properties'
response = requests.get(api_url, auth=(api_key, api_secret))
response.raise_for_status()
houseData = response.json()
# different parameters we need to know
for data in houseData:
p = data['property']
roof = p["roof"]
cooling = p["cooling"]
style = p["style"]
area = p["area"]
bathsFull = p["bathsFull"]
bathsHalf = p["bathsHalf"]
If you want to make sure you will have only one result, do an if statement to check this.
if len(houseData) != 1:
raise ValueError("Expecting only 1 houseData.")
data = houseData[0]
...
How would I go about grabbing the values from the URL, a and b, then passing them to the add function? The result should come out to be: {"c": x}, where x is the sum of a and b.
The code:
op = 'add'
a = random.randint(0,10)
b = random.randint(0,10)
/%s?a=%s&b=%s' % (op, a, b)
result = res.json()
if op=='add':
assert a+b == result['c']
The function:
def add():
import json
return json.dumps({'c': ???})
Use urlparse, a standard library module designed for tasks like these!
urlparse.parse_qs(qs[, keep_blank_values[, strict_parsing]])
Parse a query string given as a string argument (data of type
application/x-www-form-urlencoded). Data are returned as a dictionary.
The dictionary keys are the unique query variable names and the values
are lists of values for each name.
The optional argument keep_blank_values is a flag indicating whether
blank values in percent-encoded queries should be treated as blank
strings. A true value indicates that blanks should be retained as
blank strings. The default false value indicates that blank values are
to be ignored and treated as if they were not included.
The optional argument strict_parsing is a flag indicating what to do
with parsing errors. If false (the default), errors are silently
ignored. If true, errors raise a ValueError exception.
Example:
>>> urlparse.parse_qs('a=1&b=1') # raw query string
{'a': ['1'], 'b': ['1']}
Note that you can parse an entire URL into its components (including a query string) using other functions in urlparse as well.
from urlparse import parse_qs, urlparse
import json
def add(a, b):
return json.dumps({'c': a+b})
url = 'add?a=1&b=1'
q_dict = parse_qs(urlparse(url).query, keep_blank_values=True)
# q_dict = {'a': ['1'], 'b': ['1']}
print add(a=q_dict['a'][0], b=q_dict['b'][0])
I figured out how to solve my problem! I appreciate all the help, but none of the solutions provided here applied to my question. This is entirely my fault because I worded the question totally different.
I am currently in the process of using python to transmit a python dictionary from one raspberry pi to another over a 433Mhz link, using virtual wire (vw.py) to send data.
The issue with vw.py is that data being sent is in string format.
I am successfully receiving the data on PI_no2, and now I am trying to reformat the data so it can be placed back in a dictionary.
I have created a small snippet to test with, and created a temporary string in the same format it is received as from vw.py
So far I have successfully split the string at the colon, and I am now trying to get rid of the double quotes, without much success.
my_status = {}
#temp is in the format the data is recieved
temp = "'mycode':['1','2','firstname','Lastname']"
key,value = temp.split(':')
print key
print value
key = key.replace("'",'')
value = value.replace("'",'')
my_status.update({key:value})
print my_status
Gives the result
'mycode'
['1','2','firstname','Lastname']
{'mycode': '[1,2,firstname,Lastname]'}
I require the value to be in the format
['1','2','firstname','Lastname']
but the strip gets rid of all the single speech marks.
You can use ast.literal_eval
import ast
temp = "'mycode':['1','2','firstname','Lastname']"
key,value = map(ast.literal_eval, temp.split(':'))
status = {key: value}
Will output
{'mycode': ['1', '2', 'firstname', 'Lastname']}
This shouldn't be hard to solve. What you need to do is strip away the [ ] in your list string, then split by ,. Once you've done this, iterate over the elements are add them to a list. Your code should look like this:
string = "[1,2,firstname,lastname]"
string = string.strip("[")
string = string.strip("]")
values = string.split(",")
final_list = []
for val in values:
final_list.append(val)
print final_list
This will return:
> ['1','2','firstname','lastname']
Then take this list and insert it into your dictionary:
d = {}
d['mycode'] = final_list
The advantage of this method is that you can handle each value independently. If you need to convert 1 and 2 to int then you'll be able to do that while leaving the other two as str.
Alternatively to cricket_007's suggestion of using a syntax tree parser - you're format is very similar to the standard yaml format. This is a pretty lightweight and intutive framework so I'll suggest it
a = "'mycode':['1','2','firstname','Lastname']"
print yaml.load(a.replace(":",": "))
# prints the dictionary {'mycode': ['1', '2', 'firstname', 'Lastname']}
The only thing that's different between your format and yaml is the colon needs a space
It also will distinguish between primitive data types for you, if that's important. Drop the quotes around 1 and 2 and it determines that they're numerical.
Tadhg McDonald-Jensen suggested pickling in the comments. This will allow you to store more complicated objects, though you may lose the human-readable format you've been experimenting with