I've got a little problem:
I have a String in my Database which is called actions.
Now, I'm writing a method, which gets that string from the database
(that works), and then I want to turn that string into a list.
I know actions.split(), but this didn't work out so well for me, because if my string looks like this:
actions = [
{u'action': 'visit_testing', u'timestamp': datetime.datetime(2016, 2, 12, 13, 32, 14)},
{u'action': 'visit_foo', u'timestamp': datetime.datetime(2016, 2, 12, 13, 37, 50)}
]
I can't use actions.split(', ') because it would mess up the dictionaries inside.
Till now I've got the following code:
timestamp = datetime.now().replace(microsecond=0)
dict = {'timestamp': timestamp, 'action': action}
if self.actions:
actions_string = str(self.actions)
actions_stripped = actions_string.strip('[')
actions_stripped = actions_stripped.strip(']')
actions_splitted = actions_stripped.split(', ')
new_action_list = []
buffer = ''
for string in actions_splitted:
if '{' in string:
buffer = str(string)
elif '}' in string:
buffer = buffer + ', ' + str(string)
new_action_list.append(str(buffer))
buffer = ''
else:
buffer = buffer + ', ' + str(string)
self.actions = str(buffer)
self.last_action = datetime.now().replace(microsecond=0)
self.save()
else:
self.actions = '['+str(dict)+']'
self.last_action = datetime.now().replace(microsecond=0)
self.save()
Addition: If I run the method when actions is empty, it gives me a list with one dictionary, but if I run it when it already has something in it, if sets actions to "".
You should be using the json module to store valid JSON in your database. You can create a valid action list from that string using exec. But please beware that using exec or eval is a potentially dangerous practice.
import datetime
stuff = '''
actions = [{u'action': 'visit_testing', u'timestamp': datetime.datetime(2016, 2, 12, 13, 32, 14)}, {u'action': 'visit_foo', u'timestamp': datetime.datetime(2016, 2, 12, 13, 37, 50)}]
'''
exec(stuff)
print(actions)
print(actions[0]['timestamp'])
output
[{u'action': 'visit_testing', u'timestamp': datetime.datetime(2016, 2, 12, 13, 32, 14)}, {u'action': 'visit_foo', u'timestamp': datetime.datetime(2016, 2, 12, 13, 37, 50)}]
2016-02-12 13:32:14
Use json library.
import json
my_dict_or_list = json.loads(your_string)
then work with Python objects. You will gain so much time :-D
I found a way that works for me:
timestamp = datetime.datetime.now().replace(microsecond=0)
if self.actions:
new_dict = {"timestamp": timestamp, "action": action}
#tmp_actions = json.loads(self.actions)
tmp_actions = self.actions
exec(tmp_actions)
actions.append(new_dict)
self.actions = str('actions = '+str(actions))
self.last_action = datetime.datetime.now().replace(microsecond=0)
self.save()
else:
exec('''actions = ['''+str({"timestamp": timestamp, "action": action})+''']''')
self.actions = 'actions = '+str(actions)
self.last_action = datetime.datetime.now().replace(microsecond=0)
self.save()
Thanks for all the help.
Related
I am creating a Jarvis style screen and have pulled data from outlook for upcoming meetings that I wish to present on the screen.
The function pulls data from outlook and presents it in a list: -
event(Start=datetime.datetime(2020, 11, 30, 12, 30), Subject='meeting 1 description',
Duration=60)
event(Start=datetime.datetime(2020, 11, 30, 14, 0), Subject='meeting 2 description', Duration=60)
event(Start=datetime.datetime(2020, 12, 1, 8, 30), Subject='meeting 3 description', Duration=60)
event(Start=datetime.datetime(2020, 12, 1, 10, 15), Subject='meeting 4 description', Duration=45)
event(Start=datetime.datetime(2020, 12, 1, 11, 0), Subject='meeting 5 description ',
Duration=90)"
This is great, but what I want to do now is have this present as:
Start time = 'start time'
Subject = 'Meeting description'
Duration = 'duration of meeting'
Is there a way of slicing up the string in a list item and then pulling that into the code as I want it presented? Basically splitting the item in a list into component parts?
Here is the code that pulls the lists: -
def get_date(datestr):
try: # py3
adate = datetime.datetime.fromtimestamp(datestr.Start.timestamp())
except Exception:
adate = datetime.datetime.fromtimestamp(int(datestr.Start))
return adate
def getCalendarEntries(days=3, dateformat="%d/%m/%Y"):
Outlook = win32com.client.Dispatch("Outlook.Application")
ns = Outlook.GetNamespace("MAPI")
appointments = ns.GetDefaultFolder(9).Items
appointments.Sort("[Start]")
appointments.IncludeRecurrences = "True"
today = datetime.datetime.today()
begin = today.date().strftime(dateformat)
tomorrow = datetime.timedelta(days=days) + today
end = tomorrow.date().strftime(dateformat)
appointments = appointments.Restrict(
"[Start] >= '" + begin + "' AND [END] <= '" + end + "'")
events = []
for a in appointments:
adate = get_date(a)
events.append(event(adate, a.Subject, a.Duration))
return events
if __name__ == "__main__":
events = getCalendarEntries()"""
Thanks all,
Graeme
This maybe a bit hacky but the syntax for event in your string is the same as one would define a dictionary. So we can replace 'event' with 'dict' and call eval which basically evaluates a string as if it was Python code. so for example if you run this
import datetime
event_str = r"event(Start=datetime.datetime(2020, 11, 30, 12, 30), Subject='meeting 1 description', Duration=60)"
dict_str = event_str.replace('event','dict')
my_dict = eval(dict_str)
print(my_dict)
this will print
{'Start': datetime.datetime(2020, 11, 30, 12, 30), 'Subject': 'meeting 1 description', 'Duration': 60}
So my_dict will be a dictionary that you can pull various bits out of, such as my_dict['Start'] will give you the start (as datetime), etc
you would need to call this construct on each element of your events list, eg the following should create a list of dictionaries, one for each event
all_dicts = [eval(e.replace('event','dict')) for e in events]
of course you can save yourself all this trouble if you created dictionaries in the first place, so replace the relevant line in your loop with
events.append(dict(Start=adate, Subject=a.Subject, Duration=a.Duration))
and then use dict functionatility to get the fields via events[i]['Start'] etc
So this is what I came up with after your suggestion - worked like a treat - thank you so much :) You are a super star!
import win32com.client
import datetime
from collections import namedtuple
event = namedtuple("event", "Start Subject Duration")
def get_date(datestr):
try: # py3
adate = datetime.datetime.fromtimestamp(datestr.Start.timestamp())
except Exception:
adate = datetime.datetime.fromtimestamp(int(datestr.Start))
return adate
def getCalendarEntries(days=3, dateformat="%d/%m/%Y"):
Outlook = win32com.client.Dispatch("Outlook.Application")
ns = Outlook.GetNamespace("MAPI")
appointments = ns.GetDefaultFolder(9).Items
appointments.Sort("[Start]")
appointments.IncludeRecurrences = "True"
today = datetime.datetime.today()
begin = today.date().strftime(dateformat)
tomorrow = datetime.timedelta(days=days) + today
end = tomorrow.date().strftime(dateformat)
appointments = appointments.Restrict(
"[Start] >= '" + begin + "' AND [END] <= '" + end + "'")
events = []
for a in appointments:
adate = get_date(a)
events.append(dict(Start=adate, Subject=a.Subject, Duration=a.Duration))
return events
if __name__ == "__main__":
events = getCalendarEntries()
print ("Time:", events[1]['Start'])
print ("Subject:",events[1]['Subject'])
print ("Duration:",events[1]['Duration'])
I'm starting out on Python on a job (I'm used to R) where I have to get daily data from an API that returns the datetime and value (which is a certain number of listeners on a podcast) and then send that data to a bigquery database.
After I split up the date and time, I need to add a new column that indicates which program was playing in that moment. In other words:
if time is >= than 11:00 and <= 11:59 then add a 'program name' value to the row into the column 'program'.
I've ran into several problems, namely the fact that time has been split as strings (could be due to the fact that we use google data studio, which has extremely rigid datetime implementation).
How would you go about it?
if response.status_code == 200:
data = response.text
result = json.loads(data)
test = result
#Append Items
for k in test:
l = []
l.append(datetime.datetime.strptime(k["time"], "%Y-%m-%dT%H:%M:%S.%fZ").strftime("%Y-%m-%d"))
l.append(datetime.datetime.strptime(k["time"], "%Y-%m-%dT%H:%M:%S.%fZ").astimezone(pytz.timezone("America/Toronto")).strftime("%H:%M"))
l.append(k["value"])
You need to have a 'DB' of the programs timetable. See below.
Your loop will call the function below with the time value and you will have the program name.
import datetime
from collections import namedtuple
Program = namedtuple('Program', 'name start end')
PROGRAMS_DB = [Program('prog1', datetime.time(3, 0, 0), datetime.time(3, 59, 0)),
Program('prog2', datetime.time(18, 0, 0), datetime.time(18, 59, 0)),
Program('prog3', datetime.time(4, 0, 0), datetime.time(4, 59, 0))]
def get_program_name(time_val):
for program in PROGRAMS_DB:
if program.start <= time_val <= program.end:
return program.name
data_from_the_web = [{"time": "2019-02-19T18:10:00.000Z", "value": 413, "details": None},
{"time": "2019-02-19T15:12:00.000Z", "value": 213, "details": None}]
for entry in data_from_the_web:
t = datetime.datetime.strptime(entry["time"], "%Y-%m-%dT%H:%M:%S.%fZ").time()
entry['prog'] = get_program_name(t)
for entry in data_from_the_web:
print(entry)
Output
{'prog': 'prog2', 'details': None, 'value': 413, 'time': '2019-02-19T18:10:00.000Z'}
{'prog': None, 'details': None, 'value': 213, 'time': '2019-02-19T15:12:00.000Z'}
I am trying to convert the rows returned in a SQLAlchemy query to dictionaries. When I try to use the ._asdict() method, I am only getting a key-value pair for the first column in my results.
Is there something else I should do to create a key-value pair in the dictionary for all columns in the result row?
class Project(db.Model):
__tablename__ = 'entries'
id = db.Column(db.Integer, primary_key=True)
time_start = db.Column(db.DateTime(timezone=False))
time_end = db.Column(db.DateTime(timezone=False))
name = db.Column(db.String(256), nullable=True)
analyst = db.Column(db.String(256), nullable=True)
def __init__(id, time_start, time_end, project_name, analyst):
self.id = id
self.time_start = time_start
self.time_end = time_end
self.name = name
self.analyst = analyst
latest_projects = db.session.query((func.max(Project.time_end)), Project.analyst).group_by(Project.analyst)
for row in latest_projects.all():
print (row._asdict())
{'analyst': 'Bob'}
{'analyst': 'Jane'}
{'analyst': 'Fred'}
I was expecting to see results like this...
{'analyst': 'Bob', 'time_end': '(2018, 11, 21, 14, 55)'}
{'analyst': 'Jane', 'time_end': '(2017, 10, 21, 08, 00)'}
{'analyst': 'Fred', 'time_end': '(2016, 09, 06, 01, 35)'}
You haven't named the func.max() column, so there is no name to use as a key in the resulting dictionary. Aggregate function columns are not automatically named, even when aggregating a single column; that you based that column on on the time_end column doesn't matter here.
Give that column a label:
latest_projects = db.session.query(
func.max(Project.time_end).label('time_end'),
Project.analyst
).group_by(Project.analyst)
Demo:
>>> latest_projects = db.session.query(
... func.max(Project.time_end).label('time_end'),
... Project.analyst
... ).group_by(Project.analyst)
>>> for row in latest_projects.all():
... print (row._asdict())
...
{'time_end': datetime.datetime(2018, 11, 21, 14, 55), 'analyst': 'Bob'}
{'time_end': datetime.datetime(2016, 9, 6, 1, 35), 'analyst': 'Fred'}
{'time_end': datetime.datetime(2017, 10, 21, 8, 0), 'analyst': 'Jane'}
I have something like this where trade_date, effective_date and termination_date are date values:
tradedates = dict(((k, k.strftime('%Y-%m-%d'))
for k in (trade_date,effective_date,termination_date)))
I get this:
{datetime.date(2005, 7, 25): '2005-07-25',
datetime.datetime(2005, 7, 27, 11, 26, 38): '2005-07-27',
datetime.datetime(2010, 7, 26, 11, 26, 38): '2010-07-26'}
What I would like is:
{'trade_date':'2005-07-25','effective_date':'2005-07-27','termination_date':'2010-07-26'}
How do I achieve this?
Using vars:
>>> import datetime
>>>
>>> trade_date = datetime.date(2005, 7, 25)
>>> effective_date = datetime.datetime(2005, 7, 27, 11, 26, 38)
>>> termination_date = datetime.datetime(2010, 7, 26, 11, 26, 38)
>>>
>>> d = vars() # You can access the variable as d['name']
>>> tradedates = {
... name: d[name].strftime('%Y-%m-%d')
... for name in ('trade_date', 'effective_date', 'termination_date')
... }
>>> tradedates
{'effective_date': '2005-07-27', 'termination_date': '2010-07-26', 'trade_date': '2005-07-25'}
For something that size, I'd create the dict directly:
result = {
'trade_date': format(trade_date, '%Y-%m-%d'),
'effective_date': format(effective_date, '%Y-%m-%d'),
# etc....
}
I am not sure if I got your question right. But let me explain what I understood and my answer for that:
You know the variable names: trade_date,effective_date,termination_date
And they have data in them
You could easily do:
tradedates = dict()
for k in ('trade_date','effective_date','termination_date'):
tradedates[k] = eval(k).strftime('%Y-%m-%d') // eval will evaluate them as a variable name not as a string.
This will give you a final dict something like:
{
'trade_date': <date_string_according_to_the_format_above>
'effective_date': <date_string_according_to_the_format_above>
'termination_date': <date_string_according_to_the_format_above>
}
I'm using the collection.Counter function to create a dictionary of paths to its listing of its mime types in order. It's a great little module, however the Counter doesn't change it's values from path to path.
I have a dictionary called package_mime_types that each entry looks like this:
package_mime_types['/a/path/to/somewhere'] = [('text/plain'),('text/plain'),('application/msword')]...
As you can imagine, the values in that dictionary are very long. I'm trying to convert it to a listing like this:
package_mime_types['/a/path/to/somewhere'] = ['text/plain':780, 'application/msword':400, 'audio/mp3':30]
This is my little iteration that's supposed to do that:
for package_path, mime_types_list in package_mime_types.items():
c = collections.Counter(mime_types_list)
package_mime_types[package_path] = c
return package_mime_types
The end result works, but all the Counter arrays are the exact same for each path.
/path1/ relates to Counter({'text/plain': 2303, 'audio/x-wav': 90, 'text/html': 17, 'application/msword': 17, 'application/x-trash': 6, 'application/x-tar': 4, 'application/xml': 1, 'text/x-sh': 1})
/path2/ relates to Counter({'text/plain': 2303, 'audio/x-wav': 90, 'text/html': 17, 'application/msword': 17, 'application/x-trash': 6, 'application/x-tar': 4, 'application/xml': 1, 'text/x-sh': 1})
/path3/ relates to Counter({'text/plain': 2303, 'audio/x-wav': 90, 'text/html': 17, 'application/msword': 17, 'application/x-trash': 6, 'application/x-tar': 4, 'application/xml': 1, 'text/x-sh': 1})
/path4/ relates to Counter({'text/plain': 2303, 'audio/x-wav': 90, 'text/html': 17, 'application/msword': 17, 'application/x-trash': 6, 'application/x-tar': 4, 'application/xml': 1, 'text/x-sh': 1})
/path5/ relates to Counter({'text/plain': 2303, 'audio/x-wav': 90, 'text/html': 17, 'application/msword': 17, 'application/x-trash': 6, 'application/x-tar': 4, 'application/xml': 1, 'text/x-sh': 1})
Am I missing something with using the Counter?
I'm facepalming myself right now. It wasn't a problem with the Counter at all but rather the iteration I was doing to create the listing of the file types. I didn't make a new array each time the iteration was populating my dictionary. So all of the files were associated with each path.
def find_mimes(package_paths):
package_mime_types = {}
mime_types_list =[]
## Walking through directories looking for the mime types
for package_path in package_paths:
print(package_path, "is being walked through")
for root, dirs, files in os.walk(package_path, followlinks = True):
for file in files:
if mimetypes.guess_type(os.path.join(root, file)) != (None, None):
mime_types_list.append(mimetypes.guess_type(os.path.join(root, file))[0])
package_mime_types[package_path] = mime_types_list
See how mime_types_list is above the iteration? It was a static variable. Moving into the package_path loop fixed it.
def find_mimes(package_paths):
package_mime_types = {}
## Walking through directories looking for the mime types
for package_path in package_paths:
##Setting mime_types_list array back to empty for every path. (Duh)
##Now mime_types_list will be empty before the walking starts
mime_types_list =[]
print(package_path, "is being walked through")
for root, dirs, files in os.walk(package_path, followlinks = True):
for file in files:
if mimetypes.guess_type(os.path.join(root, file)) != (None, None):
mime_types_list.append(mimetypes.guess_type(os.path.join(root, file))[0])
package_mime_types[package_path] = mime_types_list