We Keep Coding

Pandas Calculating Worked Duration and adding new column in end of existing csv

I am pretty new to pandas And I got certain data of employees which have start and end date in date, month and year which is basically a column having lists.
Here is my data format is as follows as extracted from csv column.
data =[
{
"starts_at":{
"day":1,
"month":8,
"year":2021
},
"ends_at":None
},
{
"starts_at":{
"day":1,
"month":9,
"year":2020
},
"ends_at":{
"day":30,
"month":4,
"year":2021
}
},
{
"starts_at":None,
"ends_at":{
"day":30,
"month":4,
"year":2021
}
}
]
basically If ends_at is None than user is working currently(ongoing) and if end date is specified than user has ended contract with company.
There are certain fault data like start_date is None and end_date have dates these are also things I considered but all by python way.
I was told to do by pandas way but I think I am missing techniques of pandas rather than using double for loops in n*2 time complexity.
here is how I made my hands dirty with the pythonic way
from datetime import datetime
from datetime import date
from dateutil import relativedelta as rdelta
today = date.today()
df =pd.read_csv('/home/navaraj/Downloads/profile-details.csv')
pd.set_option('display.max_colwidth', None)
df["experiences"] = df["experiences"].apply(eval)
print(df['experiences'])
for k in df["experiences"]:
for x in k:
starts=(x.get('starts_at'))
if starts is not None:
ends=(x.get('ends_at'))
end_date_day=end_date_month=end_date_year=None
status=None
if ends is None:
ends=today
status="On going"
else:
end_date_day=ends['day']
end_date_month=ends['month']
end_date_year=ends['year']
ends=datetime.strptime(f'{end_date_year}-{end_date_month}-{end_date_day}', '%Y-%m-%d').date()
status="ended"
starts_day=starts['day']
starts_month=starts['month']
starts_year=starts['year']
started=datetime.strptime(f'{starts_year}-{starts_month}-{starts_day}', '%Y-%m-%d').date()
rd = rdelta.relativedelta(ends,started)
result="{0.years} years and {0.months} months".format(rd)
print(result,status)
Problem and expection:
I just wanted those last line of data i.e result and working status(Ongoing or Ended) to be attached at the end of current csv file for every users that I am working any Help will be really awesome.

Assuming 'attached at the end' means at the end of each row, defining a function to use in a call to pd.apply is one pandas way to avoid explicitly looping over the df.
Below is for 'status', but the same can be done for 'result'
function
def determine_status(row: pd.Series) -> str:
if row['starts_at']:
if row['ends_at']:
return "ended"
return "On going"
applying to the df with this call, setting a new column equal to the result
df.loc[:,'status'] = df.apply(determine_status, axis=1)
output
starts_at ends_at status
0 {'day': 1, 'month': 8, 'year': 2021} None On going
1 {'day': 1, 'month': 9, 'year': 2020} {'day': 30, 'month': 4, 'year': 2021} ended
2 None {'day': 30, 'month': 4, 'year': 2021} None

TypeError: an integer is required (got type datetime.datetime) when trying to convert a datetime object within a dictionary to a string

So I'm working in Python 3.7.4 with user-inputted dates, that are stored by another program in a variables in a dictionary format. These can potentially be any date in the immediate past. For instance November 6th 2019 looks like this:
{'Select start date': datetime.datetime(2019, 11, 6, 0, 0)}
I don't care about the item label, but I'd like to convert this dictionary date to the format 06 Nov 2019 (which I know is strftime('%d %b %Y')) but I'm not sure how to make it understand that the above is a datetime object and do a conversion on it when it's actually a dictionary object, and without throwing up errors.
I've read a lot on here about this but almost all questions here just look at either today's date (so datetime.datetime.now() is used), or a specific date, rather than a user-inputted date that could vary, and lives inside a dictionary. I've already seen plenty of stuff like this:
import datetime
d = datetime.datetime(2019, 11, 06, 0, 0, 0, 000000)
d.strftime("%a %b %d %Y %H:%M:%S %z")
...but it doesn't seem to apply in exactly this case. Commands like strftime and strptime don't seem to work because of the dictionary format, and I can't use static examples like the above because the actual date that I want to convert won't always be the same. How can I make this work without going some crazy long way using string manipulation? I feel like there's a really easy solution to this that I'm missing.
Code example (that doesn't work):
import datetime
dic = {'Select start date': datetime.datetime(2019, 11, 7, 0, 0)}
for key, value in dic.items():
d = datetime.datetime(value)
d.strftime("%d %b %Y")
Produces the error:
TypeError: an integer is required (got type datetime.datetime)

I see now what is happening. The line d = datetime.datetime(value) is the issue, you are passing a datetime object to the datetime.datetime() method which is why you get the valueError. This is unnecessary because you are already getting a datetime object out of your dictionary, so there is no reason to construct a new one. Here's the fixed code:
import datetime
dic = {'Select start date': datetime.datetime(2019, 11, 7, 0, 0)}
for key, datetime_obj in dic.items():
datetime_str = datetime_obj.strftime("%d %b %Y")
print(datetime_str)
An easy way to debug this is to use type(). So for example:
for key, value in dic.items():
print(type(value))
will give you <classdatetime.datetime>. Looking up the documentation for datetime.datetime() will tell you that this method only accepts integers.

Convert isoformat to datetime

I have a problem with converting isoformat to datetime.
I have this:
[{'timex-value': '2019-W32T15:00', 'start': 18, 'end': 31, 'text': '3pm next week', 'type': 'TIME', 'value': '2019-W32T15:00'}]
How can I convert '2019-W32T15:00' to datetime format in python?

To get a date time object you would want something like this. However you you will not get a the correct answer because you do not have a day of the week.
import date time as dt
dt.datetime.strptime('2019-W32T15:00','%Y-W%WT%H:%M')
If you want a successful conversion to a date time then specify the first day of that week by adding a 1 to the timex-value string.
import date time as dt
dt.datetime.strptime('2019-W32T15:00'+'-1','%Y-W%WT%H:%M-%w')
Result: datetime.datetime(2019, 8, 12, 15, 0)

Converting dates in Python

I have dates in the form 26/11/2015. How can I convert them into the format 26-Nov-2015 and still keep them as dates and not strings?

Your question does not make much sense. If you keep them as dates, they have no format. The format is only manifested when you convert them to strings.
So the answer is: Store the dates as date (or datetime) objects, and use datetime.strftime with some specific format whenever you need them as a string:
>>> from datetime import date
>>> d = date(2016, 11, 26)
>>> d.strftime("%Y/%m/%d")
'2016/11/26'
>>> d.strftime("%d-%b-%Y")
'26-Nov-2016'
Conversely, use strptime to parse strings in different formats to dates:
>>> datetime.datetime.strptime("26-Nov-2015", "%d-%b-%Y")
datetime.datetime(2015, 11, 26, 0, 0)

from datetime import datetime
date = datetime.strptime('26/11/2015', '%d/%m/%Y')
print date.strftime("%d-%B-%Y")
In the above example, we are taking your input string 'dd/mm/yyyy' and turning it into a python datetime saving it to a variable called date (for future usage as per your request), and then printing it out in the format requested.

You want to use the datetime module I think. For example:
from datetime import date
a = date(2015, 11, 26)
a.strftime("%A %d of %B, %Y")
should give you 'Thursday 26 of November, 2015'
Or for your specific formatting request:
a.strftime("%d-%b-%Y") #'26-Nov-2015'
Hope this helps, good luck!

python parse java calendar to isodate

I've data like this.
startDateTime: {'timeZoneID': 'America/New_York', 'date': {'year': '2014', 'day': '29', 'month': '1'}, 'second': '0', 'hour': '12', 'minute': '0'}
This is just a representation for 1 attribute. Like this i've 5 other attributes. LastModified, created etc.
I wanted to derive this as ISO Date format yyyy-mm-dd hh:mi:ss. is this the right way for doing this?
def parse_date(datecol):
x=datecol;
y=str(x.get('date').get('year'))+'-'+str(x.get('date').get('month')).zfill(2)+'-'+str(x.get('date').get('day')).zfill(2)+' '+str(x.get('hour')).zfill(2)+':'+str(x.get('minute')).zfill(2)+':'+str(x.get('second')).zfill(2)
print y;
return;

That works, but I'd say it's cleaner to use the string formatting operator here:
def parse_date(c):
d = c["date"]
print "%04d-%02d-%02d %02d:%02d:%02d" % tuple(map(str, (d["year"], d["month"], d["day"], c["hour"], c["minute"], c["second"])))
Alternatively, you can use the time module to convert your fields into a Python time value, and then format that using strftime. Remember the time zone, though.