We Keep Coding

replace the date section of a string in python

if I have a string 'Tpsawd_20220320_default_economic_v5_0.xls'.
I want to replace the date part (20220320) with a date variable (i.e if I define the date = 20220410, it will replace 20220320 with this date). How should I do it with build-in python package? Please note the date location in the string can vary. it might be 'Tpsawd_default_economic_v5_0_20220320.xls' or 'Tpsawd_default_economic_20220320_v5_0.xls'

Yes, this can be done with regex fairly easily~
import re
s = 'Tpsawd_20220320_default_economic_v5_0.xls'
date = '20220410'
s = re.sub(r'\d{8}', date, s)
print(s)
Output:
Tpsawd_20220410_default_economic_v5_0.xls
This will replace the first time 8 numbers in a row are found with the given string, in this case date.

Python Pandas: convert DD.MM.YYYY to datetime

I have a DataFrame that looks like this:
date | ...
09.01.2000 |
02.03.2001 | ...
The format is DD.MM.YYYY. I want to select only data where it is year 2014 (for example). I convert them into datetime using pd.to_datetime:
date = pd.to_datetime(table["date"], format = "%d.%m.%Y")
Then I want my table to take 3 new columns Day, Month, Year. I do:
table[["day", "month", "year"]] = date.dt.day, date.dt.month, date.dt.year
But it throws error: Must have equal len keys and value when setting with an ndarray
How do I convert them into datetime properly to use df.loc[df["year"] == 2014]?

You need to do manually:
table['day'], table['month'], table['year'] = date.dt.day, date.dt.month, date.dt.year
Explanation: date.dt.day, date.dt.month, date.dt.year is short hand for a tuple with length 3, which is most likely different from len(table).
On the other hand, since you already have date, you can also slice with:
table[date.dt.year==2014]

Pandas sets datetime to first day of month if missing day?

When I used Pandas to convert my datetime string, it sets it to the first day of the month if the day is missing.
For example:
pd.to_datetime('2017-06')
OUT[]: Timestamp('2017-06-01 00:00:00')
Is there a way to have it use the 15th (middle) day of the month?
EDIT:
I only want it to use day 15 if the day is missing, otherwise use the actual date - so offsetting all values by 15 won't work.

While this isn't possible using the actual call, you could always use regex matching to figure out if the string contains a date and proceed accordingly. Note: this code only works if using '-' delimited dates:
import re
date_str = '2017-06'
if (not bool(re.match('.+-.+-.+',date_str))):
pd.to_datetime(date_str).replace(date=15)
else:
pd.to_datetime(date_str)

Manipulating timestamp so that the new timestamp is still valid

I have a pandas dataframe in which each cell of a column contains a timestamp, saved as a string:
>>>dataset['DateTime'][1]
'2018-03-14 00:34:46'
I would like to create a new column in which those dates are manipulated in the following way:
year += 1,
month += 2,
day += 3,
hour += 4,
minute += 5,
second += 6
(Important to this manipulation is that the initial date and the new date have a one-to-one relation, so that I can transform the date back later onwards)
In my case, the output I am looking for is as follows:
>>> dataset['newTimestamp'][1]
'2019-05-17 04:39:52'
To do so I am using the datetime library to create datetime objects (as a test, I have started with one variable first):
timestamp = dataset['DateTime'][1]
p = datetime.datetime.strptime(timestamp, "%Y-%m-%d %H:%M:%S")
Currently I am doing arithmetics on the single variables:
year = p.year + 1
if p.month < 12:
month = p.month + 1
else:
month = 1
year += 1
However, as with the months, there are exceptions when you can and when you cannnot add values so that the new timestamp is still a real timestamp (12 + 1 = 13, which is not an actual month).
I could program every rule explicitly, but that seems too much work and I expect there are better ways. How could I do this faster?

Use DateOffset.
Also, have a look at relativedelta module for this kind of manipulations:
dataset['newTimestamp'] = pd.to_datetime(dataset['DateTime']) + pd.DateOffset(years=1, months=2, days=3, hours=4, minutes=5, seconds=6)

You should try out beautiful-date library:
pip install beautiful-date
And use it like so:
from beautiful_date import *
...
dataset['DateTime'].apply(lambda d: d + 1 * years + 2 * months + ... + 6 * seconds)
should do the trick.

strptime() and strftime() are the functions you are looking for.
Just go ahead and google the two fuctions. surely enough,you will be abe to solve the stated problem.
these can be used to directly manipulate date-time quantities

how to generate a range of yyyymm values? [duplicate]

This question already has answers here:
Generate a list of datetimes between an interval
(5 answers)
Closed 8 years ago.
I have two yyyymm values that will be input by a user:
yyyymm_1 = '201406'
yyyymm_2 = '201501'
I want to be able to iterate through this range in increasing month order:
for yyyy and mm in the range of yyyymm_1 to yyyymm_2
my_function( yyyy, mm )
How can this be done in python?
Update:
Ideally, the solution should be as simple as possible without requiring external libraries. I'm not looking for a generic date manipulation solution, but a solution to answer the specific question I have asked above.
I had seen lots of generic solutions before posting my question. However, being a python noob, couldn't see how to adapt them to my question:
Generate a list of datetimes between an interval
Iterating through a range of dates in Python
On that note, the other questions linked to from this page are much more generic. If you are looking to generate a range of yyyymm values, I urge you to look at the selected answer on this page.

Here's another rather simple variant, without even using datetime. Just split the date, calculate the 'total month', and iterate.
def to_month(yyyymm):
y, m = int(yyyymm[:4]), int(yyyymm[4:])
return y * 12 + m
def iter_months(start, end):
for month in range(to_month(start), to_month(end) + 1):
y, m = divmod(month-1, 12) # ugly fix to compensate
yield y, m + 1 # for 12 % 12 == 0
for y, m in iter_months('201406', '201501'):
print y, m
Output:
2014 6
2014 7
...
2014 12
2015 1
For output in the same yyyymm format, use print("%d%02d" % (y, m)).

You can do this using the builtin datetime module and the third party package dateutil.
The code first converts your strings to datetime.datetime objects using datetime.datetime.strptime. It then uses the relativedelta function from dateutil to create a period of one month that can be added to your datetimes.
Within the while loop you can either work with the datetime objects directly, or construct the month and year as strings using strftime, I've shown an example of both in print functions.
import datetime as dt
from dateutil.relativedelta import relativedelta
yyyymm_1 = '201406'
yyyymm_2 = '201501'
MONTH = relativedelta(months=+1)
fmt = '%Y%m'
date_1 = dt.datetime.strptime(yyyymm_1, fmt).date()
date_2 = dt.datetime.strptime(yyyymm_2, fmt).date()
d = date_1
while d <= date_2:
print(d)
print(d.strftime('%Y'), d.strftime('%m'))
d += MONTH