I am trying to use logging when try fails. I have a for loop for converting a string of date into datetime format.
For example, converting "03/05/2021" to 2021-05-03. However, there are typoed dates such as 03/052021. If the loop encounters such typoed date, I want it to create a log.
for id in range(1,items):
try:
dt_bd_lists.append(datetime.strptime(bd_lists[i+1], '%d/%m/%Y'))
#print(dt_bd_lists[id])
except:
dt_bd_lists.append(bd_lists[id+1])
#LOG_FILENAME = 'error_log'
#logging.basicConfig(
#filename=LOG_FILENAME,
#level=logging.ERROR
#)
#logging.error('Error processing line %(lineno)d for ID %d', id)
For logging message, I want to create, "Error processing (line number) for (ID)."
Unfortunately, I am getting logging error and am stuck. What would solve this issue?
This will not help you with the logging, I would need the error produced by the logging for that, but maybe you can "clean" your data.
By replacing all slashes / with empty strings "" you can ignore those typos. Then you need to adjust the format of the date to "%d%m%Y" and you are good to go (just remove the slashes).
from datetime import datetime
date_strings = ["03/05/2021", "03/052021"]
for date_string in date_strings:
# replace / with empty string
date_string = date_string.replace("/", "")
date_time_obj = datetime.strptime(date_string, '%d%m%Y')
print(date_time_obj)
Related
I receive a date value on a POST request, and it has this format 20220509T000000Z.
In the system where I am working on, sometimes users can send wrong date values like 2022059T000000Z and 2022509T000000Z. That date is saved as string on our database, but if it is incorrect we get issues later doing some calculations and displaying information.
So, I am looking for a way in Python to validate the string is a correct date format, something like this:
#parameter date is the string I receive on the POST
def validate(date):
if date is valid:
# a valid date would be 20220509T000000Z
return true
else:
# an incorrect date would be 2022509T000000Z
return 'Error, incorrect date values'
Thanks in advance
One way is to use dateutil parser in a try and except:
import dateutil
#parameter date is the string I receive on the POST
def validate(date):
try:
dateutil.parser.parse(date)
# a valid date would be 20220509T000000Z
return True
except Exception as e:
# an incorrect date would be 2022509T000000Z
return 'Error, incorrect date values'
I am trying to convert a datetime from an API that stores datetime values as UTC. I need to convert the datetime to my local time 'Pacific/Auckland'
The API I am using is Sunrise-Sunset https://sunrise-sunset.org/api
The specific location I am requesting is Christchurch, New Zealand https://sunrise-sunset.org/search?location=christchurch
import requests
api_url = 'https://api.sunrise-sunset.org/json?lat=-43.525650&lng=172.639847&formatted=0'
response = requests.get(api_url)
if response.status_code == 200:
sunset_today = response.json()['results']['sunset']
print(sunset_today) # outputs '2021-09-26T06:31:41+00:00'
I have searched StackOverflow and Google extensively, but cannot seem to find a solution that fits my needs.
The question I am asking is
How can I convert the UTC value to my local datetime ('Pacific/Auckland')?
FYI, I don't want to add bloat to the application, but from previous (unsuccessful) attempts at solving this problem I have already installed the tzlocal and pytz packages.
I am writing my application in Django 3.2.7 and have adjusted my settings.py TIME_ZONE = 'Pacific/Auckland'
Edit
When trying to convert the string to a datetime I get the following error.
time data '2021-09-26T06:31:41+00:00' does not match format '%Y-%m-%dT%H:%M:%S %Z'
sunset_today = response.json()['results']['sunset']
format = '%Y-%m-%dT%H:%M:%S %Z'
parsed_date = datetime.strptime(sunset_today, format)
print(parsed_date)
# ERROR: time data '2021-09-26T06:31:41+00:00' does not match format '%Y-%m-%dT%H:%M:%S %Z'*
To convert timezone aware string to python datetime easier to use fromisoformat, since you are getting ISO formatted string from API anyway:
import datetime
sunset_today = response.json()['results']['sunset']
parsed_date = datetime.datetime.fromisoformat(sunset_today)
# 2021-09-26 06:31:41+00:00
I solved the problem using the dateutil and pytz library
import requests
import pytz
from dateutil import parser
api_url = 'https://api.sunrise-sunset.org/json?lat=-43.525650&lng=172.639847&formatted=0'
response = requests.get(api_url)
nz_zone = pytz.timezone('Pacific/Auckland')
if response.status_code == 200:
sunset_today = response.json()['results']['sunset']
converted_date = parser.parse(sunset_today).astimezone(nz_zone)
print(converted_date) # outputs 2021-09-26 19:31:41+13:00
I am trying to write a script to GET project data from Insightly and post to 10000ft. Essentially, I want to take any newly created project in one system and create that same instance in another system. Both have the concept of a 'Project'
I am extremely new at this but I only to GET certain Project parameters in Insightly to pass into the other system (PROJECT_NAME, LINKS:ORGANIZATION_ID, DATE_CREATED_UTC) to name a few.
I plan to add logic to only POST projects with a DATE_CREATED_UTC > yesterday, but I am clueless on how to setup the script to grab the JSON strings and create python variables (JSON datestring to datetime). Here is my current code. I am simply just printing out some of the variables I require to get comfortable with the code.
import urllib, urllib2, json, requests, pprint, dateutil
from dateutil import parser
import base64
#Set the 'Project' URL
insightly_url = 'https://api.insight.ly/v2.1/projects'
insightly_key =
api_auth = base64.b64encode(insightly_key)
headers = {
'GET': insightly_url,
'Authorization': 'Basic ' + api_auth
}
req = urllib2.Request(insightly_url, None, headers)
response = urllib2.urlopen(req).read()
data = json.loads(response)
for project in data:
project_date = project['DATE_CREATED_UTC']
project_name = project['PROJECT_NAME']
print project_name + " " + project_date
Any help would be appreciated
Edits:
I have updated the previous code with the following:
for project in data:
project_date = datetime.datetime.strptime(project['DATE_CREATED_UTC'], '%Y-%m-%d %H:%M:%S').date()
if project_date > (datetime.date.today() - datetime.timedelta(days=1)):
print project_date
else:
print 'No New Project'
This returns every project that was created after yesterday, but now I need to isolate these projects and post them to the other system
Here is an example of returning a datetime object from a parsed string. We will use the datetime.strptime method to accomplish this. Here is a list of the format codes you can use to create a format string.
>>> from datetime import datetime
>>> date_string = '2014-03-04 22:30:55'
>>> format = '%Y-%m-%d %H:%M:%S'
>>> datetime.strptime(date_string, format)
datetime.datetime(2014, 3, 4, 22, 30, 55)
As you can see, the datetime.strptime method returns a datetime object.
When I create my logfile, I want the name to contain the datetime.
In Python you can get the current datetime as:
>>> from datetime import datetime
>>> datetime.now()
datetime.datetime(2012, 2, 3, 21, 35, 9, 559000)
The str version is
>>> str(datetime.now())
'2012-02-03 21:35:22.247000'
Not a very nice str to append to the logfile name! I would like my logfile to be something like:
mylogfile_21_35_03_02_2012.log
Is there something Python can do to make this easy? I am creating the log file as:
fh = logging.FileHandler("mylogfile" + datetimecomp + ".log")
You need datetime.strftime(), this allows you to format the timestamp using all of the directives of C's strftime(). In your specific case:
>>> datetime.now().strftime('mylogfile_%H_%M_%d_%m_%Y.log')
'mylogfile_08_48_04_02_2012.log'
You could also use a TimedRotatingFileHandler that will handle the date and the rollover every day (or whenever you want) for you.
from logging.handlers import TimedRotatingFileHandler
fh = TimedRotatingFileHandler('mylogfile', when='midnight')
By default the format will be depending on the rollover interval:
The system will save old log files by appending extensions to the filename. The extensions are date-and-time based, using the strftime format %Y-%m-%d_%H-%M-%S or a leading portion thereof, depending on the rollover interval.
But you can modify that as showed here, by doing something like:
from logging.handlers import TimedRotatingFileHandler
fh = TimedRotatingFileHandler('mylogfile', when='midnight')
fh.suffix = '%Y_%m_%d.log'
Yes. Have a look at the datetime API, in particular strftime.
from datetime import datetime
print datetime.now().strftime("%d_%m_%Y")
Another Solution using format():
#generates a date for a generic filename
import datetime
date_raw = datetime.datetime.now()
date_processed = "{}-{}-{}_{}-{}-{}".format(date_raw.year, date_raw.month,
date_raw.day, date_raw.hour, date_raw.minute, date_raw.second)
#example value: date_processed = 2020-1-7_17-17-48
I used this in my own project
edit: as I found out about f(ormatted)-strings, this would be another solution:
date_processed = f"{date_raw.year}-{date_raw.month}-{date_raw.day}_{date_raw.hour}-{date_raw.minute}-{date_raw.second}"
We can use datetime.now() to get current timestamp. Here is my code that I am using to create log file with timestamp -
import logging
from datetime import datetime
LOG_FILENAME = datetime.now().strftime('D:/log/logfile_%H_%M_%S_%d_%m_%Y.log')
for handler in logging.root.handlers[:]:
logging.root.removeHandler(handler)
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)
logging.info('Forecastiong Job Started...')
logging.debug('abc method started...')
from time import strftime
fh = logging.FileHandler(strftime("mylogfile_%H_%M_%m_%d_%Y.log"))
To print hour, minutes, day, month and year, use the following statement
from datetime import datetime
print datetime.now().strftime("%H_%M_%d_%m_%Y")
I have a script for deleting images older than a date.
Can I pass this date as an argument when I call to run the script?
Example: This script delete_images.py deletes images older than a date (YYYY-MM-DD)
python delete_images.py 2010-12-31
Script (works with a fixed date (xDate variable))
import os, glob, time
root = '/home/master/files/' # one specific folder
#root = 'D:\\Vacation\\*' # or all the subfolders too
# expiration date in the format YYYY-MM-DD
### I have to pass the date from the script ###
xDate = '2010-12-31'
print '-'*50
for folder in glob.glob(root):
print folder
# here .jpg image files, but could be .txt files or whatever
for image in glob.glob(folder + '/*.jpg'):
# retrieves the stats for the current jpeg image file
# the tuple element at index 8 is the last-modified-date
stats = os.stat(image)
# put the two dates into matching format
lastmodDate = time.localtime(stats[8])
expDate = time.strptime(xDate, '%Y-%m-%d')
print image, time.strftime("%m/%d/%y", lastmodDate)
# check if image-last-modified-date is outdated
if expDate > lastmodDate:
try:
print 'Removing', image, time.strftime("(older than %m/%d/%y)", expDate)
os.remove(image) # commented out for testing
except OSError:
print 'Could not remove', image
The quick but crude way is to use sys.argv.
import sys
xDate = sys.argv[1]
A more robust, extendable way is to use the argparse module:
import argparse
parser=argparse.ArgumentParser()
parser.add_argument('xDate')
args=parser.parse_args()
Then to access the user-supplied value you'd use args.xDate instead of xDate.
Using the argparse module you automatically get a help message for free when a user types
delete_images.py -h
It also gives a helpful error message if the user fails to supply the proper inputs.
You can also easily set up a default value for xDate, convert xDate into a datetime.date object, and, as they say on TV, "much, much more!".
I see later in you script you use
expDate = time.strptime(xDate, '%Y-%m-%d')
to convert the xDate string into a time tuple. You could do this with argparse so args.xDate is automatically a time tuple. For example,
import argparse
import time
def mkdate(datestr):
return time.strptime(datestr, '%Y-%m-%d')
parser=argparse.ArgumentParser()
parser.add_argument('xDate',type=mkdate)
args=parser.parse_args()
print(args.xDate)
when run like this:
% test.py 2000-1-1
yields
time.struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=-1)
PS. Whatever method you choose to use (sys.argv or argparse), it would be a good idea to pull
expDate = time.strptime(xDate, '%Y-%m-%d')
outside of the for-loop. Since the value of xDate never changes, you only need to compute expDate once.
Little bit more polish to unutbu's answer:
import argparse
import time
def mkdate(datestr):
try:
return time.strptime(datestr, '%Y-%m-%d')
except ValueError:
raise argparse.ArgumentTypeError(datestr + ' is not a proper date string')
parser=argparse.ArgumentParser()
parser.add_argument('xDate',type=mkdate)
args=parser.parse_args()
print(args.xDate)
The command line options can be accessed via the list sys.argv. So you can simply use
xDate = sys.argv[1]
(sys.argv[0] is the name of the current script.)
you can use runtime arguments for this approach. Please see following link: http://www.faqs.org/docs/diveintopython/kgp_commandline.html