i'm a python beginner and having an issue. Trying to pull API info and convert the extracted JSON time object to a datetime object in Python so I can run the date.weekday() function on it eventually (the overall goal is to extract all the dates from the API and see which the output by days - I plan on populating a empty dictionary once I can extract all the dates).
For some reason, even with my conditional statements, I'm still printing (2015, 04, 06) with all the zeroes. That's my problem.
I have a feeling I'm getting something basic wrong, and that there's also a better way to go about this than doing all the ifs/elses with the 0-padding in the date object.
here's my code so far:
from datetime import date
import datetime
import json
import requests
r = requests.get('https://api.github.com/repos/mbostock/d3/commits?since=2015-04-12330:00:000')
jsonoutput = r.json()
modified = (jsonoutput[0]["commit"]["author"]["date"])
#at this point, modified gives something like: "2015-04-06T22:28:16Z"
if modified[5] == 0:
if modified[8] == 0:
new_format = ("(" + modified[0:4] + ", " + modified[6] + ", " + modified[9] + ")")
#with the particular jsonoutput[0] pulled here, this one should be triggered
else:
new_format = ("(" + modified[0:4] + ", " + modified[6] + ", " + modified[8:10] + ")")
else:
if modified[8] == 0:
new_format = ("(" + modified[0:4] + ", " + modified[5:7] + ", " + modified[9] + ")")
else:
new_format = ("(" + modified[0:4] + ", " + modified[5:7] + ", " + modified[8:10] + ")")
print(new_format)
print(date.weekday(datetime.date(new_format)))
The error happens in your current code because new_format is defined as a string, while the datetime.date takes the arguments as integers. Also, since you compare the string "0" with the numerical 0, the modified is simply not created.
Instead of this:
new_format = ("(" + modified[0:4] + ", " + modified[6] + ", " + modified[8:10]) + ")")
do this:
new_format = (int(modified[0:4]), int(modified[5:7]), int(modified[8:10]))
The above will work with all of your cases, so you can remove the convoluted if-else blocks as well.
Alternatively, you could split this modeified string by "T", and then use another split by "-" to get the integer values:
new_format = map(int, modified.split("T")[0].split("-"))
You'll also need to unpack the list when passing as the argument, so your complete code becomes:
import json
import requests
from datetime import date
r = requests.get('https://api.github.com/repos/mbostock/d3/commits?since=2015-04-12330:00:000')
jsonoutput = r.json()
modified = jsonoutput[0]["commit"]["author"]["date"]
new_format = (int(modified[0:4]), int(modified[5:7]), int(modified[8:10]))
print(date.weekday(date(*new_format)))
Also, as others have already pointed out in their answers, it might be a better idea to dateutil.parser.parse than to write your own parsing logic. (dateutil is not a builtin package, you'll have to install it) :)
What you get from the json is actually a datetime representation in an ISO format
You can refer to this SO answer https://stackoverflow.com/a/15228038/58129 to convert the string
You're trying to make your own parsing functions where Python has its own.
from dateutil import parser
from datetime import date
my_date = dateutil.parser.parse(modified)
is_week_day = date.weekday(my_date)
If dateutil is not installed on your machine, try pip install python-dateutil
However, if you want to go with Python's standard library:
from datetime import date, datetime
from time import strptime
mytime = strptime(modified, '%Y-%m-%dT%H:%M:%SZ')
my_date = datetime(*my_time[:6])
is_week_day = date.weekday(my_date)
Related
I have a string that looks like
{'dir1/dir2/k8s-dag-model.py', 'dir1/dir2/[updated]airflow-issue.txt', 'dir1/dir2/1.log', 'dir1/dir2/cloud_formation.py', 'dir1/dir2/example_sftp_to_s3.py', 'dir1/dir2/catalog_sales.csv', 'dir1/dir2/dep-dremio0.sh', 'dir1/dir2/store_sales (1).csv', 'dir1/dir2/example_datasync_1.py', 'dir1/dir2/spark-svc.yaml'}
I want to convert it to set how can i do that?
using set(a.split(",")) converting it into
{" 'dir1/dir2/spark-svc.yaml'}", " 'dir1/dir2/store_sales (1).csv'", " 'dir1/dir2/[updated]airflow-issue.txt'", " 'dir1/dir2/dep-dremio0.sh'", " 'dir1/dir2/example_sftp_to_s3.py'", " 'dir1/dir2/catalog_sales.csv'", " 'dir1/dir2/cloud_formation.py'", " 'dir1/dir2/1.log'", "{'dir1/dir2/k8s-dag-model.py'", " 'dir1/dir2/example_datasync_1.py'"}
here I have to remove ' and {,} . Is there a standard way to do it.
You can convert a String to a set/list using literal_eval
import ast
x = "{'dir1/dir2/k8s-dag-model.py', 'dir1/dir2/[updated]airflow-issue.txt', 'dir1/dir2/1.log', 'dir1/dir2/cloud_formation.py', 'dir1/dir2/example_sftp_to_s3.py', 'dir1/dir2/catalog_sales.csv', 'dir1/dir2/dep-dremio0.sh', 'dir1/dir2/store_sales (1).csv', 'dir1/dir2/example_datasync_1.py', 'dir1/dir2/spark-svc.yaml'}"
y = ast.literal_eval(x)
print(y)
Since the same works for a list too a similar answer for a list: https://stackoverflow.com/a/1894296/5236575
Docs: https://docs.python.org/3/library/ast.html#ast.literal_eval
I started learning python two days ago. Today I built a web scraping script which pulls data from yahoo finance and puts it in a csv file. The problem I have is that some values are string because yahoo finance displays them as such.
For example: Revenue: 806.43M
When I copy them into the csv I cant use them for calculation so I was wondering if it is possible to separate the "806.43" and "M" while still keeping both to see the unit of the number and put them in two different columns.
for the excel writing I use this command:
f.write(revenue + "," + revenue_value + "\n")
where:
print(revenue)
Revenue (ttm)
print(revenue_value)
806.43M
so in the end I should be able to use a command which looks something like this
f.write(revenue + "," + revenue_value + "," + revenue_unit + "\n")
where revenue_value is 806.43 and revenue_unit is M
Hope someone could help with the problem.
I believe the easiest way is to parse the number as string and convert it to a float based on the unit in the end of the string.
The following should do the trick:
def parse_number(number_str) -> float:
mapping = {
"K": 1000,
"M": 1000000,
"B": 1000000000
}
unit = number_str[-1]
number_float = float(number_str[:-1])
return number_float * mapping[unit]
And here's an example:
my_number = "806.43M"
print(parse_number(my_number))
>>> 806430000.0
You can always try regular expressions.
Here's a pretty good online tool to let you practice using Python-specific standards.
import re
sample = "Revenue (ttm): 806.43M"
# Note: the `(?P<name here>)` section is a named group. That way we can identify what we want to capture.
financials_pattern = r'''
(?P<category>.+?):?\s+? # Capture everything up until the colon
(?P<value>[\d\.]+) # Capture only numeric values and decimal points
(?P<unit>[\w]*)? # Capture a trailing unit type (M, MM, etc.)
'''
# Flags:
# re.I -> Ignore character case (upper vs lower)
# re.X -> Allows for 'verbose' pattern construction, as seen above
res = re.search(financials_pattern, sample, flags = re.I | re.X)
Print our dictionary of values:
res.groupdict()
Output:
{'category': 'Revenue (ttm)',
'value': '806.43',
'unit': 'M'}
We can also use .groups() to list results in a tuple.
res.groups()
Output:
('Revenue (ttm)', '806.43', 'M')
In this case, we'll immediately unpack those results into your variable names.
revenue = None # If this is None after trying to set it, don't print anything.
revenue, revenue_value, revenue_unit = res.groups()
We'll use fancy f-strings to print out both your f.write() call along with the results we've captured.
if revenue:
print(f'f.write(revenue + "," + revenue_value + "," + revenue_unit + "\\n")\n')
print(f'f.write("{revenue}" + "," + "{revenue_value}" + "," + "{revenue_unit}" + "\\n")')
Output:
f.write(revenue + "," + revenue_value + "," + revenue_unit + "\n")
f.write("Revenue (ttm)" + "," + "806.43" + "," + "M" + "\n")
I'm trying to get make a comments section for a website with the backend written in python. As of now, everything works fine except I cannot figure out how to format the date and time the way I want.
What I am trying to have at the time of posting is either of these:
Posted on Tue, 06/12/18 at - 11:20
or
Posted on 06/12/18 at - 11:21
Currently, what I have when the method is called is this:
import time
from datetime import *
time = ("Posted on " + str(datetime.now().day) + "/"
+ str(datetime.now().month) + " at: " + str(datetime.now().hour)
+ ":" + str(datetime.now().minute))
You can use datetime.datetime.strftime() to build any format you want:
import datetime
time = datetime.datetime.now().strftime("Posted on %d/%m/%y at - %H:%M")
print(time) # Posted on 07/12/17 at - 00:29
Read up more on special classes you can use when building your date: strftime() and strptime() Behavior
datetime.datetime.now().strftime('%d/%m/%y at - %H:%M')
'06/12/17 at - 16:29'
I have a very simple problem that I have been unable to find the solution to, so I thought I'd try my "luck" here.
I have a string that is created using variables and static text altogether. It is as follows:
filename_gps = 'id' + str(trip_id) + '_gps_did' + did + '_start' + str(trip_start) + '_end' + str(trip_end) + '.json'
However my problem is that pylint is complaining about this string reprensentation as it is too long. And here is the problem. How would I format this string representation over multiple lines without it looking weird and still stay within the "rules" of pylint?
At one point I ended up having it looking like this, however that is incredible "ugly" to look at:
filename_gps = 'id' + str(
trip_id) + '_gps_did' + did + '_start' + str(
trip_start) + '_end' + str(
trip_end) + '.json'
I found that it would follow the "rules" of pylint if I formatted it like this:
filename_gps = 'id' + str(
trip_id) + '_gps_did' + did + '_start' + str(
trip_start) + '_end' + str(
trip_end) + '.json'
Which is much "prettier" to look at, but in case I didn't have the "str()" casts, how would I go about creating such a string?
I doubt that there is a difference between pylint for Python 2.x and 3.x, but if there is I am using Python 3.x.
Don't use so many str() calls. Use string formatting:
filename_gps = 'id{}_gps_did{}_start{}_end{}.json'.format(
trip_id, did, trip_start, trip_end)
If you do have a long expression with a lot of parts, you can create a longer logical line by using (...) parentheses:
filename_gps = (
'id' + str(trip_id) + '_gps_did' + did + '_start' +
str(trip_start) + '_end' + str(trip_end) + '.json')
This would work for breaking up a string you are using as a template in a formatting operation, too:
foo_bar = (
'This is a very long string with some {} formatting placeholders '
'that is broken across multiple logical lines. Note that there are '
'no "+" operators used, because Python auto-joins consecutive string '
'literals.'.format(spam))
I don't know if this question is too specific but I'll give it a shot anyway:
I found a very nice bootstrap calendar that is very useful. The component is found here: http://bootstrap-calendar.azurewebsites.net/
I am trying to populate that calendar with events. I am using django. I made a function to return the elements on JSON format. I am making up the dates for now, just so its easier to view on the calendar for now:
def view_list_json(request):
i = 0
json_string = '{"sucess": 1, result: ['
for run in Model.objects.all():
start = datetime.datetime.now() - datetime.timedelta(days = i)
end = datetime.datetime.now() - datetime.timedelta(days = i) + datetime.timedelta(minutes = 40)
start_str = str(int(time.mktime(start.timetuple())))
end_str = str(int(time.mktime(end.timetuple())))
json_string += '{ "id": "' + str(run.id) + '"'
json_string += ', "title": "Foo"'
json_string += ', "url":"#"'
json_string += ', "class": "event-success"'
json_string += ', "start":"' + start_str + '"'
json_string += ', "end":"' + end_str + '"},'
i += 1
json_string = json_string[:-1]
json_string += ']}'
return HttpResponse(json_string, content_type="application/json")
JSON format I am trying to create:
https://github.com/Serhioromano/bootstrap-calendar/blob/master/events.json.php
The component seems to be reading this function correctly. I don't get any errors in this sense on Firebug console...
... Yet the data doesn't seem to load.
Any ideas on how I can approach the problem?
This JSON is not valid at all. There are quite a few problems, like extra close braces after each element, missing commas between elements, and missing open square brackets at the beginning of the next element.
But really you shouldn't be trying to build up a JSON string like that - it's too prone to errors. Even if you're hard-coding them, you should still build it up using standard Python lists and dicts, then serialize using the built-in json library.