I don't know if this question is too specific but I'll give it a shot anyway:
I found a very nice bootstrap calendar that is very useful. The component is found here: http://bootstrap-calendar.azurewebsites.net/
I am trying to populate that calendar with events. I am using django. I made a function to return the elements on JSON format. I am making up the dates for now, just so its easier to view on the calendar for now:
def view_list_json(request):
i = 0
json_string = '{"sucess": 1, result: ['
for run in Model.objects.all():
start = datetime.datetime.now() - datetime.timedelta(days = i)
end = datetime.datetime.now() - datetime.timedelta(days = i) + datetime.timedelta(minutes = 40)
start_str = str(int(time.mktime(start.timetuple())))
end_str = str(int(time.mktime(end.timetuple())))
json_string += '{ "id": "' + str(run.id) + '"'
json_string += ', "title": "Foo"'
json_string += ', "url":"#"'
json_string += ', "class": "event-success"'
json_string += ', "start":"' + start_str + '"'
json_string += ', "end":"' + end_str + '"},'
i += 1
json_string = json_string[:-1]
json_string += ']}'
return HttpResponse(json_string, content_type="application/json")
JSON format I am trying to create:
https://github.com/Serhioromano/bootstrap-calendar/blob/master/events.json.php
The component seems to be reading this function correctly. I don't get any errors in this sense on Firebug console...
... Yet the data doesn't seem to load.
Any ideas on how I can approach the problem?
This JSON is not valid at all. There are quite a few problems, like extra close braces after each element, missing commas between elements, and missing open square brackets at the beginning of the next element.
But really you shouldn't be trying to build up a JSON string like that - it's too prone to errors. Even if you're hard-coding them, you should still build it up using standard Python lists and dicts, then serialize using the built-in json library.
Related
I have a python script that scrapes a webpage and downloads the mp3s found on it.
I am trying to name the files using elements that I have successfully captured in a separate function.
I am having trouble naming the downloaded files, this is what I have so far:
def make_safe_filename(text: str) -> str:
"""convert forbidden chars to underscores"""
return ''.join(c if (c.isalnum() or c.isspace()) else '_' for c in text).strip('_ ')
filename = make_safe_filename(a['artist'] + a['title'] + a['date'] + a['article_url'])
I am trying to save the file name as "Artist - Title - Date - Article_url" however I am struggling to do this. At the moment the variables are all mashed together without spaces, eg. ArtistTitleDateArticle_url.mp3
I've tried
filename = make_safe_filename(a['artist'] + "-" + a['title'] + "-" + a['date'] + "-" +
a['article_url'])
but this throws up errors.
Can anyone shed some light on where I am going wrong? I know it's something to do with combining variables but I am stuck. Thanks in advance.
I am guessing your a is a dictionary? Maybe you could clarify this in your question? Also what do you typically have in a['article_url']? Could you also post the traceback?
This is my attempt (note: no changes to the function):
def make_safe_filename(text: str) -> str:
"""convert forbidden chars to underscores"""
return ''.join(c if (c.isalnum() or c.isspace()) else '_' for c in text).strip('_ ')
a = {
'artist': 'Metallica',
'title': 'Nothing Else Matters',
'date': '1991',
'article_url': 'unknown',
}
filename = make_safe_filename(a['artist'] + '-' + a['title'] + '-' + a['date'] + '-' + a['article_url'])
print(filename)
Which produced the following output:
Metallica_Nothing Else Matters_1991_unknown
You code should actually work, but if you add the - before passing the joined string to the function, it will just replace those with _ as well. Instead, you could pass the individual fields and then join those in the function, after replacing the "illegal" characters for each field individually. Also, you could regular expressions and re.sub for the actual replacing:
import re
def safe_filename(*fields):
return " - ".join(re.sub("[^\w\s]", "_", s) for s in fields)
>>> safe_filename("Art!st", "S()ng", "§$%")
'Art_st - S__ng - ___'
Of course, if your a is a dictionary and you always want the same fields from that dict (artist, title, etc.) you could also just pass the dict itself and extract the fields within the function.
I had a similar problem recently, the best solution is probably to use regex, but I'm too lazy to learn that, so I wrote a replaceAll function:
def replaceAll(string, characters, replacement):
s = string
for i in characters:
s = s.replace(i, replacement)
return s
and then I used it to make a usable filename:
fName = replaceAll(name, '*<>?|"/\\.,\':', "")
in your case it would be:
filename = replaceAll(a['artist'] + a['title'] + a['date'] + a['article_url'], '*<>?|"/\\.,\':', "-")
I wrote code to append a json response into a list for some API work I am doing, but it stores the single quotes around the alphanumerical value I desire. I would like to get rid of the single quotes. Here is what I have so far:
i = 0
deviceID = []
while i < deviceCount:
deviceID.append(devicesRanOn['resources'][i])
deviceID[i] = re.sub('[\W_]', '', deviceID[i])
i += 1
if i >= deviceCount:
break
if (deviceCount == 1):
print ('Device ID: ', deviceID)
elif (deviceCount > 1):
print ('Device IDs: ', deviceID)
the desired input should look like this:
input Device IDs:
['14*************************00b29', '58*************************c3df4']
Output:
['14*************************00b29', '58*************************c3df4']
Desired Output:
[14*************************00b29, 58*************************c3df4]
As you can see, I am trying to use RegEx to filter non Alphanumeric and replace those with nothing. It is not giving me an error nor is it preforming the actions I am looking for. Does anyone have a recommendation on how to fix this?
Thank you,
xOm3ga
You won't be able to use the default print. You'll need to use your own means of making a representation for the list. But this is easy with string formatting.
'[' + ', '.join(f'{id!s}' for id in ids) + ']'
The f'{id:!s} is an f-string which formats the variable id using it's __str__ method. If you're on a version pre-3.6 which doesn't use f-strings, you can also use
'%s' % id
'{!s}'.format(id)
PS:
You can simplify you're code significantly by using a list comprehension and custom formatting instead of regexes.
ids = [device for device in devicesRanOn['resources'][:deviceCount]]
if deviceCount == 1:
label = 'Device ID:'
elif deviceCount > 1:
label = 'Device IDs:'
print(label, '[' + ', '.join(f'{id!s}' for id in ids) + ']')
i'm a python beginner and having an issue. Trying to pull API info and convert the extracted JSON time object to a datetime object in Python so I can run the date.weekday() function on it eventually (the overall goal is to extract all the dates from the API and see which the output by days - I plan on populating a empty dictionary once I can extract all the dates).
For some reason, even with my conditional statements, I'm still printing (2015, 04, 06) with all the zeroes. That's my problem.
I have a feeling I'm getting something basic wrong, and that there's also a better way to go about this than doing all the ifs/elses with the 0-padding in the date object.
here's my code so far:
from datetime import date
import datetime
import json
import requests
r = requests.get('https://api.github.com/repos/mbostock/d3/commits?since=2015-04-12330:00:000')
jsonoutput = r.json()
modified = (jsonoutput[0]["commit"]["author"]["date"])
#at this point, modified gives something like: "2015-04-06T22:28:16Z"
if modified[5] == 0:
if modified[8] == 0:
new_format = ("(" + modified[0:4] + ", " + modified[6] + ", " + modified[9] + ")")
#with the particular jsonoutput[0] pulled here, this one should be triggered
else:
new_format = ("(" + modified[0:4] + ", " + modified[6] + ", " + modified[8:10] + ")")
else:
if modified[8] == 0:
new_format = ("(" + modified[0:4] + ", " + modified[5:7] + ", " + modified[9] + ")")
else:
new_format = ("(" + modified[0:4] + ", " + modified[5:7] + ", " + modified[8:10] + ")")
print(new_format)
print(date.weekday(datetime.date(new_format)))
The error happens in your current code because new_format is defined as a string, while the datetime.date takes the arguments as integers. Also, since you compare the string "0" with the numerical 0, the modified is simply not created.
Instead of this:
new_format = ("(" + modified[0:4] + ", " + modified[6] + ", " + modified[8:10]) + ")")
do this:
new_format = (int(modified[0:4]), int(modified[5:7]), int(modified[8:10]))
The above will work with all of your cases, so you can remove the convoluted if-else blocks as well.
Alternatively, you could split this modeified string by "T", and then use another split by "-" to get the integer values:
new_format = map(int, modified.split("T")[0].split("-"))
You'll also need to unpack the list when passing as the argument, so your complete code becomes:
import json
import requests
from datetime import date
r = requests.get('https://api.github.com/repos/mbostock/d3/commits?since=2015-04-12330:00:000')
jsonoutput = r.json()
modified = jsonoutput[0]["commit"]["author"]["date"]
new_format = (int(modified[0:4]), int(modified[5:7]), int(modified[8:10]))
print(date.weekday(date(*new_format)))
Also, as others have already pointed out in their answers, it might be a better idea to dateutil.parser.parse than to write your own parsing logic. (dateutil is not a builtin package, you'll have to install it) :)
What you get from the json is actually a datetime representation in an ISO format
You can refer to this SO answer https://stackoverflow.com/a/15228038/58129 to convert the string
You're trying to make your own parsing functions where Python has its own.
from dateutil import parser
from datetime import date
my_date = dateutil.parser.parse(modified)
is_week_day = date.weekday(my_date)
If dateutil is not installed on your machine, try pip install python-dateutil
However, if you want to go with Python's standard library:
from datetime import date, datetime
from time import strptime
mytime = strptime(modified, '%Y-%m-%dT%H:%M:%SZ')
my_date = datetime(*my_time[:6])
is_week_day = date.weekday(my_date)
I'm fairly new to Python and I've written a scraper that prints the data I scrap the exact way I need it, but I'm having trouble writing the data to a file. I need it to look the exact same way and be in the same order as it does when it prints in IDLE
import requests
import re
from bs4 import BeautifulSoup
year_entry = raw_input("Enter year: ")
week_entry = raw_input("Enter week number: ")
week_link = requests.get("http://sports.yahoo.com/nfl/scoreboard/?week=" + week_entry + "&phase=2&season=" + year_entry)
page_content = BeautifulSoup(week_link.content)
a_links = page_content.find_all('tr', {'class': 'game link'})
for link in a_links:
r = 'http://www.sports.yahoo.com' + str(link.attrs['data-url'])
r_get = requests.get(r)
soup = BeautifulSoup(r_get.content)
stats = soup.find_all("td", {'class':'stat-value'})
teams = soup.find_all("th", {'class':'stat-value'})
scores = soup.find_all('dd', {"class": 'score'})
try:
game_score = scores[-1]
game_score = game_score.text
x = game_score.split(" ")
away_score = x[1]
home_score = x[4]
home_team = teams[1]
away_team = teams[0]
away_team_stats = stats[0::2]
home_team_stats = stats[1::2]
print away_team.text + ',' + away_score + ',',
for stats in away_team_stats:
print stats.text + ',',
print '\n'
print home_team.text + ',' + home_score +',',
for stats in home_team_stats:
print stats.text + ',',
print '\n'
except:
pass
I am totally confused on how to get this to print to a txt file the same way it prints in IDLE. The code is built to only run on completed weeks of the NFL season. So if you test the code, I recommend year = 2014 and week = 12 (or before)
Thanks,
JT
To write to a file you need to build up the line as a string, then write that line to a file.
You'd use something like:
# Open/create a file for your output
with open('my_output_file.csv', 'wb') as csv_out:
...
# Your BeautifulSoup code and parsing goes here
...
# Then build up your output strings
for link in a_links:
away_line = ",".join([away_team.text, away_score])
for stats in away_team_stats:
away_line += [stats.text]
home_line = ",".join(home_team.text, home_score])
for stats in home_team_stats:
home_line += [stats.text]
# Write your output strings to the file
csv_out.write(away_line + '\n')
csv_out.write(home_line + '\n')
This is a quick and dirty fix. To do it properly you probably want to look into the csv module (docs)
From the structure of your output I agree with Jamie that using CSV is a logical choice.
But since you're using Python 2, it's possible to use an alternate form of the print statement to print to a file.
From https://docs.python.org/2/reference/simple_stmts.html#the-print-statement
print also has an extended form, defined by the second portion of the
syntax described above. This form is sometimes referred to as “print
chevron.” In this form, the first expression after the >> must
evaluate to a “file-like” object, specifically an object that has a
write() method as described above. With this extended form, the
subsequent expressions are printed to this file object. If the first
expression evaluates to None, then sys.stdout is used as the file for
output.
Eg,
outfile = open("myfile.txt", "w")
print >>outfile, "Hello, world"
outfile.close()
However, this syntax is not supported in Python 3, so I guess it's probably not a good idea to use it. :) FWIW, I generally use the file write() method in my code when writing to files, except that I tend to use print >>sys.stderr for error messages.
I have an excel book with a couple of sheets. Each sheet has two columns with PersonID and LegacyID. We are basically trying to update some records in the database based on personid. This is relatively easy to do TSQL and I might even be able to get it done pretty quick in powershell but since I have been trying to learn Python, I thought I would try this in Python. I used xlrd module and I was able to print update statements. below is my code:
import xlrd
book = xlrd.open_workbook('D:\Scripts\UpdateID01.xls')
sheet = book.sheet_by_index(0)
myList = []
for i in range(sheet.nrows):
myList.append(sheet.row_values(i))
outFile = open('D:\Scripts\update.txt', 'wb')
for i in myList:
outFile.write("\nUPDATE PERSON SET LegacyID = " + "'" + str(i[1]) + "'" + " WHERE personid = " + "'" + str(i[0])
+ "'")
Two problems - when I read the output file, I see the LegacyID printed as float. How do I get rid of .0 at the end of each id? Second problem, python doesn't print each update statement in a new line in the output text file. How to I format it?
Edit: Please ignore the format issue. It did print in new lines when I opened the output file in Notepad++. The float issue still remains.
Can you turn the LegacyID into ints ?
i[1] = int(i[1])
outFile.write("\nUPDATE PERSON SET LegacyID = " + "'" + str(i[1]) + "'" + " WHERE personid = " + "'" + str(i[0])
+ "'")
try this..
# use 'a' if you want to append in your text file
outFile = open(r'D:\Scripts\update.txt', 'a')
for i in myList:
outFile.write("\nUPDATE PERSON SET LegacyID = '%s' WHERE personid = '%s'" %( int(i[1]), str(i[0])))
Since you are learning Python (which is very laudable!) you should start reading about string formatting in the Python docs. This is the best place to start whenever you have a question light this.
Hint: You may want to convert the float items to integers using int().