I'm trying to use memcache to cache data retrevied from the datastore. Storing stings works fine. But can't one store an object? I get an error "TypeError: 'str' object is not callable" when trying to store with this:
pageData = StandardPage(page)
memcache.add(memcacheid, pageData, 60)
I've read in the documentation that it requires "The value type can be any value supported by the Python pickle module for serializing values." But don't really understand what that is. Or how to convert pageData to it.
Any ideas?
..fredrik
EDIT:
I was a bit unclear. PageType is an class that amongst other thing get data from the datastore and manipulate it. The class looks like this:
class PageType():
def __init__(self, page):
self.pageData = PageData(data.Modules.gql('WHERE page = :page', page = page.key()).fetch(100))
self.modules = []
def renderEditPage(self):
self.addModules()
return self.modules
class StandardPage(PageTypes.PageType):
templateName = 'Altan StandardPage'
templateFile = 'templates/page.html'
def __init__(self, page):
self.pageData = PageTypes.PageData(data.Modules.gql('WHERE page = :page', page = page.key()).fetch(100))
self.modules = []
self.childModules = []
for child in page.childPages:
self.childModules.append(PageTypes.PageData(data.Modules.gql('WHERE page = :page', page = child.key()).fetch(100)))
def addModules(self):
self.modules.append(PageTypes.getStandardHeading(self, 'MainHeading'))
self.modules.append(PageTypes.getStandardTextBox(self, 'FirstTextBox'))
self.modules.append(PageTypes.getStandardTextBox(self, 'SecondTextBox'))
self.modules.append(PageTypes.getStandardHeading(self, 'ListHeading'))
self.modules.append(PageTypes.getStandardTextBox(self, 'ListTextBox'))
self.modules.append(PageTypes.getDynamicModules(self))
You can use db.model_to_protobuf to turn your object into something that can be stored in memcache. Similarly, db.model_from_protobuf will get your object back.
Resource:
Datastore Functions
Related
This is the thing, I was trying to make a crud for a rest api for a movie app, I was able to successfully perform the functionalities with flask to create a movie within the json, to delete a movie, and to list them, but I have problems with being able to edit one of the movies inside the JSON since I don't know how to access a sector of it to be able to modify it, this is what I was trying:
def __init__(self, idpelicula, nombre, descripcion, elenco, restriccionedad, fecha, calificacion):
self.idpelicula = idpelicula
self.nombre = nombre
self.descripcion = descripcion
self.elenco = elenco
self.restriccionedad = restriccionedad
self.fecha = fecha
self.calificacion = calificacion
movies_list = [
Peliculas(1, "Shrek", "Pelicula de Ogro", "Shrek, Fiona", "+APTA", "2001", "10"),
Peliculas(2, "Shrek2", "Otra Pelicula de Ogro", "Shrek, Fiona", "+APTA", "2005", "10"),
]
#app.route("/movies")
def list_movies():
return jsonify([pelicula.__dict__ for pelicula in movies_list])
#app.route("/movies/<id_search_movie>")
def obtener_pelicula(id_search_movie):
for pelicula in movies_list:
if pelicula.idpelicula == int(id_search_movie):
return jsonify(pelicula.__dict__)
return "Pelicula no encontrada", 404
#app.route("/movies", methods=['POST'])
def create_movie():
movie_data = request.get_json()
new_movie = Peliculas(movie_data["idpelicula"], movie_data["nombre"], movie_data["descripcion"],
movie_data["elenco"], movie_data["restriccionedad"], movie_data["fecha"],
movie_data["calificacion"])
movies_list.append(new_movie)
return "OK", 200
#app.route("/movies/<id_delete_movie>", methods=['DELETE'])
def delete_movie(id_delete_movie):
for pelicula in movies_list:
if pelicula.idpelicula == int(id_delete_movie):
movies_list.remove(pelicula)
return "OK", 200
This is what I dont know how to do, (I know its wrong, and that this isn't even the logic to do this), but this is what I tried:
#app.route("/movies", methods=['PUT'])
def edit_movie():
movie_data = request.get_json()
new_movie = Peliculas(movie_data["idpelicula"], movie_data["nombre"], movie_data["descripcion"],
movie_data["elenco"], movie_data["restriccionedad"], movie_data["fecha"],
movie_data["calificacion"])
movies_list.append(new_movie)
return "OK", 200
Any help, even from the point of view of the logic behind doing this will come handy. Thanks to you all.
There is a ton of problems with the way you are coding this at the moment, but here i will show you how you can edit an object in the way you are currently doing it.
So with your approach the edit view should be like this:
#app.route("/movies", methods=['PUT'])
def edit_movie():
# Accessing data from request
movie_data = request.get_json()
# Accesing Object from the movie-list by ID
for movie in movies_list:
if movie.idpelicula == movie_data["idpelicula"]:
edited_movie = movie
# Modifying the object attributes
for name, value in movie_data.items():
if hasattr(edited_movie, name):
setattr(edited_movie, name, value)
return "OK", 200
The problem here is that you were creating a new instance instead of editing an existing one.
in the future you could explore the usage of SQLAlchemy models so you can persist the data in a database, and still be managing objects.
PD1: Finally i recommend you be coherent with the naming, by this i mean if you are calling your variables and classes in spanish Pelicula, peliculas, try to use the same naming throught all your code, same if you use english.
PD2: Study some basic python usage of classes and variable assignment, so you can improve your code.
Greetings from Chile.
I am relatively new to Django, but not to python, My model is trying to use a class (defined in a separate file) in which data is coming from a REST API, the retrieved data is in a nested dictionary. The code will run fine in python, but when I try it in Django (makemigrations), I get an error:
File "c:\blah-blah\Clone_PR.py", line 20, in GetFoundOnSelectItems
values = self._issueEdit["fields"]["customfield_13940"]["allowedValues"]
TypeError: 'NoneType' object is not subscriptable
I tried using type hints, but that does not work either.
models.py
from dal import autocomplete
from django.db import models
from django.contrib import messages
from .Login import jlogin
from .Jira_Constants import ProductionServer, TestServer, StageServer
from .Clone_PR import Issue
jira = None
issue = Issue()
class ClonePrLogin(models.Model):
username = models.CharField(max_length=30)
password = models.CharField(max_length=30)
#classmethod
def LoginToJira(cls):
global jira
jira = jlogin(ProductionServer, cls.username, cls.password)
class PrEntry(models.Model):
prToClone = models.CharField(max_length=20)
#classmethod
def GetIssueAndMeta(cls):
global issue
issue.initialize(jira, cls.prToClone)
class ClonePr(models.Model):
issueKey = issue.issueKey
issue.GetFoundOnSelectItems()
foundOnList = issue.foundOnSelectItems
foundOn = autocomplete.Select2ListChoiceField(choice_list=foundOnList)
Clone_PR.py
from typing import List, Dict
class Issue():
def __init__(self):
self.jiraInst = None
self.issueKey = ''
self._issue = None
self._issueEdit = None
# self._issueEdit = Dict[str, Dict[str, Dict[str, List[Dict[str, str]]]]]
self.foundOnSelectItems = []
def initialize(self, jira, prKey):
self.jiraInst = jira
self.issueKey = prKey
self._issue = jira.issue(prKey)
self._issueEdit = jira.editmeta(prKey)
def GetFoundOnSelectItems(self):
values = self._issueEdit["fields"]["customfield_13940"]["allowedValues"]
items = [x["value"] for x in values]
self.foundOnSelectItems = items
In Django, running makemigrations will load all the modules. You said you're familiar with Python so you should know that the declarations inside the class:
class ClonePr(models.Model):
issueKey = issue.issueKey
issue.GetFoundOnSelectItems()
foundOnList = issue.foundOnSelectItems
foundOn = autocomplete.Select2ListChoiceField(choice_list=foundOnList)
will run when the modules load. You're calling issue.GetFoundOnSelectItems() at that time, which in turn calls values = self._issueEdit["fields"]["customfield_13940"]["allowedValues"], except that self._issueEdit = None upon the initiation of instance Issue above with this line: issue = Issue().
I highly recommend you spend some time to become more familiar with how Django starts up an app. The module-level and nested model declarations here are both antipatterns and may cause data issues in the future.
Apologies if this isn't totally clear - I'm a Python copy-the-code-and-try-to-make-it-work developer.
I'm using the Google NLP API in Python 2.7.
When I use analyze_entities(), I can get and print the name, entity type and salience.
Mentions is supposed to contain the noun type: PROPER or COMMON, per this page:
https://cloud.google.com/natural-language/docs/reference/rest/v1beta1/Entity#EntityMention
I can't get mention type from the returned dictionary.
Here's my hideous code:
def entities_text(text, client):
"""Detects entities in the text."""
language_client = client
# Instantiates a plain text document.
document = language_client.document_from_text(text)
# Detects entities in the document. You can also analyze HTML with:
# document.doc_type == language.Document.HTML
entities = document.analyze_entities()
return entities
articles = os.listdir('articles')
for f in articles:
language_client = language.Client()
fname = "articles/" + f
thisfile = open(fname,'r')
content = thisfile.read()
entities = entities_text(content, language_client)
for e in entities:
name = e.name.strip()
type = e.entity_type.strip()
if e.name.strip()[0].isupper() and len(e.name.strip()) > 2:
print name, type, e.salience, e.mentions
That returns this:
RELATED OTHER 0.0019081507 [u'RELATED']
Zoe 3 PERSON 0.0016676666 [u'Zoe 3']
Where the value in [] is the mentions.
If I try to get mentions.type, I get an attribute not found error.
I'd appreciate any input.
1) Do not call the "AnalyzeEntities" function, but call the "AnnotateText" one instead.
2) Check for "Proper". Examine its value, it should be "PROPER" and not "PROPER_UNKNOWN" nor "NOT_PROPER".
I'm a bit new to Python dev -- I'm creating a larger project for some web scraping. I want to approach this as "Pythonically" as possible, and would appreciate some help with the project structure. Here's how I'm doing it now:
Basically, I have a base class for an object whose purpose is to go to a website and parse some specific data on it into its own array, jobs[]
minion.py
class minion:
# Empty getJobs() function to be defined by object pre-instantiation
def getJobs(self):
pass
# Constructor for a minion that requires site authorization
# Ex: minCity1 = minion('http://portal.com/somewhere', 'user', 'password')
# or minCity2 = minion('http://portal.com/somewhere')
def __init__(self, title, URL, user='', password=''):
self.title = title
self.URL = URL
self.user = user
self.password = password
self.jobs = []
if (user == '' and password == ''):
self.reqAuth = 0
else:
self.reqAuth = 1
def displayjobs(self):
for j in self.jobs:
j.display()
I'm going to have about 100 different data sources. The way I'm doing it now is to just create a separate module for each "Minion", which defines (and binds) a more tailored getJobs() function for that object
Example: minCity1.py
from minion import minion
from BeautifulSoup import BeautifulSoup
import urllib2
from job import job
# MINION CONFIG
minTitle = 'Some city'
minURL = 'http://www.somewebpage.gov/'
# Here we define a function that will be bound to this object's getJobs function
def getJobs(self):
page = urllib2.urlopen(self.URL)
soup = BeautifulSoup(page)
# For each row
for tr in soup.findAll('tr'):
tJob = job()
span = tr.findAll(['span', 'class="content"'])
# If row has 5 spans, pull data from span 2 and 3 ( [1] and [2] )
if len(span) == 5:
tJob.title = span[1].a.renderContents()
tJob.client = 'Some City'
tJob.source = minURL
tJob.due = span[2].div.renderContents().replace('<br />', '')
self.jobs.append(tJob)
# Don't forget to bind the function to the object!
minion.getJobs = getJobs
# Instantiate the object
mCity1 = minion(minTitle, minURL)
I also have a separate module which simply contains a list of all the instantiated minion objects (which I have to update each time I add one):
minions.py
from minion_City1 import mCity1
from minion_City2 import mCity2
from minion_City3 import mCity3
from minion_City4 import mCity4
minionList = [mCity1,
mCity2,
mCity3,
mCity4]
main.py references minionList for all of its activities for manipulating the aggregated data.
This seems a bit chaotic to me, and was hoping someone might be able to outline a more Pythonic approach.
Thank you, and sorry for the long post!
Instead of creating functions and assigning them to objects (or whatever minion is, I'm not really sure), you should definitely use classes instead. Then you'll have one class for each of your data sources.
If you want, you can even have these classes inherit from a common base class, but that isn't absolutely necessary.
I am using the following code to serialize my appengine datastore to JSON
class DictModel(db.Model):
def to_dict(self):
return dict([(p, unicode(getattr(self, p))) for p in self.properties()])
class commonWordTweets(DictModel):
commonWords = db.StringListProperty(required=True)
venue = db.ReferenceProperty(Venue, required=True, collection_name='commonWords')
class Venue(db.Model):
id = db.StringProperty(required=True)
fourSqid = db.StringProperty(required=False)
name = db.StringProperty(required=True)
twitter_ID = db.StringProperty(required=True)
This returns the following JSON response
[
{
"commonWords": "[u'storehouse', u'guinness', u'badge', u'2011"', u'"new', u'mayor', u'dublin)']",
"venue": "<__main__.Venue object at 0x1028ad190>"
}
]
How can I return the actual venue name to appear?
Firstly, although it's not exactly your question, it's strongly recommended to use simplejson to produce json, rather than trying to turn structures into json strings yourself.
To answer your question, the ReferenceProperty just acts as a reference to your Venue object. So you just use its attributes as per normal.
Try something like:
cwt = commonWordTweets() # Replace with code to get the item from your datastore
d = {"commonWords":cwt.commonWords, "venue": cwt.venue.name}
jsonout = simplejson.dumps(d)