I have a simple Firebase that I mostly interact with via Javascript, which works really well. However, I also have a Python program that needs to get data from existing children and put/update data on existing children. I tried python-firebasin, which would do what I want, but it is unreliable (hangs, fails, etc.).
So I'm looking at the python-firebase REST wrapper. This seems efficient, and works well. However, every time I try to post() data, I get not just the data I'm posting, but some kind of unique string paired with it, all inserted as a child.
For example, via Javascript, I might say:
db = new Firebase('https://myfirebase.firebaseio.com/testval/');
db.transaction(function(current) { return 1; });
This would then give me a Firebase that looked like:
|---testval: 1
But when I try to do something similar with the Python Firebase REST wrapper, such as:
db = firebase.FirebaseApplication('https://myfirebase.firebaseio.com/')
db.post('/testval/',1)
My Firebase looks something like this:
|---testval:
|---JI4BiBbICSEAnM9mDXf: 1
In other words, it inserts a new child, gives it a new string, and then appends the data. Is there any way to insert/modify data on my Firebase using the REST wrapper that would do it cleanly like I'm doing with Javascript? Without adding children, without adding these unique strings?
Try this instead:
db.put(1)
db.post() is the equivalent of .push() in the JavaScript API, so it creates a unique ID for you. db.put() is equivalent to .set() and will just set the data, which appears to be what you want.
Note that there is no equivalent for transactions in the REST API, but your example was just using a transaction to do a .set() so hopefully you don't actually need them.
Try this:
db = firebase.FirebaseApplication('https://myfirebase.firebaseio.com/')
db.put('', 'testval', 1)
put takes three arguments : first is url or path, second is the key name or the snapshot name and third is the data(json)
Related
I am trying to execute this following code to push data to Salesforce using the simple_salesforce python library :
from simple_salesforce import Salesforce
staging_df = hive.execute("select * from hdmni")
staging_df = staging_df.toPandas()
# # staging_df['birth_date']= staging_df['birth_date'].dt.date
staging_df['birth_date'] = staging_df['birth_date'].astype(str)
staging_df['encounter_start_date'] = staging_df['encounter_start_date'].astype(str)
staging_df['encounter_end_date'] = staging_df['encounter_end_date'].astype(str)
bulk_data = []
for row in staging_df.itertuples():
d= row._asdict()
del d['Index']
bulk_data.append(d)
sf = Salesforce(password='', username='', security_token='')
sf.bulk.Delivery_Detail__c.insert(bulk_data)
I am getting this error while trying to send dictionary to salesforce :
SalesforceMalformedRequest: Malformed request
https://subhotutorial-dev-ed.my.salesforce.com/services/async/38.0/job/7500o00000HtWP6AAN/batch/7510o00000Q15TnAAJ/result.
Response content: {'exceptionCode': 'InvalidBatch',
'exceptionMessage': 'Records not processed'}
There's something about your query that is not correct. While I don't know your use case, by reading this line, you can tell that you are attempting to insert into a custom object/entity in Salesforce:
sf.bulk.Delivery_Detail__c.insert(bulk_data)
The reason you can tell is because of the __c suffix, which gets appended onto custom objects and fields (that's two underscores, by the way).
Since you're inserting into a custom object, your fields would have to be custom, too. And note, you've not appended that suffix onto them.
Note: Every custom object/entity in Salesforce does come with a few standard fields to support system features like record key (Id), record name (Name), audit fields (CreatedById, CreatedDate, etc.). These wouldn't have a suffix. But none of the fields you reference are any of these standard system fields...so the __c suffix would be expected.
I suspect that what Salesforce is expecting in your insert operation are field names like this:
Birth_Date__c
Encounter_Start_Date__c
Encounter_End_Date__c
These are referred to as the API name for both objects and fields, and anytime code interacts with them (whether via integration, or on code that executes directly on the Salesforce platform) you need to make certain you're using this API name.
Incidentally, you can retrieve this API name through a number of ways. Probably easiest is to log into your Salesforce org, and in Setup > Object Manager > [some object] > Fields and Relationships you can view details of each field, including the API name. Here's a screen shot.
You can also use SObject describe APIs, either in native Apex code, or via integration and either the REST or SOAP APIs. Here's part of the response from the describe API request to the describe REST endpoint for the same object as my UI example above, found here at https://[domain]/services/data/v47.0/sobjects/Expense__c/describe:
Looking at the docs for the simple-salesforce python library you're using, they've surfaced the describe API. You can find some info under Other Options. You would invoke it as sf.SObject.describe where "SObject" is the actual object you want to find the information about. For instance, in your case you would use:
sf.Delivery_Detail__c.describe()
As a good first troubleshooting step when interacting with a Salesforce object, I'd always recommend double-checking correctly referencing an API name. I can't tell you how many times I've bumped into little things like adding or missing an underscore. Especially with the __c suffix.
I am new to python and Django. I haven't found anything in documentations so I have to write here.
I have such problem.
I have cars table where you can find it's make, model. year and so on.
Usually i make request by just Cars.objects.filter(make=x, model=y, year=z)
I want to make search and all params are in array. There are many params and is it possible to make something like
Cars.objects.filter(array)
Ok. How i am getting my data. It is regular post form which I send by ajax to my view class. I have list of allowed params which can be found in users request. I have different names for tableview name and input name, but i have everything i need in prepared array. It can take user's sent car-name Acura and make something like {'make':123}. At the end i have an array which is fully compatible with my db {'make': 123, 'model':321}. Can i make a request using this array and not write by hands every param?
if arguments are stored in this format:
args = [('make', 'x'),('model', 'y'),('year', 'z')]
You can try:
arg_dict = dict(args)
Cars.objects.filter(**arg_dict)
I have a database with a bunch of regular documents that look something like this (example from wiki):
{
"_id":"some_doc_id",
"_rev":"D1C946B7",
"Subject":"I like Plankton",
"Author":"Rusty",
"PostedDate":"2006-08-15T17:30:12-04:00",
"Tags":["plankton", "baseball", "decisions"],
"Body":"I decided today that I don't like baseball. I like plankton."
}
I'm working in Python with couchdb-python and I want to know if it's possible to add a field to each document. For example, if I wanted to have a "Location" field or something like that.
Thanks!
Regarding IDs
Every document in couchdb has an id, whether you set it or not. Once the document is stored you can access it through the doc._id field.
If you want to set your own ids you'll have to assign the id value to doc._id. If you don't set it, then couchdb will assign a uuid.
If you want to update a document, then you need to make sure you have the same id and a valid revision. If say you are working from a blog post and the user adds the Location, then the url of the post may be a good id to use. You'd be able to instantly access the document in this case.
So what's a revision
In your code snippet above you have the doc._rev element. This is the identifier of the revision. If you save a document with an id that already exists, couchdb requires you to prove that the document is still the valid doc and that you are not trying to overwrite someone else's document.
So how do I update a document
If you have the id of your document, you can just access each document by using the db.get(id) function. You can then update the document like this:
doc = db.get(id)
doc['Location'] = "On a couch"
db.save(doc)
I have an example where I store weather forecast data. I update the forecasts approximately every 2 hours. A separate process is looking for data that I get from a different provider looking at characteristics of tweets on the day.
This looks something like this.
doc = db.get(id)
doc_with_loc = GetLocationInformationFromOtherProvider(doc) # takes about 40 seconds.
doc_with_loc["_rev"] = doc["_rev"]
db.save(doc_with_loc) # This will fail if weather update has also updated the file.
If you have concurring processes, then the _rev will become invalid, so you have to have a failsave, eg. this could do:
doc = db.get(id)
doc_with_loc = GetLocationInformationFromAltProvider(doc)
update_outstanding = true
while update_outstanding:
doc = db.get(id) //reretrieve this to get
doc_with_loc["_rev"] = doc["_rev"]
update_outstanding = !db.save(doc_with_loc)
So how do I get the Ids?
One option suggested above is that you actively set the id, so you can retrieve it. Ie. if a user sets a given location that is attached to a URL, use the URL. But you may not know which document you want to update - or even have a process that finds all the document that don't have a location and assign one.
You'll most likely be using a view for this. Views have a mapper and a reducer. You'll use the first one, forget about the last one. A view with a mapper does the following:
It returns a simplyfied/transformed way of looking at your data. You can return multiple values per data or skip some. It gives the data you emit a key, and if you use the _include_docs function it will give you the document (with _id and rev alongside).
The simplest view is the default view db.view('_all_docs') this will return all documents and you may not want to update all of them. Views for example will be stored as a document as well when you define these.
The next simple way is to have view that only returns items that are of the type of the document. I tend to have a _type="article in my database. Think of this as marking that a document belongs to a certain table if you had stored them in a relational database.
Finally you can filter elements that have a location so you'd have a view where you can iterate over all those docs that still need a location and identify this in a separate process. The best documentation on writing view can be found here.
I'm trying to load JSON back into an object. The "loads" method seems to work without error, but the object doesn't seem to have the properties I expect.
How can I go about examining/inspecting the object that I have (this is web-based code).
results = {"Subscriber": {"firstname": "Neal", "lastname": "Walters"}}
subscriber = json.loads(results)
for item in inspect.getmembers(subscriber):
self.response.out.write("<BR>Item")
for subitem in item:
self.response.out.write("<BR> SubItem=" + subitem)
The attempt above returned this:
Item
SubItem=__class__
I don't think it matters, but for context:
The JSON is actually coming from a urlfetch in Google App Engine to
a rest web service created using this utility:
http://code.google.com/p/appengine-rest-server.
The data is being retrieved from a datastore with this definition:
class Subscriber(db.Model):
firstname = db.StringProperty()
lastname = db.StringProperty()
Thanks,
Neal
Update #1: Basically I'm trying to deserialize JSON back into an object.
In theory it was serialized from an object, and I want to now get it back into an object.
Maybe the better question is how to do that?
Update #2: I was trying to abstract a complex program down to a few lines of code, so I made a few mistakes in "pseudo-coding" it for purposes of posting here.
Here's a better code sample, now take out of website where I can run on PC.
results = '{"Subscriber": {"firstname": "Neal", "lastname": "Walters"}}'
subscriber = json.loads(results)
for key, value in subscriber.items():
print " %s: %s" %(key, value)
The above runs, what it displays doesn't look any more structured than the JSON string itself. It displays this:
Subscriber: {u'lastname': u'Walters', u'firstname': u'Neal'}
I have more of a Microsoft background, so when I hear serialize/deserialize, I think going from an object to a string, and from a string back to an object. So if I serialize to JSON, and then deserialize, what do I get, a dictionary, a list, or an object? Actually, I'm getting the JSON from a REST webmethod, that is on my behalf serializing my object for me.
Ideally I want a subscriber object that matches my Subscriber class above, and ideally, I don't want to write one-off custom code (i.e. code that would be specific to "Subscriber"), because I would like to do the same thing with dozens of other classes. If I have to write some custom code, I will need to do it generically so it will work with any class.
Update #3: This is to explain more of why I think this is a needed tool. I'm writing a huge app, probably on Google App Engine (GAE). We are leaning toward a REST architecture for several reasons, but one is that our web GUI should access the data store via a REST web layer. (I'm a lot more used to SOAP, so switching to REST is a small challenge in itself). So one of the classic ways of getting and update data is through a business or data tier. By using the REST utility mention above, I have the choice of XML or JSON. I'm hoping to do a small working prototype of both before we develop the huge app). Then, suppose we have a successful app, and GAE doubles it prices. Then we can rewrite just the data tier, and take our Python/Django user tier (web code), and run it on Amazon or somewhere else.
If I'm going to do all that, why would I want everything to be dictionary objects. Wouldn't I want the power of full-blown class structure? One of the next tricks is sort of an object relational mapping (ORM) so that we don't necessarily expose our exact data tables, but more of a logical layer.
We also want to expose a RESTful API to paying users, who might be using any language. For them, they can use XML or JSON, and they wouldn't use the serialize routine discussed here.
json only encodes strings, floats, integers, javascript objects (python dicts) and lists.
You have to create a function to turn the returned dictionary into a class and then pass it to a json.loads using the object_hook keyword argument along with the json string. Heres some code that fleshes it out:
import json
class Subscriber(object):
firstname = None
lastname = None
class Post(object):
author = None
title = None
def decode_from_dict(cls,vals):
obj = cls()
for key, val in vals.items():
setattr(obj, key, val)
return obj
SERIALIZABLE_CLASSES = {'Subscriber': Subscriber,
'Post': Post}
def decode_object(d):
for field in d:
if field in SERIALIZABLE_CLASSES:
cls = SERIALIZABLE_CLASSES[field]
return decode_from_dict(cls, d[field])
return d
results = '''[{"Subscriber": {"firstname": "Neal", "lastname": "Walters"}},
{"Post": {"author": {"Subscriber": {"firstname": "Neal",
"lastname": "Walters"}}},
"title": "Decoding JSON Objects"}]'''
result = json.loads(results, object_hook=decode_object)
print result
print result[1].author
This will handle any class that can be instantiated without arguments to the constructor and for which setattr will work.
Also, this uses json. I have no experience with simplejson so YMMV but I hear that they are identical.
Note that although the values for the two subscriber objects are identical, the resulting objects are not. This could be fixed by memoizing the decode_from_dict class.
results in your snippet is a dict, not a string, so the json.loads would raise an exception. If that is fixed, each subitem in the inner loop is then a tuple, so trying to add it to a string as you are doing would raise another exception. I guess you've simplified your code, but the two type errors should already show that you simplified it too much (and incorrectly). Why not use an (equally simplified) working snippet, and the actual string you want to json.loads instead of one that can't possibly reproduce your problem? That course of action would make it much easier to help you.
Beyyond peering at the actual string, and showing some obvious information such as type(subscriber), it's hard to offer much more help based on that clearly-broken code and such insufficient information:-(.
Edit: in "update2", the OP says
It displays this: Subscriber: {u'lastname': u'Walters', u'firstname': u'Neal'}
...and what else could it possibly display, pray?! You're printing the key as string, then the value as string -- the key is a string, and the value is another dict, so of course it's "stringified" (and all strings in JSON are Unicode -- just like in C# or Java, and you say you come from a MSFT background, so why does this surprise you at all?!). str(somedict), identically to repr(somedict), shows the repr of keys and values (with braces around it all and colons and commas as appropriate separators).
JSON, a completely language-independent serialization format though originally centered on Javascript, has absolutely no idea of what classes (if any) you expect to see instances of (of course it doesn't, and it's just absurd to think it possibly could: how could it possibly be language-independent if it hard-coded the very concept of "class", a concept which so many languages, including Javascript, don't even have?!) -- so it uses (in Python terms) strings, numbers, lists, and dicts (four very basic data types that any semi-decent modern language can be expected to have, at least in some library if not embedded in the language proper!). When you json.loads a string, you'll always get some nested combination of the four datatypes above (all strings will be unicode and all numbers will be floats, BTW;-).
If you have no idea (and don't want to encode by some arbitrary convention or other) what class's instances are being serialized, but absolutely must have class instances back (not just dicts etc) when you deserialize, JSON per se can't help you -- that metainformation cannot possibly be present in the JSON-serialized string itself.
If you're OK with the four fundamental types, and just want to see some printed results that you consider "prettier" than the default Python string printing of the fundamental types in question, you'll have to code your own recursive pretty-printing function depending on your subjective definition of "pretty" (I doubt you'd like Python's own pprint standard library module any more than you like your current results;-).
My guess is that loads is returning a dictionary. To iterate over its content, use something like:
for key, value in subscriber.items():
self.response.out.write("%s: %s" %(key, value))
I'm trying to send a Python list in to client side (encoded as JSON). This is the code snippet which I have written:
array_to_js = [vld_id, vld_error, False]
array_to_js[2] = True
jsonValidateReturn = simplejson.dumps(array_to_js)
return HttpResponse(jsonValidateReturn, mimetype='application/json')
How do I access it form client side? Can I access it like the following?
jsonValidateReturn[0]
Or how do I assign a name to the returned JSON array in order to access it?
Actually I'm trying to convert a server side Ajax script that returns an array (see Stack Overflow question Creating a JSON response using Django and Python that handles client side POST requests, so I wanted the same thing in return with Python, but it didn't go well.
The JSON array will be dumped without a name / assignment.
That is, in order to give it a name, in your JavaScript code you would do something like this:
var my_json_data_dump = function_that_gets_json_data();
If you want to visualize it, for example, substitute:
var my_json_data_dump = { 'first_name' : Bob, 'last_name': smith };
Also, like Iganacio said, you're going to need something like json2.js to parse the string into the object in the last example. You could wrap that parsing step inside of function_that_gets_json_data, or if you're using jQuery you can do it with a function like jQuery.getJSON().
json2.js is still nice to have, though.
In response to the comment (I need space and markup):
Yes, of course. All the Python side is doing is encoding a string representation (JSON) for you. You could do something like 'var blah = %s' % json.dumps(obj_to_encode) and then on the client side, instead of simply parsing the response as JSON, you parse it as JavaScript.
I wouldn't recommend this for a few reasons:
You're no longer outputting JSON. What if you want to use it in a context where you don't want the variable name, or can't parse JavaScript?
You're evaluating JavaScript instead of simply parsing JSON. It's an operation that's open to security holes (if someone can seed the data, they might be able to execute a XSS attack).
I guess you're facing something I think every Ajax developer runs in to. You want one place of truth in your application, but now you're being encouraged to define variables and whatnot in JavaScript. So you have to cross reference your Python code with the JavaScript code that uses it.
I wouldn't get too hung up on it. I can't see why you would absolutely need to control the name of the variable from Python in this manner. If you're counting on the variable name being the same so that you can reference it in subsequent JavaScript or Python code, it's something you might obviate by simply restructuring your code. I don't mean that as a criticism, just a really helpful (in general) suggestion!
If both client and server are in Python, here's what you need to know.
Server. Use a dictionary to get labels on the fields. Write this as the response.
>>> import json
>>> json.dumps( {'vld_id':1,'vls_error':2,'something_else':True} )
'{"vld_id": 1, "something_else": true, "vls_error": 2}'
Client. After reading the response string, create a Python dictionary this way.
>>> json.loads( '{"vld_id": 1, "something_else": true, "vls_error": 2}' )
{u'vld_id': 1, u'something_else': True, u'vls_error': 2}