We Keep Coding

Don't understand this ConfigParser.InterpolationSyntaxError

So I have tried to write a small config file for my script, which should specify an IP address, a port and a URL which should be created via interpolation using the former two variables. My config.ini looks like this:
[Client]
recv_url : http://%(recv_host):%(recv_port)/rpm_list/api/
recv_host = 172.28.128.5
recv_port = 5000
column_list = Name,Version,Build_Date,Host,Release,Architecture,Install_Date,Group,Size,License,Signature,Source_RPM,Build_Host,Relocations,Packager,Vendor,URL,Summary
In my script I parse this config file as follows:
config = SafeConfigParser()
config.read('config.ini')
column_list = config.get('Client', 'column_list').split(',')
URL = config.get('Client', 'recv_url')
If I run my script, this results in:
Traceback (most recent call last):
File "server_side_agent.py", line 56, in <module>
URL = config.get('Client', 'recv_url')
File "/usr/lib64/python2.7/ConfigParser.py", line 623, in get
return self._interpolate(section, option, value, d)
File "/usr/lib64/python2.7/ConfigParser.py", line 691, in _interpolate
self._interpolate_some(option, L, rawval, section, vars, 1)
File "/usr/lib64/python2.7/ConfigParser.py", line 716, in _interpolate_some
"bad interpolation variable reference %r" % rest)
ConfigParser.InterpolationSyntaxError: bad interpolation variable reference '%(recv_host):%(recv_port)/rpm_list/api/'
I have tried debugging, which resulted in giving me one more line of error code:
...
ConfigParser.InterpolationSyntaxError: bad interpolation variable reference '%(recv_host):%(recv_port)/rpm_list/api/'
Exception AttributeError: "'NoneType' object has no attribute 'path'" in <function _remove at 0x7fc4d32c46e0> ignored
Here I am stuck. I don't know where this _remove function is supposed to be... I tried searching for what the message is supposed to tell me, but quite frankly I have no idea. So...
Is there something wrong with my code?
What does '< function _remove at ... >' mean?

There was indeed a mistake in my config.ini file. I did not regard the s at the end of %(...)s as a necessary syntax element. I suppose it refers to "string" but I couldn't really confirm this.

My .ini file for starting the Python Pyramid server had a similar problem.
And to use the variable from the .env file, I needed to add the following: %%(VARIEBLE_FOR_EXAMPLE)s
But I got other problems, and I solved them with this: How can I use a system environment variable inside a pyramid ini file?

Python jsonpickle error: 'OrderedDict' object has no attribute '_OrderedDict__root'

I'm hitting this exception with jsonpickle, when trying to pickle a rather complex object that unfortunately I'm not sure how to describe here. I know that makes it tough to say much, but for what it's worth:
>>> frozen = jsonpickle.encode(my_complex_object_instance)
>>> thawed = jsonpickle.decode(frozen)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/jsonpickle/__init__.py",
line 152, in decode
return unpickler.decode(string, backend=backend, keys=keys)
:
:
File "/Library/Python/2.7/site-packages/jsonpickle/unpickler.py",
line 336, in _restore_from_dict
instance[k] = value
File "/Library/Python/2.7/site-packages/botocore/vendored/requests/packages/urllib3/packages/ordered_dict.py",
line 49, in __setitem__
root = self.__root
AttributeError: 'OrderedDict' object has no attribute '_OrderedDict__root'
I don't find much of assistance when googling the error. I do see what looks like the same issue was resolved at some time past for simpler objects:
https://github.com/jsonpickle/jsonpickle/issues/33
The cited example in that report works for me:
>>> jsonpickle.decode(jsonpickle.encode(collections.OrderedDict()))
OrderedDict()
>>> jsonpickle.decode(jsonpickle.encode(collections.OrderedDict(a=1)))
OrderedDict([(u'a', 1)])
Has anyone ever run into this themselves and found a solution? I ask with the understanding that my case may be "differently idiosynchratic" than another known example.

The requests module for me seems to be running into problems when I .decode(). After looking at the jsonpickle code a bit, I decided to fork it and change the following lines to see what was going on (and I ended up keeping a private copy of jsonpickle with the changes so I can move forward).
In jsonpickle/unpickler.py (in my version it's line 368), search for the if statement section in the method _restore_from_dict():
if (util.is_noncomplex(instance) or
util.is_dictionary_subclass(instance)):
instance[k] = value
else:
setattr(instance, k, value)
and change it to this (it will logERROR the ones that are failing and then you can either keep the code in place or change your OrderedDict's version that have __root)
if (util.is_noncomplex(instance) or
util.is_dictionary_subclass(instance)):
# Currently requests.adapters.HTTPAdapter is using a non-standard
# version of OrderedDict which doesn't have a _OrderedDict__root
# attribute
try:
instance[k] = value
except AttributeError as e:
import logging
import pprint
warnmsg = 'Unable to unpickle {}[{}]={}'.format(pprint.pformat(instance), pprint.pformat(k), pprint.pformat(value))
logging.error(warnmsg)
else:
setattr(instance, k, value)

TypeError: cannot concatenate 'str' and 'SFType' objects (simple-salesforce)

my project is to extract the contents of all my salesforce tables, including the custom ones. To do this, I need to know the names of the columns (fields), since SOQL does not support "SELECT * from TABLENAME".
With simple-salesforce, I know that the following works:
sf = Salesforce(username='foo#bar.com', password='abcd', security_token='ZCdsdPdE4eI2DZMl5gwCFIGEFU')
field_data = sf.Contact.describe()["fields"]
But my problem is that I need to parameterize the "Contact" string in the actual method call above, so that I can call this method for objects that I do not know the names of (ie not defined in standard salesforce). For example I need to do:
field_data = sf.CustomTableName.describe()["fields"]
When I try and use the SFType class:
contact = SFType('Contact',sf.sessionid,sf.sf_instance)
f = contact.describe()
I get this error:
Traceback (most recent call last):
File "./simple-example.py", line 13, in <module>
f = contact.describe()["fields"]
File "/Library/Python/2.7/site-packages/simple_salesforce/api.py", line 430, in describe
result = self._call_salesforce('GET', self.base_url + 'describe')
File "/Library/Python/2.7/site-packages/simple_salesforce/api.py", line 570, in _call_salesforce
'Authorization': 'Bearer ' + self.session_id,
TypeError: cannot concatenate 'str' and 'SFType' objects
Thanks in advance for any advice.

If you look in the source code for simple-salesforce (as of 2015-11-12) you'll see that in the init() of Salesforce() we set the session to self.session_id and instance to self.sf_instance
In your case, you're using sf.sessionid, and because simple-salesforce is setup to return a SFType() object whenever a method or property does not exist on Salesforce() (and sessionid does not exist on Salesforce()) you're actually inserting a SFType() object into the init of your SFType()
SFType.__init__() doesn't do any form of validation to confirm you're passing in strings as arguments, so the error you're getting is from simple-salesforce trying to use the SFType() object you're passing in as a string.
Try this code:
contact = SFType('Contact', sf.session_id, sf.sf_instance)
f = contact.describe()

I ran into the same issue and seemed to have fixed this by removing the protocol ("https://") from the instance_url. This is weird but seems to work for me now and I can do contact.describe()
Something like this:
contact = SFType(sf_object, session_id, instance_url.replace("https://",''))
contact.describe()

How do I use python-WikEdDiff?

I recently installed python-WikEdDiff package to my system. I understand it is a python extension of the original JavaScript WikEdDiff tool. I tried to use it but I couldn't find any documentation for it. I am stuck at using WikEdDiff.diff(). I wish to use the other functions of this class, such as getFragments() and others, but on checking, it shows the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/dist-packages/WikEdDiff/diff.py", line 1123, in detectBlocks
self.getSameBlocks()
File "/usr/local/lib/python3.4/dist-packages/WikEdDiff/diff.py", line 1211, in getSameBlocks
while j is not None and self.oldText.tokens[j].link is None:
IndexError: list index out of range
On checking, I found out that the tokens[] structure in the object remains empty whereas it should have been initialized.
Is there an initialize function that I need to call apart from the default constructor? Or is it something to do with the `WikEdDiffConfig' config structure I passed to the constructor?

You get this error because the WikEdDiff object was cleared internally inside diff(), as shown in this section of the code:
def diff( self, oldString, newString ):
...
# Free memory
self.newText.tokens.clear()
self.oldText.tokens.clear()
# Assemble blocks into fragment table
fragments = self.getDiffFragments()
# Free memory
self.blocks.clear()
self.groups.clear()
self.sections.clear()
...
return fragments
If you just need the fragments, use the returned variable of diff() like this:
import WikEdDiff as WED
config=WED.WikEdDiffConfig()
w = WED.WikEdDiff(config)
f = w.diff("abc", "efg")
# do whatever you want with f, but don't use w
print(' '.join([i.text+i.type for i in f]))
# outputs '{ [ (> abc- ) abc< efg+ ] }'

how to use TTreeReader in PyROOT

I'm trying to get up and running using the TTreeReader approach to reading TTrees in PyROOT. As a guide, I am using the ROOT 6 Analysis Workshop (http://root.cern.ch/drupal/content/7-using-ttreereader) and its associated ROOT file (http://root.cern.ch/root/files/tutorials/mockupx.root).
from ROOT import *
fileName = "mockupx.root"
file = TFile(fileName)
tree = file.Get("MyTree")
treeReader = TTreeReader("MyTree", file)
After this, I am a bit lost. I attempt to access variable information using the TTreeReader object and it doesn't quite work:
>>> rvMissingET = TTreeReaderValue(treeReader, "missingET")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/ROOT/v6-03-01/root/lib/ROOT.py", line 198, in __call__
result = _root.MakeRootTemplateClass( *newargs )
SystemError: error return without exception set
Where am I going wrong here?

TTreeReaderValue is a templated class, as shown in the example on the TTreeReader documentation, so you need to specify the template type.
You can do this with
rvMissingET = ROOT.TTreeReaderValue(ROOT.Double)(treeReader, "missingET")
The Python built-ins can be used for int and float types, e.g.
rvInt = ROOT.TTreeReaderValue(int)(treeReader, "intBranch")
rvFloat = ROOT.TTreeReaderValue(float)(treeReader, "floatBranch")
Also note that using TTreeReader in PyROOT is not recommended. (If you're looking for faster ntuple branch access in Python, you might look in to the Ntuple class I wrote.)