I have a HTTP endpoint in Google Functions named process_me:
def process(request: Request) -> str:
return process_message(request)
which generates /process_me traces in GCP.
Now to trace a bit further I add OpenTelemetry:
def process(request: Request) -> str:
with TRACER.start_as_current_span("process_message") as span:
return process_message(request)
which in turn leads to traces no longer starting with /process_me. Instead they are named/have URL process_message.
Everything that is traced underneath the span is properly attached to process_message, though.
So I wonder how do I add Span process_message to Span /process_me (instead of replacing it)?
Notice that /process_me is the trace entry of the Loadbalancer. So I want this to go beyond Network Services.
The best I could come up with was:
def _build_trace_links(headers: EnvironHeaders) -> List[Link]:
"""
Tries to extract GCP trace context from headers.
Example header would be:
```
X-Cloud-Trace-Context: TRACE_ID/SPAN_ID;o=TRACE_TRUE
```
"""
trace_header: str = headers.get("X-Cloud-Trace-Context", "")
if not trace_header:
return []
ids, opts = trace_header.split(";", 1)[0]
b = opts.split("=", 1)[1]
if b.upper() != "TRACE_TRUE":
return []
trace_id, span_id = ids.split("/", 1)
parent_context = SpanContext(
trace_id=int(trace_id, 16),
span_id=int(span_id),
is_remote=True,
)
return [Link(context=parent_context)]
This way you can extract the parent and add at least a link to process_message.
I could see you found some relevant information already but it is not parent-child relationship in the trace and you are doing your own parsing and linking. I believe what you need here is the propagator that does all the work for you. I am not really super familiar with Google Cloud Function but I think this should help.
Install the propagator
python -m pip install opentelemetry-propagator-gcp
And the set the global text map propagator
...
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.cloud_trace_propagator import CloudTraceFormatPropagator
...
set_global_textmap(CloudTraceFormatPropagator())
...
Is it possible to create a new instance of type T using generics in python?
I know how to do it in C# and have found plenty of examples but cant find anything for python.
For example:
public class Animal<T> where T : ISound, new(){
public T GetInstance()
{
return new T();
}}
Is there a python equivalent to the above C# snippet?
This is what I thought my python code needed to look like:
from typing import TypeVar, Generic
T = TypeVar('T'
class crud(Generic[T])
def create(self, endpoint: str, body, files=None) ->T:
url = self._build_url_str(endpoint)
res = self.http.post(url, json=body).json()
return T.__init__(res)
But I get TypeError: A single constraint is not allowed.
I google that and what I found didn't work or seemed irelevant.
Also, type T has what it needs to parse the response in the constructor so it has to be parametrized.
As stated in the answer here, you can do the following:
class Animal(Generic[T]):
def get_instance(self) -> T:
return self.__orig_class__.__args__[0]()
I have a Pyramid web service, and code samples are as follows:
View declaration:
#view_config(route_name="services/Prices/GetByTicker/")
def GET(request):
ticker = request.GET('ticker')
startDate = request.GET('startDate')
endDate = request.GET('endDate')
period = request.GET('period')
Routing:
config.add_route('services/Prices/GetByTicker/', 'services/Prices/GetByTicker/{ticker}/{startDate}/{endDate}/{period}')
Now I know this is all screwed up but I don't know what the convention is for Pyramid. At the moment this works inasmuch as the request gets routed to the view successfully, but then I get a "Dictionary object not callable" exception.
The URL looks horrible:
#root/services/Prices/GetByTicker/ticker=APPL/startDate=19981212/endDate=20121231/period=d
Ideally I would like to be able to use a URL something like:
#root/services/Prices/GetByTicker/?ticker=APPL&startDate=19981212&endDate=20121231&period=d
Any Pyramid bods out there willing to take five minutes to explain what I'm doing wrong?
from you sample code, i think you use the URL Dispatch
so it should be like this
config.add_route('services/Prices/GetByTicker/', 'services/Prices/GetByTicker/')
then the URL like:
#root/services/Prices/GetByTicker/?ticker=APPL&startDate=19981212&endDate=20121231&period=d
will match it
--edit--
you don't have to use a name like "services/Prices/GetByTicker" for route_name,and you can get the GET params use request.params['key']
View declaration:
#view_config(route_name="services_Prices_GetByTicker")
def services_Prices_GetByTicker(request):
ticker = request.params['ticker']
startDate = request.params['startDate']
endDate = request.params['endDate']
period = request.params['period']
Routing:
config.add_route('services_Prices_GetByTicker', 'services/Prices/GetByTicker/')
The query string is turned into the request.GET dictionary. You are using parenthesis to call the dictionary instead of accessing items via the brackets. For a url such as
#root/services/Prices/GetByTicker/?ticker=APPL&startDate=19981212&endDate=20121231&period=d
request.GET['ticker'] # -> 'APPL' or an exception if not available
request.GET.get('ticker') # -> 'APPL' or None if not available
request.GET.get('ticker', 'foo') # -> 'APPL' or 'foo' if not available
request.GET.getall('ticker') # -> ['APPL'] or [] if not available
The last option is useful if you expect ticker to be supplied multiple times.
request.params is a combination of request.GET and request.POST where the latter is a dictionary representing the request's body in a form upload.
Anyway, the answer is that request.GET('ticker') syntactically is not one of the options I mentioned, stop doing it. :-)
[EDIT: I'm running Python 2.7.3]
I'm a network engineer by trade, and I've been hacking on ncclient (the version on the website is old, and this was the version I've been working off of) to make it work with Brocade's implementation of NETCONF. There are some tweaks that I had to make in order to get it to work with our Brocade equipment, but I had to fork off the package and make tweaks to the source itself. This didn't feel "clean" to me so I decided I wanted to try to do it "the right way" and override a couple of things that exist in the package*; three things specifically:
A "static method" called build() which belongs to the HelloHandler class, which itself is a subclass of SessionListener
The "._id" attribute of the RPC class (the original implementation used uuid, and Brocade boxes didn't like this very much, so in my original tweaks I just changed this to a static value that never changed).
A small tweak to a util function that builds XML filter attributes
So far I have this code in a file brcd_ncclient.py:
#!/usr/bin/env python
# hack on XML element creation and create a subclass to override HelloHandler's
# build() method to format the XML in a way that the brocades actually like
from ncclient.xml_ import *
from ncclient.transport.session import HelloHandler
from ncclient.operations.rpc import RPC, RaiseMode
from ncclient.operations import util
# register brocade namespace and create functions to create proper xml for
# hello/capabilities exchange
BROCADE_1_0 = "http://brocade.com/ns/netconf/config/netiron-config/"
register_namespace('brcd', BROCADE_1_0)
brocade_new_ele = lambda tag, ns, attrs={}, **extra: ET.Element(qualify(tag, ns), attrs, **extra)
brocade_sub_ele = lambda parent, tag, ns, attrs={}, **extra: ET.SubElement(parent, qualify(tag, ns), attrs, **extra)
# subclass RPC to override self._id to change uuid-generated message-id's;
# Brocades seem to not be able to handle the really long id's
class BrcdRPC(RPC):
def __init__(self, session, async=False, timeout=30, raise_mode=RaiseMode.NONE):
self._id = "1"
return super(BrcdRPC, self).self._id
class BrcdHelloHandler(HelloHandler):
def __init__(self):
return super(BrcdHelloHandler, self).__init__()
#staticmethod
def build(capabilities):
hello = brocade_new_ele("hello", None, {'xmlns':"urn:ietf:params:xml:ns:netconf:base:1.0"})
caps = brocade_sub_ele(hello, "capabilities", None)
def fun(uri): brocade_sub_ele(caps, "capability", None).text = uri
map(fun, capabilities)
return to_xml(hello)
#return super(BrcdHelloHandler, self).build() ???
# since there's no classes I'm assuming I can just override the function itself
# in ncclient.operations.util?
def build_filter(spec, capcheck=None):
type = None
if isinstance(spec, tuple):
type, criteria = spec
# brocades want the netconf prefix on subtree filter attribute
rep = new_ele("filter", {'nc:type':type})
if type == "xpath":
rep.attrib["select"] = criteria
elif type == "subtree":
rep.append(to_ele(criteria))
else:
raise OperationError("Invalid filter type")
else:
rep = validated_element(spec, ("filter", qualify("filter")),
attrs=("type",))
# TODO set type var here, check if select attr present in case of xpath..
if type == "xpath" and capcheck is not None:
capcheck(":xpath")
return rep
And then in my file netconftest.py I have:
#!/usr/bin/env python
from ncclient import manager
from brcd_ncclient import *
manager.logging.basicConfig(filename='ncclient.log', level=manager.logging.DEBUG)
# brocade server capabilities advertising as 1.1 compliant when they're really not
# this will stop ncclient from attempting 1.1 chunked netconf message transactions
manager.CAPABILITIES = ['urn:ietf:params:netconf:capability:writeable-running:1.0', 'urn:ietf:params:netconf:base:1.0']
# BROCADE_1_0 is the namespace defined for netiron configs in brcd_ncclient
# this maps to the 'brcd' prefix used in xml elements, ie subtree filter criteria
with manager.connect(host='hostname_or_ip', username='username', password='password') as m:
# 'get' request with no filter - for brocades just shows 'show version' data
c = m.get()
print c
# 'get-config' request with 'mpls-config' filter - if no filter is
# supplied with 'get-config', brocade returns nothing
netironcfg = brocade_new_ele('netiron-config', BROCADE_1_0)
mplsconfig = brocade_sub_ele(netironcfg, 'mpls-config', BROCADE_1_0)
filterstr = to_xml(netironcfg)
c2 = m.get_config(source='running', filter=('subtree', filterstr))
print c2
# so far it only looks like the supported filters for 'get-config'
# operations are: 'interface-config', 'vlan-config' and 'mpls-config'
Whenever I run my netconftest.py file, I get timeout errors because in the log file ncclient.log I can see that my subclass definitions (namely the one that changes the XML for hello exchange - the staticmethod build) are being ignored and the Brocade box doesn't know how to interpret the XML that the original ncclient HelloHandler.build() method is generating**. I can also see in the generated logfile that the other things I'm trying to override are also being ignored, like the message-id (static value of 1) as well as the XML filters.
So, I'm kind of at a loss here. I did find this blog post/module from my research, and it would appear to do exactly what I want, but I'd really like to be able to understand what I'm doing wrong via doing it by hand, rather than using a module that someone has already written as an excuse to not have to figure this out on my own.
*Can someone explain to me if this is "monkey patching" and is actually bad? I've seen in my research that monkey patching is not desirable, but this answer and this answer are confusing me quite a bit. To me, my desire to override these bits would prevent me from having to maintain an entire fork of my own ncclient.
**To give a little more context, this XML, which ncclient.transport.session.HelloHandler.build() generates by default, the Brocade box doesn't seem to like:
<?xml version='1.0' encoding='UTF-8'?>
<nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
<nc:capabilities>
<nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability>
<nc:capability>urn:ietf:params:netconf:capability:writeable-running:1.0</nc:capability>
</nc:capabilities>
</nc:hello>
The purpose of my overridden build() method is to turn the above XML into this (which the Brocade does like:
<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<capabilities>
<capability>urn:ietf:params:netconf:base:1.0</capability>
<capability>urn:ietf:params:netconf:capability:writeable-running:1.0</capability>
</capabilities>
</hello>
So it turns out that the "meta info" should not have been so hastily removed, because again, it's difficult to find answers to what I'm after when I don't fully understand what I want to ask. What I really wanted to do was override stuff in a package at runtime.
Here's what I've changed brcd_ncclient.py to (comments removed for brevity):
#!/usr/bin/env python
from ncclient import manager
from ncclient.xml_ import *
brcd_new_ele = lambda tag, ns, attrs={}, **extra: ET.Element(qualify(tag, ns), attrs, **extra)
brcd_sub_ele = lambda parent, tag, ns, attrs={}, **extra: ET.SubElement(parent, qualify(tag, ns), attrs, **extra)
BROCADE_1_0 = "http://brocade.com/ns/netconf/config/netiron-config/"
register_namespace('brcd', BROCADE_1_0)
#staticmethod
def brcd_build(capabilities):
hello = brcd_new_ele("hello", None, {'xmlns':"urn:ietf:params:xml:ns:netconf:base:1.0"})
caps = brcd_sub_ele(hello, "capabilities", None)
def fun(uri): brcd_sub_ele(caps, "capability", None).text = uri
map(fun, capabilities)
return to_xml(hello)
def brcd_build_filter(spec, capcheck=None):
type = None
if isinstance(spec, tuple):
type, criteria = spec
# brocades want the netconf prefix on subtree filter attribute
rep = new_ele("filter", {'nc:type':type})
if type == "xpath":
rep.attrib["select"] = criteria
elif type == "subtree":
rep.append(to_ele(criteria))
else:
raise OperationError("Invalid filter type")
else:
rep = validated_element(spec, ("filter", qualify("filter")),
attrs=("type",))
if type == "xpath" and capcheck is not None:
capcheck(":xpath")
return rep
manager.transport.session.HelloHandler.build = brcd_build
manager.operations.util.build_filter = brcd_build_filter
And then in netconftest.py:
#!/usr/bin/env python
from brcd_ncclient import *
manager.logging.basicConfig(filename='ncclient.log', level=manager.logging.DEBUG)
manager.CAPABILITIES = ['urn:ietf:params:netconf:capability:writeable-running:1.0', 'urn:ietf:params:netconf:base:1.0']
with manager.connect(host='host', username='user', password='password') as m:
netironcfg = brcd_new_ele('netiron-config', BROCADE_1_0)
mplsconfig = brcd_sub_ele(netironcfg, 'mpls-config', BROCADE_1_0)
filterstr = to_xml(netironcfg)
c2 = m.get_config(source='running', filter=('subtree', filterstr))
print c2
This gets me almost to where I want to be. I still have to edit the original source code to change the message-id's from being generated with uuid1().urn because I haven't figured out or don't understand how to change an object's attributes before __init__ happens at runtime (chicken/egg problem?); here's the offending code in ncclient/operations/rpc.py:
class RPC(object):
DEPENDS = []
REPLY_CLS = RPCReply
def __init__(self, session, async=False, timeout=30, raise_mode=RaiseMode.NONE):
self._session = session
try:
for cap in self.DEPENDS:
self._assert(cap)
except AttributeError:
pass
self._async = async
self._timeout = timeout
self._raise_mode = raise_mode
self._id = uuid1().urn # Keeps things simple instead of having a class attr with running ID that has to be locked
Credit goes to this recipe on ActiveState for finally cluing me in on what I really wanted to do. The code I had originally posted I don't think was technically incorrect - if what I wanted to do was fork off my own ncclient and make changes to it and/or maintain it, which wasn't what I wanted to do at all, at least not right now.
I'll edit my question title to better reflect what I had originally wanted - if other folks have better or cleaner ideas, I'm totally open.
Is there a way to display the name/content/functions of all attributes of a given object in a jinja template. This would make it easier to debug a template that is not acting as expected.
I am building a website using the hyde framework and this would come in quite handy since I am still learning the intricacies of both jinja and hyde.
Originally, I had thought it would work to use the attr filter, but this seems to require a name value. I would like to to not have to specify the name in order to get all available attributes for the object.
Some google searching showed django syntax looks like the following, but I am not familiar with django so this may only apply to database items. Long story short, I would like a method that works kind of like this for any object named obj
{% for field, value in obj.get_fields %}
{{ field }} : {{ value }} </br>
{% endfor %}
final solution:
#jayven was right, I could create my own jinja2 filter. Unfortunately, using the stable version of hyde (0.8.4), this is not a trivial act of having a filter in the pythonpath and setting a simple yaml value in the site.yaml file (There is a pull-request for that). That being said, I was able to figure it out! So the following is my final solution which ends up being very helpful for debugging any unkown attributes.
It's easy enough to create site-specific hyde extensions just create a local python package with the following directory tree
hyde_ext
__init__.py
custom_filters.py
Now create the extension:
from hyde.plugin import Plugin
from jinja2 import environmentfilter, Environment
debug_attr_fmt = '''name: %s
type: %r
value: %r'''
#environmentfilter
def debug_attr(env, value, verbose=False):
'''
A jinja2 filter that creates a <pre> block
that lists all the attributes of a given object
inlcuding the value of those attributes and type.
This filter takes an optional variable "verbose",
which prints underscore attributes if set to True.
Verbose printing is off by default.
'''
begin = "<pre class='debug'>\n"
end = "\n</pre>"
result = ["{% filter escape %}"]
for attr_name in dir(value):
if not verbose and attr_name[0] == "_":
continue
a = getattr(value, attr_name)
result.append(debug_attr_fmt % (attr_name, type(a), a))
result.append("{% endfilter %} ")
tmpl = Environment().from_string("\n\n".join(result))
return begin + tmpl.render() + end
#return "\n\n".join(result)
# list of custom-filters for jinja2
filters = {
'debug_attr' : debug_attr
}
class CustomFilterPlugin(Plugin):
'''
The curstom-filter plugin allows any
filters added to the "filters" dictionary
to be added to hyde
'''
def __init__(self, site):
super(CustomFilterPlugin, self).__init__(site)
def template_loaded(self,template):
super(CustomFilterPlugin, self).template_loaded(template)
self.template.env.filters.update(filters)
To let hyde know about the extension add hyde_ext.custom_filters.CustomFilterPlugin to the "plugins" list of the site.yaml file.
Lastly, test it out on a file, you can add this to some random page {{resource|debug_attr}} or the following to get even the underscore-attributes {{resource|debug_attr(verbose=True)}}
Of course, I should add, that it seems like this might become much easier in the future whenever hyde 1.0 is released. Especially since there is already a pull request waiting to implement a simpler solution. This was a great way to learn a little more about how to use jinja and hyde though!
I think you can implement a filter yourself, for example:
from jinja2 import *
def show_all_attrs(value):
res = []
for k in dir(value):
res.append('%r %r\n' % (k, getattr(value, k)))
return '\n'.join(res)
env = Environment()
env.filters['show_all_attrs'] = show_all_attrs
# using the filter
tmpl = env.from_string('''{{v|show_all_attrs}}''')
class Myobj(object):
a = 1
b = 2
print tmpl.render(v=Myobj())
Also see the doc for details: http://jinja.pocoo.org/docs/api/#custom-filters