I have a generic flask application. The application is instrumented using opentelemetry-instrumentation-flask, I am shipping this data with the opentelemetry-exporter-otlp to an Elastic APM server. This is all working fine and it is done as the documentation shows.
There are some endpoints of the application that I would like not to track with the instrumentation, as they are noisy and add little-to-no value for me (For example, health endpoints). I want the instrumentation to ignore them, but I cannot find how.
How can this be done? I have been checking the documentation for Python and after searching over the internet, I could not find any clear answer about how this could be done... because I believe this must be doable.
You can do that using environment variable OTEL_PYTHON_FLASK_EXCLUDED_URLS. It takes a comma separated regular expressions for urls you want to exclude. For more detailed info please check here https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-flask#exclude-lists
Related
So I recently became interested into knowing how to create a URL shortener without using bittly or other things, but I am not very good at using python to connect with other stuff. All I know is:
Checking to see if the URL is available ( Only to see if it has HTTP:// and unavailable characters, nothing to see if the domain is occupied or not. )
All of the other things... I need help with.
By the way, I COMPLETLY do not understand how to do that, so it would be great if you add comments to show me what is going on.
I suggest you take a look at Flask, it is a framework for building web applications (APIs, web apps, etc.).
DigitalOcean has a nice tutorial on this.
You can either use hashing algorithms for the custom shortened urls, or even let the user pick more readable names (like bit.ly/my-url). In this case you would be storing in a database the shortened url and the long url.
I have a setup where I will have having around 50 sites using the same code, configured slightly differently. Rather than deploy the same code over and over again, duplicated across different folders and repositories, is there any way in Flask to centralise the working code of the site, as some sort of library?
In Django they have something a bit like this:
https://docs.djangoproject.com/en/dev/ref/contrib/sites/
Some ideas
Deploy 50 instances of UWSGI, duplicating the same code and different config
Deploy 50 instances of UWSGI, with the python code added as a sort of module or extension so there is only one instance of the code: http://flask.pocoo.org/docs/extensiondev/
Deploy 1 instance of UWSGI which has only one instance of the code and handles different hostnames: http://flask.pocoo.org/docs/patterns/appdispatch/
The code that I am duplicating is designed to query an API and show the results. The differences between the sites are two fold:
Templating - Although the sites will look similar, they will not be identical. They will have slightly different CSS and images.
The API query. Most of the smaller sites are for towns and cities. This means that the API request from these sites will be slightly modified so as to return results only in that area.
sitelondon.com might query the API for items only in London by default
sitehtml.com might query the API for items which have the "html" keyword by default
The focus on my end is performance for the user. I will be running these initially on a server with 2GB RAM which should be plenty.
Normally for these types of scenarios the "behaves differently based on Host" logic is built in the application.
So the better solution is #3 but my suggestion is to not use app dispatch.
Build the logic to get the configuration for the hostname directly in the main application (for example you can load the specialized configurations in a #before_request handler and use a single DB instance).
If you plan to use only one small server, as you said, this solution is light on resource.
50 different uWSGI instances with their own processes would fill your memory and start swapping easily.
I agree with #Paolo that #3 is your best option.
You could simplify it even more with URL rewriting in your web server. If you rewrite the URLs such that a query for http://sitelondon.com/example becomes http://sitelondon.com/london/example, and a query for http://sitehtml.com/example becomes http://sitehtml.com/html/example then you can easily get the site through the routes:
#app.route('/<site>/example')
def example(site):
return render_template(site + '/example.html')
With this setup you can organize your templates in sub-folders based on the site name, and then selecting the proper template becomes a matter of building the template path.
I hope this helps!
I'm trying to find a solution to my current problem. Let me explain: I need to find a Search UI that can consume a REST service of my choice and be highly configurable. I've searched the web and found Blacklight Search UI (written in Ruby) for Solr. I've also looked at Haystack (for django) which seems to be more promissing because somewhere in the docs i found out that you can link Haystack to your custom search engine. Out of the box Haystack supports Solr, Xapian and 2 others which i can't remember now.
What i'm trying to find is a UI written in Java, PHP(last resort!) or Python that will allow me to specify the endpoints for my APIs and with a few configurations (i'm not expecting it to run out of the box) it should be able to query the APIs and return results.
If that is not possible then could somebody suggest me something that gets close to what I described and allows me to write my own backend code that will link to the APIs ? A Haystack example will also do...
Thanks
I'm interested in this topic as well. I know about the SESAT framework supporting FAST, Solr, Yahoo!, generic XML and more, but it is old and not well maintained, and also tries to do much more than a simple front-end.
You also have AJAX-Solr which obviously only supports Solr.
I have forwarded your question on Twitter, hope others will fill in as well.
I have a large amount of automatically generated html files that I would like to push to my Plone website with a script. I currently generate the files, log into Plone, click edit on each individual page and copy and paste the html into the editor. I'd like to automate this. It would be nice to retain the plone versioning, have a auto generated comment for the edit, and come from a specific user.
I've read and tried Webdav with little luck at getting it working consistently and know that there is a way to connect to plone via ftp, but haven't tried it. I'm not sure if these are the methods that I need.
My google searches aren't leading me to anything useful. Any ideas on where to start looking for a solution to this? Or any tips on implementing it?
You can script anything in Plone via the following methods:
Through-the-web via API calls (e.g. XML-RPC, wsapi, etc.)
The bin/instance run script provided by plone.recipe.zope2instance (See charm for an example of this).
You can also use a migration framework like:
collective.transmogrifier
which allows you to write migration code, and trigger it via GenericSetup or Browser view. Additionally, there are applications written on top of Transmogrifier aimed roughly at what you are describing, the most popular of which is:
funnelweb
I would recommend that you consider using or writing a Transmogrifier "blueprint(s)" to do your import, and execute the pipeline with a tool that makes that easy:
mr.migrator
You can find blueprints by searching PyPI for "transmogrify". One popular set of blueprints is:
quintagroup.transmogrifier
One of the main attractions to the Transmogrifier approach, aside from getting the job done, is the ability to share useful blueprints with others.
I think transmogrifier is the best tool for this job, but this will definitely be a programming task no matter how you do it. It's used for many such migration jobs such as migrating from drupal.
There's an add-on, wsapi4plone.core that pumazi at WebLion started that provides web services for portals which you can then hook into. You can create, modify, delete content via XML-RPC calls. The only caveat is that it doesn't yet work with Collections (criteria specifically).
project: http://pypi.python.org/pypi/wsapi4plone.core
docs: http://packages.python.org/wsapi4plone.core/
You can also do it programmatically by hooking into the ZODB via Python (zopepy or some other method).
These should get you started:
http://plone.org/documentation/kb/manipulating-plone-objects-programmatically/reading-and-writing-field-values - you should be able to get an understanding of accessors and mutators (setters and getters), in your case you are going to be more than likely working with obj.Text (getter) and obj.setText (setter).
https://weblion.psu.edu/trac/weblion/wiki/AutomatingObjectCreation - lots of examples (slightly outdated but still relevant)
http://plone.org/documentation/faq/upload-images-files
Try to enable Webdav or ftp in Plone, then you can access Plone via webdav or ftp clients, pushing the html files. Plone (Zope) will recognises the html files as Pages.
I am really new to python, just played around with the scrapy framework that is used to crawl websites and extract data.
My question is, how to I pass parameters to a python script that is hosted somewhere online.
E.g. I make following request mysite.net/rest/index.py
Now I want to pass some parameters similar to php like *.php?id=...
Yes that would work. Although you would need to write handlers for extracting the url parameters in index.py. Try import cgi module for this in python.
Please note that there are several robust python based web frameworks available (aka Django, Pylons etc.) which automatically parses your url & forms a dictionary of all it's parameters, plus they do much more like session management, user authentication etc. I would highly recommend you use them for faster code turn-around and less maintenance hassles.