I am currently building a set of multiple graphs for my personal company using Datadog. I love how it works but there is only one thing I have not been able to sort out. Whenever my data is generated every 5 minutes, there are times where one or multiple values will come in at '0' which is what I want. The problem is Datadog is for some reason not taking these values into account and so until that same value finally comes in with something other than '0' then nothing will show up saying the value was '0' and then it changed to something else. Instead the graph chooses to create a straight line from the last recorded non-zero value straight to the newest non-zero value. I would love to know how I can get Datadog to consider the zeros and graph them.
In addition, if possible, I would also love to know how I could say something like "if this previous value existed and then on the next set of data it does not show up at all (not even as "0") just assign a "0" to it until it once again appears on the data". Of course for this to be looked into I would need the first problem dealt with.
Here is an image of how it is looking right now which is NOT how I wanted to look. The Red line shows where all the "0" values land, the Green boxes show the last recorded non-zero values.
Example of '0' values not being graph properly
I have tried looking through most of the documentation of Datadog as well as their posted YouTube videos with no luck. They for some reason do not address this even when it is in front of them when showing examples. I expected to find some info online but there seems to be little resources at the moment. This resulted in me thinking this could be the best place to finally get an answer.
I believe you are looking for Interpolation. There are a couple of use cases you specified, you may have to experiment with a few of the options depending on what your data looks like. For example Fill Zero satisfies one of them.
Datadog Graph Functions
Related
Like it says, I'm trying to find a method to extract data from PDFs in Python. I've explored a few solutions already, but I'm not finding any solution that fit the need.
The PDF I have is scanned in, but I can use Tesseract to turn it into a text pdf if necessary. The goal in the short term is to grab a few values from the PDF and store them. The large scale goal is to get a large number of these PDFs and perform this task automatically. I know how to store the data if I can get it out of the PDF, my problem is actually getting the values out.
I'm not at liberty to display the PDF, below is an example of what the document looks like.
Sorry for my crude art, I figured this would be easier than recreating an empty copy of the PDF, but I can make a better mock up if necessary. The fields I would like to extract are highlighted in red. Wherever it says TITLE: next to a field is where title would appear on the document, usually on a separate line, save for the field at the bottom.
I've tried using a few tools, notably Azure Cognitive Services and PyPDF2, however the issues I'm usually running into is that the output has each group of words as an individual line in the output, which does not work if the title of a form field is above it, like the example table below
left
center
right
One
Two
Three
The output returns left, then center, then right, then One, then Two, then Three. If the field for Two or One was left blank, searching for 3 rows below right would not give me the expected output.
I've run into a few other bugs with other solutions, like needing to have bounding boxes on my PDF for it to work, but I'm starting to run out of solutions to find, and I was wondering if anyone had any ideas for how I can get this task done.
There are multiple pages, however I only really need 1-2, and I only have 1 scanned with Tesseract. The format stays relatively the same, although each pdf is independently scanned in so there could be minor changes there.
Any and all help is greatly appreciated.
I am using https://explainerdashboard.readthedocs.io/en/latest/ to show/explain the predicted results from my lightgbm classification model.
I am able to deploy and run it fine. But, inside a particular tab(lets take 'Individual Predictions' as example), when I click dropdown on the index, I have three issues:
It doesn't show all the values unless I type something in the search box. Then it lists all values.
It doesn't search the typed values in search box correctly. The words just disappear after I type something.
Most importantly, the index value selected from the drop down of one component doesn't show the same index name across all components. It is getting filtered across components but still shows old index name on the other components. (But, if i click random index button it seems to show the right index across all components)
Any tips/suggestion welcome or is it just another unfixed bug in the library?
(If anybody could tag 'explainerdashboard' then that would be a great help)
I have data set, and there is something wrong with it fundamentally and I don't understand what.
Till I get to index 4807 everything is ok, and afterwards isn't. All datatypes are float64.
Image on the left is good data, and you can see that Spyder adds colors to it, while on right it doesn't.
You have two different dataframes and I suspect that the one on the right, i.e. train-bad is very large. If so, you need to know that the coloring is not available if the contents of your Dataframe are too large. This is to avaoid lags when browsing.
I have (financial) data that I get in real time using an API and I'd like to display it in a customised manner (a bit like the result of a javascript code). For example, if I want to display 10x10 prices and update them as I receive the data and customise them to be green if it is higher than the previous price, red if lower or so, how should I do, what should I use?
I assume there exist a way to do so using python, but I can't formulate my demand briefly so I only get results that confuse me more using search engines...
Could someone help me by explaining where I can get started with that?
I'll give you an overview because what you want is a generalized approach and most UI packages (if not all) should be able to handle this. First, you need to pick a package to write your UI with. There are a number of these available for Python: see here. I'm not sure what your other requirements are so you'll have to choose the one you want yourself. Once you've picked it out, you'll basically go through and create a grid structure composed of individual cells. Each cell will contain a currency value. You'll then add an event for each cell that captures an "on-change" event for the value in the cell. If the new value is greater than the old one, you color it green. If it's less, color it red. You may also want to add a timer for each cell so that the color fades after a period of time.
I am using Python to read in data in a user-unfriendly format and transform it into an easier-to-read format. The records I am outputting are usually going to be just a last name, first name, and room code. I
I would like to output a series of pages, each containing a contiguous subset of the total records, divided into multiple columns, each of which contains a contiguous subset of the total records on the page. (So in other words, you'd read down the first column, move to the next column, move to the next column, etc., and then start over on the next page...)
The problem I am facing now is that for output formats, I'm almost certainly limited to HTML (and Javascript, CSS, etc.) What is the best way to get the data into this columnar format? If I knew for certain that the printable area of the paper would hold 20 records vertically and five horizontally, for instance, I could easily print tables of 5x20, but I don't know if there's a way to indicate a page break -- and I don't know if there's any way to calculate programmatically how many records will fit on the page.
How would you approach this?
EDIT: The reason I said that I was limited in output: I have to produce the file on one computer, then bring it to a different computer upon which we cannot install new software and on which the selection of existing software is not optimal. The file itself is only going to be used to make a physical printout (which is what the end users will actually work with), but my time on the computer that I can print from is going to be limited, so I need to have the file all ready to go and print right away without a lot of tweaking.
Right now I've managed to find a word processor that I can use on the target machine, so I'm going to see if I can target a format that the word processor uses.
EDIT: Once I knew there was a word processor I could use, I made a simple skeleton file with the settings that I wanted (column and tab settings, monospaced font in a small point size, etc.) and then measured how many characters I got per line of a column and how many lines I got per column. I've watched the runs pretty carefully to make sure that there weren't some strange lines that somehow overflowed the characters-per-line guideline (which shouldn't happen with monospaced font, of course, but how many times do you end up having to figure out why that thing that "shouldn't" happen is happening anyways?)
If there hadn't been a word processor on the target machine that I could use, I probably would have looked at PDF as an output format.
"If I knew for certain that the printable area of the paper would hold 20 records vertically and five horizontally"
You do know that.
You know the size of your paper. You know the size of your font. You can easily do the math.
"almost certainly limited to HTML..." doesn't make much sense. Is this a web application? The page can have a "Previous" and "Next" button to step through the pages? Pick a size that looks good to you and display one page full with "Previous" and "Next" buttons.
If it's supposed to be one HTML page that prints correctly, that's hard. There are CSS things you can do, but you'll be happier creating a PDF file.
Get PyX or ReportLab and create a PDF that prints properly.
I -- personally -- have no patience with any of this. I try put this kind of thing into a CSV file. My users can then open CSV with a tool spreadsheet (Open Office Org has a good one) and then adjust the columns and print with it.