I have a python code that exports data into a csv, then I load that csv into Tableau. I don't want to load a csv into tableau.
Is there a way I can just take the return value of my python script and put it directly into tableau?
Question is similar to Export data from Python into Tableau using JSON? , but they inquire about Json format and the answer was to use csv. I want to skip csv step if possible. The data im returning will be in a tabular format.
Consider using TabPy. It returns values created by scripts as calculated fields within a workbook.
Otherwise, Tableau purely operates as a layer on top of a source of data. There is no such thing, unfortunately, as loading data directly into it.
The way I typically handle jobs like this is to have Python load data directly into a data store of some sort. All new information - or overwritten information - is then viewable with the existing data connection of your Tableau workbook.
A simple way to do this, without having to use a formal database is to consistently load the CSV into the same place with the same name. Then once Tableau is opened with the pre-existing connection, all that is needed is a click of 'refresh.'
I understand that your goal is to have Tableau populate your data directly from Python, and I'm sorry this isn't the answer you probably wanted, but there is some backend work that needs to be done. If done properly, though, the end effect to any user could appear to render directly from Python.
Make sure your current Tableau extract is a "Tableau Data Source" and then use the "Tableau Data Extract Command-Line Utility" to push the data into Tableau. You use os.system or subprocess to call the tableau executable. One caveat is this thing only work on Windows.
https://onlinehelp.tableau.com/current/pro/desktop/en-us/extracting_TDE.htm
Loading a csv file example from the Tableau site:
C:\Program Files\Tableau\Tableau 2019.1\bin>tableau addfiletoextract --server https://our_server_name --username OurServerSignIn --password "OurServerPwd" --project "New Animations" --datasource "CurrentYrOverYrStats" --file "C:\Users\user1\Documents\DataUploadFiles\AprMay.csv"
One of the existing options in the Tableau Desktop "Connect" screen is "Web Data Connector," which can connect to any kind of web server and get data through a well-documented protocol. I suspect this would include a web server running on localhost written in Python.
If you want to go this route, it would require either changing your existing Python script to turn it into a web server that transmits the script's output over the web data connection, or (preferably) making some very general Python code that could that could wrap an arbitrary script of any type in this way, exposing its output as a web data connection.
You will, of course, have to leave that Python script running, listening for connections from Tableau.
Related
I need to share some network data-visualization to people outside of my organization, that are not super good with IT (meaning they wouldn't know how to execute some python script I could give them). This data is rather confidential so I can't simply set up a Dash app and give them access, also the network graphs are rather big, so images are not satisfying.
I see two options that would meet my needs:
Sending them Cytoscape objects they just have to import on the Cytoscape desktop software
Sending HTML objects that they could just open in their browser (using cytoscape.js rendering typically).
So far, I managed to do the following:
Pulling data from elasticsearch using python, enriching the data, and setting up node and edges object into a json format.
Converting this data into ipycytoscape object that I'm able to render properly within a Jupyter Notebook
Converting to Dash cytoscape and rendering in a browser locally
Where I'm stuck is when it comes to exporting this in a format that's easy to exploit.
I couldn't find proper documentation on how to:
either export Cytoscape.js data so it's usable on the Cytoscape desktop client
Export Dash cytoscape-rendered graph into self-sufficient HTML document. I've seen some code examples working with plotly (https://plotly.com/python/interactive-html-export/)
Could someone help me on that?
Thank you very much,
Arnaud
I have a website with nothing on it and I am trying to create a python program that someone can launch, and create a temporary page on my website, so for example my website is example.com and it would create website.com/aw34ed with a random url extension, and if someone were to enter the temporary webpage it would send data to the python program where ever that person is running it. I would also like the temporary website extension/page to delete itself and not exist after a certain amount of time.
I understand this question is kind of vague so am I kind of wondering what I would use to do this and how difficult this would be to create
You could use Flask variables (as this answer states: https://stackoverflow.com/a/63839730/15447633).
#app.route('/temp/<id>')
def show_temp_page(id):
# some method for getting the temporary page, perhaps a db lookup
I'm developing my Django website since about 2 months and I begin to get a good global result with my own functions.
But, now I have to start a very hard part (to my mind) and I need some advices, ideas before to do that.
My Django website creates some PDF files from HTML templates with Django variables. Up to now, I'm saving PDF files directly on my Desktop (in a specific folder) but it's completely unsecured.
So, I installed another web application which is named LogicalDoc in order to save PDF file directly on this application. PDF files are created and sent to LogicalDoc.
LogicalDoc owns 2 API : SOAP and REST (http://wiki.logicaldoc.com/rest/#/) and I know that Django could communicate with REST method.
I'm reading this part of Django documentation too in order to understand How I can process : https://docs.djangoproject.com/en/dev/topics/http/file-uploads/
I made a scheme in order to understand what I'm exposing :
Then, I write a script which makes some things :
When the PDF file is created, I create a folder inside LogicalDoc which takes for example the following name : lastname_firstname_birthday
Two possibilities : If the folder exists,I don't create a new folder, else I create it.
Once it's done, I send the PDF file directly inside the folder by comparing PDF name with folder name to do that
I have some questions about this process :
Firstly, is it possible to make this kind of things ?
Is it hard to do that ?
What kind of advices could you give me ?
Thank you so much !
PS : If you need some part of my script, mainly PDF creating part, I can post it just after my question ;)
An idea is pretty simple, however it always requires some practice.
I strongly advice you to use REST api and forget about SOAP as the only thing it can bring to you - is 'pain' :)
If we check documentation, document/create it gives next information.
Endpoint we have to communicate with.
[protocol]://[server]:[port]/document/create
HTTP method to use - POST
List of parameters to provide with your request: body,
document, content
Even more, you can test API by clicking on "Try it out" button and check requests in "Network" tab of your browser (if you open Developer Tools)
I am not sure what kind of metadata do you have to provide in 'document' parameter but what I know you can easy get an idea of what should be done by testing it and putting XML or JSON data into 'document' parameter.
Content is an array of bytes transferred to the server (which would be your file).
To sum up, a request to 'document/create' uri will be simple
body = { 'headers': {},'object': {},}
document = "<note>data</note>"
content=open('report.xls', 'rb') #r - reading, b - binary
r = requests.post('http://logicaldoc/document/create', body=body, document=document, content=content)
Please keep in mind that file transferring requests take time and sometimes you may get timeout exception. Your code will stop and will be waiting for response, so it may be a good idea to get some practice with asyncio or celery. Just keep in mind those kind of possible issues.
I have a simulator application that continuously spits out data, formatted in JSON, to a given host name and port number (UDP). I would like to be able to point the simulator output to a Django web application so that I can monitor/process the data as it comes in.
How do I receive and process data in real time using Django? What tools or packages are available to accomplish this? I did come across this answer: How to serve data from UDP stream over HTTP in Python?, but I don't completely understand.
Ex: Similar to this page: http://money.cnn.com/data/markets/
ALSO, I don't need to store any of the streaming data in a database. I just need to perform lookups based on the streaming data. Maybe it's not a Django issue at all?
Using Javascript.
Create a webpage with all the results, and then use javascript to collect the data from the page, and update it every X seconds.
Have the webpage be the JSON data, and the javascript grab it an interpret it.
get html code using javascript with a url
Then update the page using javascript. ww3 schools has great JS tutorials
I want to email out a document that will be filled in by many people and emailed back to me. I will then parse the responses using Python and load them into my database.
What is the best format to send out the initial document in?
I was thinking an interactive .pdf but do not want to have to pay for Adobe XI. Alternatively maybe a .html file but I'm not sure how easy it is to save the state of it once its been filled in in order to be emailed back to me. A .xls file may also be a solution but I'm leaning away from it simply because it would not be a particularly professional looking format.
The key points are:
Answers can be easily parsed using Python
The format should common enough to open on most computers
The document should look relatively pleasing to the eye
Send them a web-page with a FORM section, complete with some Javascript to grab the contents of the controls and send them to you (e.g. in JSON format) when they press "submit".
Another option is to set it up as a web application. There are several Python web frameworks that could be used for that. You could then e-mail people a link to the web-app.
Why don't you use Google Docs for the form. Create the form in Google Docs and save the answer in an excel sheet. And then use any python Excel format reader (Google them) to read the file. This way you don't need to parse through mails and will be performance friendly too. Or you could just make a simple form using AppEngine and save the data directly to the database.