This is a follow-up question to How to scrape a public tableau dashboard? and the use of the impressive Tableau scraper library. The library has the ability to select an item in a worksheet, however it fails to recognize the requested value.
The tableau dashboard is here: https://tableau.ons.org.br/t/ONS_Publico/views/DemandaMxima/HistricoDemandaMxima?:embed=y&:display_count=y&:showAppBanner=true&:showVizHome=y
And my code is:
from tableauscraper import TableauScraper as TS
url = 'https://tableau.ons.org.br/t/ONS_Publico/views/DemandaMxima/HistricoDemandaMxima'
ts = TS()
ts.loads(url)
wb = ts.getWorkbook()
# Set units
wb.setParameter("Selecione DM Simp 4", "Demanda Máxima Instântanea (MW)")
# Set to daily resolution
wb.setParameter("Escala de Tempo DM Simp 4", "Dia")
# Set the start date
wb.setParameter("Início Primeiro Período DM Simp 4","01/01/2017")
# Set the end date
wb = wb.setParameter("Fim Primeiro Período DM Simp 4","12/31/2017")
# Retrieve daily worksheet
ws = wb.getWorksheet("Simples Demanda Máxima Semana Dia")
# Select subsystem
ws.select("ATRIB(Subsistema)", "Norte")
(This is where I am warned "tableauScraper - ERROR - 'Norte' is not in list")
# show data
print(ws.data)
# export data
ws.data.to_csv('C:\Temp\Data.csv')
Any help would be appreciated.
I've updated the library to make it work (I'm the author of TableauScraper library). There was multiple issues with this usecase :
the parameters/filters were not persisted between API calls, in this case, the setParameter didn't return the initial filters
the format of the parameter value for Fim Primeiro Período DM Simp 4 is 31/12/2017 (DD/MM/YYYY)
it's actually a filter and not a select API call that must be performed. In this case it's:
wb = ws.setFilter("Subsistema", "N")
Norte is a label, you can get the possible values using ws.getFilters()
This dashboard uses storypoints and the filters are embedded into it. Parsing filters inside storypoints was not implemented until now
The filter API call specifies the storyboard and the storypointId which were necessary to be implemented in order to make the API call work (specific to storypoints)
Also, the worksheet used to get/set the filter is Simples Demanda Máxima Ano (even though the filter is set to daily)
Using the latest release, the following works:
from tableauscraper import TableauScraper as TS
url = 'https://tableau.ons.org.br/t/ONS_Publico/views/DemandaMxima/HistricoDemandaMxima'
ts = TS()
ts.loads(url)
wb = ts.getWorkbook()
# Set units
wb.setParameter("Selecione DM Simp 4", "Demanda Máxima Instântanea (MW)")
# Set to daily resolution
wb.setParameter("Escala de Tempo DM Simp 4", "Dia")
# # Set the start date
wb.setParameter("Início Primeiro Período DM Simp 4", "01/01/2017")
# Set the end date
wb = wb.setParameter("Fim Primeiro Período DM Simp 4", "31/12/2017")
# Retrieve daily worksheet
ws = wb.getWorksheet("Simples Demanda Máxima Semana Dia")
print(ws.data[['Data Escala de Tempo 1 DM Simp 4-value',
'SOMA(Selecione Tipo de DM Simp 4)-value', 'ATRIB(Subsistema)-alias']])
ws = wb.getWorksheet("Simples Demanda Máxima Ano")
print(ws.getFilters())
# Select subsystem
wb = ws.setFilter("Subsistema", "N")
ws = wb.getWorksheet("Simples Demanda Máxima Semana Dia")
print(ws.data[['Data Escala de Tempo 1 DM Simp 4-value',
'SOMA(Selecione Tipo de DM Simp 4)-value', 'ATRIB(Subsistema)-alias']])
repl.it: https://replit.com/#bertrandmartel/TableauONSDemandaMaxima
Related
So, my problem is :
I have a file (data_test.json) on the web. I wrote and uploaded it myself.
{
"a" : 1,
"b" : 42,
"c" : 3.14
}
My local python code in Renpy read it rightly.
init python:
def get_stats():
# Déclaration des variables
global test_a
global test_b
global test_c
# Importation des modules
import requests
import json
# Atteindre le fichier
r = requests.get("https://lecot-christophe.com/rpu_json/data_test.json")
# Aquérir les données sous forme de texte
r.text
# Convertir en dictionnaire Python
var = json.loads(r.text)
# Boucler pour extraire et distribuer les résultats à travers les variables
for i in var:
test_a = var['a']
test_b = var['b']
test_c = var['c']
Then the python code makes an update. I think python send rightly the data cause of the response of the requests. post is positive.
def post_stats(file):
global test_a
global test_b
global test_c
global send_test
import requests
import json
url = "https://*monsite*/rpu_json/renpy_json_manager.php"
# Composition des données
file_name = file+".json"
aDict = {
"a":test_a,
"b":test_b,
"c":test_c
}
json_object = json.dumps(aDict)
# Adresser le fichier
data_test = requests.post(url, json=json_object)
send_test = data_test.status_code
PHP receive the posted data or not, and update the json file.
<?php
$json = $_POST['data'];
$info = json_encode($json);
$file = fopen('data_test.json','w+') or die("File not found");
fwrite($file, $info);
fclose($file);
?>
But the content of data-test.json is now just "null".
null
So... Where did I make a big mistake ?
I use the library TextRazor to analyze a text and save it in a text file but when I open the file it's empty
import os
textrazor.api_key = ""
client = textrazor.TextRazor(extractors=["word","entities", "topics","sentence","words"])
response = client.analyze("Twenty-four isolates (60%) were associated with pneumonia, 14 (35%) with upper respiratory tract infections, and 2 (5%) with bronchiolitis. Cough (82.5%), fever (75%), and malaise (58.8%) ")
for entity in response.entities():
print(entity.id, entity.relevance_score, entity.confidence_score, entity.freebase_types)
cmd = os.popen('ls -w').read() # Récupération de la sortie de ls -a dans la variable cmd
print(cmd) # Affichage de la sortie
with open('monfichier.txt', 'w') as file:
s=str(cmd)
file.write(s)
ls -w is not a valid command as an integer is needed after the -w. Furthermore, do not use os.popen: it is not portable. Use the Python API instead (see os.walk).
I have installed Paho-mqtt python libray by "pip3 install paho-mqtt" on my RPi which I use for home automation purposes.
I made a python script to regulate my heatpump.
My script:
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Programme de régulation de température
"""
import time
import json
import requests
from requests.auth import HTTPBasicAuth
import paho.mqtt.subscribe as subscribe
import paho.mqtt.publish as publish
#***********Récupération de l'état d'un élément de Domoticz***********
def dom_elmt(idx, valeur):
url= 'http://127.0.0.1/json.htm?type=devices&rid='
adresse= url+idx
r=requests.get(adresse)
status=r.status_code
if status == 200:
# l'API renvoie 200 si tout est OK
getinfos=r.json()
for i, v in enumerate(getinfos["result"]):
#Récupération de la donnée "Status" du switch
result= getinfos["result"][i][valeur]
return result
#***********FIN Récupération de l'état du clavier************
#***********
Prev_Setpoint = 0
while True:
msg = subscribe.simple("zigbee2mqtt/Thermo-Hygro Salle", hostname="127.0.0.1")
y = json.loads(msg.payload)
Temp = y["temperature"]
print(time.strftime('%H:%M:%S', time.localtime()) ,"Température salle : ",Temp)
#******** lecture du point de consigne idx: 230 ************
Consigne=dom_elmt('230', "SetPoint")
Consigne=float(Consigne)
if Temp > Consigne + 0.2:
Setpoint = Consigne - 0.5
if Temp >= Consigne - 0.2 and Temp <= Consigne + 0.2:
Setpoint = Consigne
if Temp < Consigne - 0.2:
Setpoint = Consigne + 0.5
Setpoint = str(Setpoint) # Transformation d'un float en str
if Setpoint != Prev_Setpoint: # Le point de consigne calculé est différent du précédent
a = "{"
b = '"temperature"'
c = ":"
d = "}"
sendtemp = a+b+c+Setpoint+d
publish.single("heatpump/set", sendtemp, hostname="127.0.0.1")
Prev_Setpoint = Setpoint # Mise à jour de Prev_Setpoint
If I launch this script on a console it works perfectly. I need but I can't launch it at boot in the rc.local.
....
if [ "$_IP" ]; then
printf "My IP address is %s\n" "$_IP"
fi
python /home/pi/thermostat.py &
exit 0
This don't work and I don't understand what happen ?! (I launched other scripts in rc.local whithout any problem)
Does someone has an idea to help me ???
Thanks in advancecd
This is because the python script will get killed when the script that started it exits even if it is pushed to the background with &.
The hacky solution is to prefix the command with nohup (which means the process will not get sent the Hang Up signal when the parent script exits).
The better option would be to write a proper systemd service definition for your script.
so I have a simple code that create a graphical interface with Tkinter on python, but when I convert it to exe using PyInstaller I get this issue , I don't understand what's happening
Error loading Python lib '/Users/salwafakhir/dist/app/Python': dlopen: dlopen(/Users/salwafakhir/dist/app/Python, 10): no suitable image found. Did find:
/Users/salwafakhir/dist/app/Python: code signature invalid for '/Users/salwafakhir/dist/app/Python'
here is a part of the code
creer une premiere fenetre
masterWindow= Tk()
#personnaliser ma fenetre
masterWindow.title("logistics interface")
masterWindow.geometry("1280x1024")
masterWindow.minsize(480 , 360)
masterWindow.config(background='white')
#creer frame
frameProduct_1W=Frame(masterWindow,bg='white')
image = PhotoImage(file="/Users/salwafakhir/Desktop/seaver-sangle-connecte-equitation-technologie.png")
label = Label(image=image)
label.pack(expand=YES)
#ajouter un premier texte
Label_title =Label(frameProduct_1W,text="Welcome to SeaverLogictec",font=("Time New Roman",40),bg='white', fg="#3A67C1").pack(expand=YES)
ajouter un second texte
Label_subtitle =Label(frameProduct_1W,text="We are here to help you with Stock management and order placing ",font=("Time New Roman",15),bg='white', fg="#3A67C1").pack(expand=YES)
#connection a la base de donne logistic
conn = MongoClient()
db = conn.admin
#ajouter un premier bouton
startButton= Button(frameProduct_1W,text="Let's get started",font=("Time New Roman",40),bg='white', fg='#3A67C1', command=open).pack(expand=YES)
#ajouter frameProduct_1W
frameProduct_1W.pack(expand=YES)
#afficher
masterWindow.mainloop()
I want to check if the token of a Telegram bot is real.
I tried this:
try:
bot = telepot.Bot(token)
except:
pprint("Il token inserito non è valido! Riavvia il programma e inserisci un altro token!")
ftok = open('C:/Limitatibot/token.txt','w')
ftok.write('')
ftok.close()
pprint('Premere un tasto per continuare...')
input()
quit()
But it didn't work :C. How could it be solved?