We Keep Coding

Scraping a public tableau dashboard

This is a follow-up question to How to scrape a public tableau dashboard? and the use of the impressive Tableau scraper library. The library has the ability to select an item in a worksheet, however it fails to recognize the requested value.
The tableau dashboard is here: https://tableau.ons.org.br/t/ONS_Publico/views/DemandaMxima/HistricoDemandaMxima?:embed=y&:display_count=y&:showAppBanner=true&:showVizHome=y
And my code is:
from tableauscraper import TableauScraper as TS
url = 'https://tableau.ons.org.br/t/ONS_Publico/views/DemandaMxima/HistricoDemandaMxima'
ts = TS()
ts.loads(url)
wb = ts.getWorkbook()
# Set units
wb.setParameter("Selecione DM Simp 4", "Demanda Máxima Instântanea (MW)")
# Set to daily resolution
wb.setParameter("Escala de Tempo DM Simp 4", "Dia")
# Set the start date
wb.setParameter("Início Primeiro Período DM Simp 4","01/01/2017")
# Set the end date
wb = wb.setParameter("Fim Primeiro Período DM Simp 4","12/31/2017")
# Retrieve daily worksheet
ws = wb.getWorksheet("Simples Demanda Máxima Semana Dia")
# Select subsystem
ws.select("ATRIB(Subsistema)", "Norte")
(This is where I am warned "tableauScraper - ERROR - 'Norte' is not in list")
# show data
print(ws.data)
# export data
ws.data.to_csv('C:\Temp\Data.csv')
Any help would be appreciated.

I've updated the library to make it work (I'm the author of TableauScraper library). There was multiple issues with this usecase :
the parameters/filters were not persisted between API calls, in this case, the setParameter didn't return the initial filters
the format of the parameter value for Fim Primeiro Período DM Simp 4 is 31/12/2017 (DD/MM/YYYY)
it's actually a filter and not a select API call that must be performed. In this case it's:
wb = ws.setFilter("Subsistema", "N")
Norte is a label, you can get the possible values using ws.getFilters()
This dashboard uses storypoints and the filters are embedded into it. Parsing filters inside storypoints was not implemented until now
The filter API call specifies the storyboard and the storypointId which were necessary to be implemented in order to make the API call work (specific to storypoints)
Also, the worksheet used to get/set the filter is Simples Demanda Máxima Ano (even though the filter is set to daily)
Using the latest release, the following works:
from tableauscraper import TableauScraper as TS
url = 'https://tableau.ons.org.br/t/ONS_Publico/views/DemandaMxima/HistricoDemandaMxima'
ts = TS()
ts.loads(url)
wb = ts.getWorkbook()
# Set units
wb.setParameter("Selecione DM Simp 4", "Demanda Máxima Instântanea (MW)")
# Set to daily resolution
wb.setParameter("Escala de Tempo DM Simp 4", "Dia")
# # Set the start date
wb.setParameter("Início Primeiro Período DM Simp 4", "01/01/2017")
# Set the end date
wb = wb.setParameter("Fim Primeiro Período DM Simp 4", "31/12/2017")
# Retrieve daily worksheet
ws = wb.getWorksheet("Simples Demanda Máxima Semana Dia")
print(ws.data[['Data Escala de Tempo 1 DM Simp 4-value',
'SOMA(Selecione Tipo de DM Simp 4)-value', 'ATRIB(Subsistema)-alias']])
ws = wb.getWorksheet("Simples Demanda Máxima Ano")
print(ws.getFilters())
# Select subsystem
wb = ws.setFilter("Subsistema", "N")
ws = wb.getWorksheet("Simples Demanda Máxima Semana Dia")
print(ws.data[['Data Escala de Tempo 1 DM Simp 4-value',
'SOMA(Selecione Tipo de DM Simp 4)-value', 'ATRIB(Subsistema)-alias']])
repl.it: https://replit.com/#bertrandmartel/TableauONSDemandaMaxima

How I can save the output in a text file

I use the library TextRazor to analyze a text and save it in a text file but when I open the file it's empty
import os
textrazor.api_key = ""
client = textrazor.TextRazor(extractors=["word","entities", "topics","sentence","words"])
response = client.analyze("Twenty-four isolates (60%) were associated with pneumonia, 14 (35%) with upper respiratory tract infections, and 2 (5%) with bronchiolitis. Cough (82.5%), fever (75%), and malaise (58.8%) ")
for entity in response.entities():
print(entity.id, entity.relevance_score, entity.confidence_score, entity.freebase_types)
cmd = os.popen('ls -w').read() # Récupération de la sortie de ls -a dans la variable cmd
print(cmd) # Affichage de la sortie
with open('monfichier.txt', 'w') as file:
s=str(cmd)
file.write(s)

ls -w is not a valid command as an integer is needed after the -w. Furthermore, do not use os.popen: it is not portable. Use the Python API instead (see os.walk).

Launching Python script including Paho-mqtt library in RPi rc.local

I have installed Paho-mqtt python libray by "pip3 install paho-mqtt" on my RPi which I use for home automation purposes.
I made a python script to regulate my heatpump.
My script:
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Programme de régulation de température
"""
import time
import json
import requests
from requests.auth import HTTPBasicAuth
import paho.mqtt.subscribe as subscribe
import paho.mqtt.publish as publish
#***********Récupération de l'état d'un élément de Domoticz***********
def dom_elmt(idx, valeur):
url= 'http://127.0.0.1/json.htm?type=devices&rid='
adresse= url+idx
r=requests.get(adresse)
status=r.status_code
if status == 200:
# l'API renvoie 200 si tout est OK
getinfos=r.json()
for i, v in enumerate(getinfos["result"]):
#Récupération de la donnée "Status" du switch
result= getinfos["result"][i][valeur]
return result
#***********FIN Récupération de l'état du clavier************
#***********
Prev_Setpoint = 0
while True:
msg = subscribe.simple("zigbee2mqtt/Thermo-Hygro Salle", hostname="127.0.0.1")
y = json.loads(msg.payload)
Temp = y["temperature"]
print(time.strftime('%H:%M:%S', time.localtime()) ,"Température salle : ",Temp)
#******** lecture du point de consigne idx: 230 ************
Consigne=dom_elmt('230', "SetPoint")
Consigne=float(Consigne)
if Temp > Consigne + 0.2:
Setpoint = Consigne - 0.5
if Temp >= Consigne - 0.2 and Temp <= Consigne + 0.2:
Setpoint = Consigne
if Temp < Consigne - 0.2:
Setpoint = Consigne + 0.5
Setpoint = str(Setpoint) # Transformation d'un float en str
if Setpoint != Prev_Setpoint: # Le point de consigne calculé est différent du précédent
a = "{"
b = '"temperature"'
c = ":"
d = "}"
sendtemp = a+b+c+Setpoint+d
publish.single("heatpump/set", sendtemp, hostname="127.0.0.1")
Prev_Setpoint = Setpoint # Mise à jour de Prev_Setpoint
If I launch this script on a console it works perfectly. I need but I can't launch it at boot in the rc.local.
....
if [ "$_IP" ]; then
printf "My IP address is %s\n" "$_IP"
fi
python /home/pi/thermostat.py &
exit 0
This don't work and I don't understand what happen ?! (I launched other scripts in rc.local whithout any problem)
Does someone has an idea to help me ???
Thanks in advancecd

This is because the python script will get killed when the script that started it exits even if it is pushed to the background with &.
The hacky solution is to prefix the command with nohup (which means the process will not get sent the Hang Up signal when the parent script exits).
The better option would be to write a proper systemd service definition for your script.

problem with send() function using socket in python

I'm using socket in python between 2 laptop to send informations. But I have an error and i can't resolve it. (I'm a debutant in python). The problem is when i trierd to send information with send() function
Traceback (most recent call last):
File "client.py", line 60, in <module>
client.send(GetMyInfo.scan())
TypeError: send() argument 1 must be string or buffer, not None
Maybe the error is due my function Getinfo ?
I'm using a class with functions (listinfo() here).
Thx
Here is the code :
import socket
import subprocess
import os,sys
#==========================================================================================
#creation d'une classe pour rendre le programme oriente objet
class GetInfo:
infos = os.uname()
currentUser = os.getlogin()
rep_actuel = os.getcwd()
myDirectory = os.listdir("/Users/")
def listInfo(self):
#afficher des informations sur le systeme
print "Informations du systeme :{} ".format(GetInfo.infos)
#afficher l'utilisateur actuel
print "Current user : {} \n".format(GetInfo.currentUser)
#afficher le path du repertoire actuel
print "Repertoire actuel : {} \n".format(GetInfo.rep_actuel)
def scan(self):
print "Liste de tous les repertoires dans Users/"
#on liste toutes les directories et sous directories
for file in GetInfo.myDirectory:
print file
#instance de l objet GetInfo (creation de l'objet)
GetMyInfo = GetInfo()
# afficage des infos de l'objet
#GetMyInfo.listInfo()
#GetMyInfo.scan()
#==========================================================================================
# on cree un objet socket
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
print "Socket successfully created"
# reservation d'un port specifique
port = 12345
# contient le nom d hote et le numero du port
#identifiant le serveur auquel on veut se connecter.
s.bind(('127.0.0.1', port))
print "socket binded to %s" %(port)
# activation du mode ecoute
# avec un nombre maximum de connexions qu il peut recevoir sur ce port sans les accepter
s.listen(5)
print "socket is listening"
#==========================================================================================
# gestion des erreurs
while True:
# on etabli la connection avec le client
client, addr = s.accept()
print 'Got connection from', addr
# send the informations about the victim.
client.send(GetMyInfo.scan())
print client.recvfrom(2048)
# on ferme la connection
s.close()

NameError: name 'move' is note defined (function)

Hey I've been working on a labyrinth game and I wrote a function to make the character (which is a robot) move. The problem is that Python says my function is not define
NameError: name 'move' is not defined
but I defined it and I imported the function file in the main file.
This is my function file (modified so it's a little bite cleaner)
import pickle
from classes.labyrinthe import *
from roboc import *
from carte import *
from creer_labyrinthe import *
def move(oldLab, direction, steps):
global isOnExit
...
return oldLab
And this is my main file (again modified...)
import os
from classes.carte import Carte
from fonctions.receiveRobotMoves import *
from fonctions.move import *
...
isOnExit = False
while isOnExit == False: #Tant que le robot n'est pas sur la sortie...
#On demande au joueur la direction qu'il veut prendre
receiveRobotMoves()
#On fait avancer le robot
conteneur = move(Labyrinthe, movesInformation['direction'],
movesInformation['steps'])
#On met à jour le labyrinthe
Labyrinthe = conteneur
Could you please tell me what I can do to make it work.