My end goal here is to turn on my tv using my Pi. I've already setup and configured everything I can think of, I can access the pi remotely via http, but I constantly get a 404 when trying to call a macro via the REST API. Script runs fine on its own, just can't seem to be called from http.
At this point, I'd take any solution that can be executed via http. Php, cgi, etc, don't care, I just need it to run beside the current setup.
Added to config file as follows:
myscript = /home/pi/harmony.py
harmony.py
import webiopi
import sys
import os
#webiopi.macro
def HarAll():
os.system("/home/pi/Desktop/harmonycontrol/HarmonyHubControl em#i.l passwort start_activity 6463490")
When I attempt to access http://piaddress:8000/macros/HarAll I get a 404. I'm positive I'm missing a step here, for some reason, webIOPi simply isn't adding the macro to the web server.
Got it figured out, this whole time I was trying to test it instead of just adding it to the app I made, I was sending http GET from web browser instead of http POST. Works perfectly.
Related
In short:
I have a Django application being served up by Apache on a Google Compute Engine VM.
I want to access a secret from Google Secret Manager in my Python code (when the Django app is initialising).
When I do 'python manage.py runserver', the secret is successfully retrieved. However, when I get Apache to run my application, it hangs when it sends a request to the secret manager.
Too much detail:
I followed the answer to this question GCP VM Instance is not able to access secrets from Secret Manager despite of appropriate Roles. I have created a service account (not the default), and have given it the 'cloud-platform' scope. I also gave it the 'Secret Manager Admin' role in the web console.
After initially running into trouble, I downloaded the a json key for the service account from the web console, and set the GOOGLE_APPLICATION_CREDENTIALS env-var to point to it.
When I run the django server directly on the VM, everything works fine. When I let Apache run the application, I can see from the logs that the service account credential json is loaded successfully.
However, when I make my first API call, via google.cloud.secretmanager.SecretManagerServiceClient.list_secret_versions , the application hangs. I don't even get a 500 error in my browser, just an eternal loading icon. I traced the execution as far as:
grpc._channel._UnaryUnaryMultiCallable._blocking, line 926 : 'call = self._channel.segregated_call(...'
It never gets past that line. I couldn't figure out where that call goes so I couldnt inspect it any further than that.
Thoughts
I don't understand GCP service accounts / API access very well. I can't understand why this difference is occurring between the django dev server and apache, given that they're both using the same service account credentials from json. I'm also surprised that the application just hangs in the google library rather than throwing an exception. There's even a timeout option when sending a request, but changing this doesn't make any difference.
I wonder if it's somehow related to the fact that I'm running the django server under my own account, but apache is using whatever user account it uses?
Update
I tried changing the user/group that apache runs as to match my own. No change.
I enabled logging for gRPC itself. There is a clear difference between when I run with apache vs the django dev server.
On Django:
secure_channel_create.cc:178] grpc_secure_channel_create(creds=0x17cfda0, target=secretmanager.googleapis.com:443, args=0x7fe254620f20, reserved=(nil))
init.cc:167] grpc_init(void)
client_channel.cc:1099] chand=0x2299b88: creating client_channel for channel stack 0x2299b18
...
timer_manager.cc:188] sleep for a 1001 milliseconds
...
client_channel.cc:1879] chand=0x2299b88 calld=0x229e440: created call
...
call.cc:1980] grpc_call_start_batch(call=0x229daa0, ops=0x20cfe70, nops=6, tag=0x7fe25463c680, reserved=(nil))
call.cc:1573] ops[0]: SEND_INITIAL_METADATA...
call.cc:1573] ops[1]: SEND_MESSAGE ptr=0x21f7a20
...
So, a channel is created, then a call is created, and then we see gRPC start to execute the operations for that call (as far as I read it).
On Apache:
secure_channel_create.cc:178] grpc_secure_channel_create(creds=0x7fd5bc850f70, target=secretmanager.googleapis.com:443, args=0x7fd583065c50, reserved=(nil))
init.cc:167] grpc_init(void)
client_channel.cc:1099] chand=0x7fd5bca91bb8: creating client_channel for channel stack 0x7fd5bca91b48
...
timer_manager.cc:188] sleep for a 1001 milliseconds
...
timer_manager.cc:188] sleep for a 1001 milliseconds
...
So, we a channel is created... and then nothing. No call, no operations. So the python code is sitting there waiting for gRPC to make this call, which it never does.
The problem appears to be that the forking behaviour of Apache breaks gRPC somehow. I couldn't nail down the precise cause, but after I began to suspect that forking was the issue, I found this old gRPC issue that indicates that forking is a bit of a tricky area.
I tried to reconfigure Apache to use a different 'Multi-processing Module', but as my experience in this is limited, I couldn't get gRPC to work under any of them.
In the end, I switched to using nginx/uwsgi instead of Apache/mod_wsgi, and I did not have the same issue. If you're trying to solve a problem like this and you have to use Apache, I'd advice further investigating Apache forking, how gRPC handles forking, and the different MPMs available for Apache.
I'm facing a similar issue. When running my Flask Application with eventlet==0.33.0 and gunicorn https://github.com/benoitc/gunicorn/archive/ff58e0c6da83d5520916bc4cc109a529258d76e1.zip#egg=gunicorn==20.1.0. When calling secret_client.access_secret_version it hangs forever.
It used to work fine with an older eventlet version, but we needed to upgrade to the latest version of eventlet due to security reasons.
I experienced a similar issue and I was able to solve with the following:
import grpc.experimental.gevent as grpc_gevent
from gevent import monkey
from google.cloud import secretmanager
monkey.patch_all()
grpc_gevent.init_gevent()
client = secretmanager.SecretManagerServiceClient()
I finally made a project that I wanted to make since a long time :
I'm using an Arduino Uno to replace my PC power button (with a simple relay) and that Arduino Board is connected to a Raspi 3 for network connection purposes
My wish is to do a webpage (or a API-Like request) that at a touch of a button (preferably in a password-protected page) It'll power the PC on
I know how to code in Python, and my script to control the Arduino is already done but I can't find a way to run, only server-side, a Python Script from a button in a webpage
I found that CherryPy framework but I don't think it'll suit my needs
Can someone give me any ideas about that please?
As already mentioned by #ForceBru, you need a python webserver.
If this can be useful to you, this is a possible unsecure implementation using flask:
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route('/turnOn')
def hello_world():
k = request.args.get('key')
if k == "superSecretKey":
# Do something ..
return 'Ok'
else:
return 'Nope'
If you put this in an app.py name file and, after having installed flask (pip install flask), you run flask run you should be able to see Ok if visiting the url http://localhost:5000/turnOn?key=superSecretKey .
You could write a brief html gui with a button and a key field in a form but I leaves that to you (you need to have fun too!).
To avoid potential security issues you could use a POST method and https.
Look at the flask documentation for more infos.
Really newbie questions.
I made a Python bot which receives some data and has to analyze it,then prints everything. To use it, i need it to run for the whole day, the problem is that i can't leave my computer on 24/7, so i need a server or something similar for it and i need to be able to check what it prints whenever i want.
I made some research and found Heroku, but i'm having some problems understanding it: i tried to deploy it there and it's working but it prints all the stuff on a shell in the app's page and not on the webpage that heroku assigned to my app, so my problem is partially solved, since i can run it for the whole day but checking what it prints is way harder.
I was thinking of making it as a Telegram bot in order to have everything there but since it prints a lot of stuff, Telegram would not be the best platform for this kind of thing.
Is there another resource to deploy it and have it, for example, on a webpage?
You can look into renting a cheapish cloud server (from digitalocean for example).
There are multiple ways of transferring data from your python script to your bot, either directly, through a websocket, or a webpage that displays it in a JSON format or otherwise.
Since you're already using python you could look into running a flask app on your node alongside your script or even combine them together.
If ran separately you could modify your script to output it's content into a file and then read the file with your flask application to display it on a webpage. For example:
with open('/tmp/data.txt', 'w') as f:
f.write(yourdata)
then in your flask application:
from flask import Flask
app = Flask(__name__)
#app.route('/')
def show_data():
with open('/tmp/data.txt', 'r') as f:
data = f.read()
return data
There are way more efficient ways of transferring data. Example above is a quick and dirty solution I wouldn't recommend running it due to security reasons especially if you are transmitting sensitive data.
I have two scripts running, one on port :80 and one on port :81. Because some of our users are having issues with stuff happening on the server with port :81, I'm trying to implement a workaround like this;
Old way of doing it, which works fine for most users:
AngularJS app makes request to example.com:81/getpdf/1
Flask server generates PNG and PDF files using PhantomJS and ImageMagick using two separate subprocess.Popen calls and the .wait() method
Using Flask's send_file(), the PDF gets sent back to the user and starts downloading
My workaround for this issue:
AngularJS makes request to example.com/getpdf/1
Flask server (:80) makes a new GET request, r = requests.get(url_with_port_81), faking the old AngularJS request to create the PNG/PDF
Instead of using send_file(), I now return the path of the generated PDF
I return send_file(r.text)
Now, using my workaround, the subprocesses I run to create the PNG/PDFs somehow crash. I have to sudo pkill python, and only when I do so, I'm getting a PNG with no data in the folder on my server.
Basically, PhantomJS has run but hasn't loaded any data (only html/css, but no important stuff that needs to come from the Flask server) and crashes. How is this even possible? I'm just faking the request the browser makes using requests.get, or am I not aware of something here?
I thought subprocess.Popen is non-blocking, so my requests for data could still be answered to fill the PNG/PDFs?
I finally found the reason my subprocess kept crashing.
Apparently, it's a bug in Python < 2.7.3, described here: http://bugs.python.org/issue12786
I had to use 'close_fds=True' in my Popen call and all was fixed. Thanks for your effort either way, #Mark Hildreth!
I am trying to record HTTP GET/POST requests sent by my browser using the library scotch.
I am using their sample code: http://darcs.idyll.org/~t/projects/scotch/doc/recipes.html#id2
import scotch.proxy
app = scotch.proxy.ProxyApp()
import scotch.recorder
recorder = scotch.recorder.Recorder(app, verbosity=1)
try:
from wsgiref.simple_server import WSGIServer, WSGIRequestHandler
server_address = ('', 8000)
httpd = WSGIServer(server_address, WSGIRequestHandler)
httpd.set_app(app)
while 1:
httpd.handle_request()
finally:
from cPickle import dump
outfp = open('recording.pickle', 'w')
dump(recorder.record_holder, outfp)
outfp.close()
print 'saved %d records' % (len(recorder.record_holder))
So I ran above code, went over to google chrome, and visited a few sites to see if that would get recorded.
However, I do not see how the code should terminate. It seems that there has to be an error in httpd.handle_request() for the code to terminate.
I tried a variation of the code where I removed the try and finally syntax, and changed the while condition so that the loop ran for 30 seconds. However, that seems to be running forever as well.
Any ideas on how to get this working? I am also open to using other python libraries available for what I am trying to do: record my browser's GET/POST requests, including logons, and replay this within python.
Thanks.
Correct me if I'm wrong, but you're trying to log the activity of your local browser by setting a local proxy. If this is the case your browser needs to go through your proxy in order for your proxy server to log the activity.
The code that you've provided sets a proxy server at localhost:8000, so you need to tell your browser about this. The actual setting depends on the browser, I'm sure you'd be able to google it easily.
When I've asked to check if the code is running I actually mean whether your local proxy accepts some kind of request from the browser. Do you see the 'saved records' print out of your code at some point?