If I am running a container within Cloud Run and do a print statement in my python code. Where can I view it? Cloud logs seem to show logs for the contain itself(build, etc)?
to debug my code often I do write statements that help me figure what's going on. Where would that print output be located?
1] You can find all the logs including your print statement output in Cloud Logging as mentioned in this link. So when you write a print statement from your service they will be automatically picked up by Cloud Logging.
2] Steps to view logs in Cloud Logging: Logs Explorer -> Cloud Run Revision.
3] You may wanna check your logging level. For example: if you have configured level as logging.ERROR in basicConfig (default is WARNING), and used logging.info() in your code, then it will not be printed. You can refer to this link for more information.
4] Also, you may try flushing the stdout which will make sure the logs get written from buffers. You may refer Stackoverflow answer on how to do this.
Related
I've created an Azure Function App with python and have published an app that runs every 5 minutes. I used to go to the Function > Monitoring to see the last 30 day runs. I've checked today and all logs have disappeared and the function does not display any runs in the Overview
The last time I checked before this happened, I had loads of logs in here but now I have none. I know the function is running because if I go to Application Insight into Live Monitoring I can see the traces and also can check that the results are being processed. I haven't changed anything to the script and not sure why this is happening. Has anyone experienced this and found a fix?
EDIT
I've recreated the Function App and noticed that it creates a DefaultResourceGroup-XXX resource group with a Default Workspace in it which I remember deleting it when I first created the Function App. I've left it on and now I see the logs in Monitoring but cannot see any connections to the Function App itself. Does anyone know how does this workspace relate to the logs and is there a way I can create a more user-friendly workspace name and link it to the App?
Thank you sheldonzy. Posting your suggestions as answer to help other community members.
On your function App go to monitor there if application insights is enabled you will see an option of Run Query in Application Insights
Open run query and check exception tables in application insights
In python 2.7, the app engine sdk was doing the work in the background to nest all logs with the parent request to have a correlation in Google StackDriver.
As of the transition to python 3, it is through the usage of google cloud logging or structured logging, and from all the different references I could found, it's important to have the same trace id in the 'sub' logs for stack driver to make a match with the 'request' log.
And still as you can see below, it still appear as different logs.
For context, I even tried this on an empty django project deployed on app engine.
Got the same result, even when following the example in the documentation:
https://cloud.google.com/run/docs/logging#writing_structured_logs
Trying to log to the stdout is giving the same result.
Edit:
After the initial request, all other request will be nested under the initial request when using the stdout.
But, the highest severity of the 'child' logs is not taken by the 'parent' log, therefore the filters won't pick up the actual log. See below:
Thanks for the question!
It looks like you're logging the trace correctly, but your logName indicates that you're not using a stdout or stderr. If you use one of these for your logs, they will correlate properly, like this:
StackDriver Logs Screenshot
You can see that the logName ends with stdout. An stdout or stderr will correlate. You can create this as shown here in the tutorial:
# Build structured log messages as an object.
global_log_fields = {}
# Add log correlation to nest all log messages
# beneath request log in Log Viewer.
trace_header = request.headers.get('X-Cloud-Trace-Context')
if trace_header and PROJECT:
trace = trace_header.split('/')
global_log_fields['logging.googleapis.com/trace'] = (
f"projects/{PROJECT}/traces/{trace[0]}")
# Complete a structured log entry.
entry = dict(severity='NOTICE',
message='This is the default display field.',
# Log viewer accesses 'component' as jsonPayload.component'.
component='arbitrary-property',
**global_log_fields)
print(json.dumps(entry))
EDIT:
To filter out the stdout and only see the Request logs in the stackdriver UI, you can de-select the stdout from the filter. Logs Filter
For a sample using the python client API, please see this article and attached sample Flask app. Combining correlated Log Lines in Google Stackdriver
I was able to achieve this kind of logging structure on Google Cloud Logging Console:
I was using the Django Framework. I wrote Django middleware which integrates Google Cloud Logging API.
"Trace" needs to be added to every log object which points to its parent log object.
please check manage nesting of logs in Google Stackdriver with Django
Please check django-google-stackdriver-nested-logging log_middleware.py source on Github.
I'm trying to narrow down some potential issues in Maximo. Can Python help me get all queries certain users run while they are logged in? If Python can do this can someone please provide an example I can work with?
You don't need to program anything to achieve that.
To show the SQL queries executed by the system (and the users), you need to set the SQL logger to level INFO. Basic config will output the SQL run in the SystemOut file.
You can start with this article:
https://www-01.ibm.com/support/docview.wss?uid=swg21264064
Hope that helps.
The direct, simple answer to your question is, No.
If you don't want to log SQL for all users, as you would get with #JPTremblay's excellent answer, then you should investigate the Configure Custom Logging action of the Logging application. In the associated dialog, on the Thread Logger tab, you should be able to configure UI logging for your particular user, and add the SQL logger in the Thread Logger Details.
I'm struggling getting something going that should be pretty simple. I'm working on some uniittests that calls code with some very verbose debug logging. How can I tell the logging module "set everything to debug, except this one module, make it info"
disable query logging - Google Appengine SDK Python
check the log filter, which can be used as module based filter.
I'm working on a website that uses Python web.py. There is a form where the user enters input and when the submit button is hit, a python page is called (matches) using the .getJSON JQuery function show below.
function buildMatches(input){
$.getJSON("/matches", {e:input}, function(json){
//Returned JSON object is worked on
}
}
The "matches" python page grabs a value from a DB, runs some string reg ex and returns a JSON object. Everything works fine, my question is how would I be able to output something from the python page "matches" to see what is exactly happening during the reg ex operations? I've tried print "" (python 2.5), but I understand that would print to the console. I've done some research but couldn't find anything for my situation. I don't necessarily need to print it out to the HTML page, just any where where I can see what's going on. Thanks in advance for any help.
I have access to the webserver (SSH, SFTP, etc.), I tried to log by importing the logging module, below is the code I used. I could get it to log if I ran the page from the command line, but not when it is called by the JS page.
import logging
logging.basicConfig(filename='./SomeClass.log', filemode='w', level=logging.DEBUG)
class SomeClass:
logging.info('Started')
logging.info('Another log')
def __init__(self, obj):
logging.info('In the init')
def another_functio(self):
logging.info('Logging inside the function')
I've tried setting the full path of the log and I still have the same problem where the log file will only be written or updated when I run this class from the console. This doesn't work when the class is called by the webserver.
logging.basicConfig(filename='/home/.../.../.../example.log', filemode='w', level=logging.DEBUG)
Depending on how much access you have to the web server you can run your code manually so web.py uses its built-in web server. Then print statements will end up on the console.
Otherwise, have you thought about simply writing to your own log file somewhere accessible with a browser?
Thanks again for all the help. After digging more into the setup of the Apache server and Python implementation I was able to find a solution to help me see what's going and debug my web app. I saw that Apache config is setup to log errors and WSGI also blocks (pukes on) std.out. What I was able to do is redirect the print command to the Apache error log files.
print >> sys.stderr, "Debugging print with variable" + variable
Then I check the Apache error log to start debugging the web app. I thought I would share this in case anyone else ran into this problem as it was a pain for me.
Thanks again.