I've created an Azure Function App with python and have published an app that runs every 5 minutes. I used to go to the Function > Monitoring to see the last 30 day runs. I've checked today and all logs have disappeared and the function does not display any runs in the Overview
The last time I checked before this happened, I had loads of logs in here but now I have none. I know the function is running because if I go to Application Insight into Live Monitoring I can see the traces and also can check that the results are being processed. I haven't changed anything to the script and not sure why this is happening. Has anyone experienced this and found a fix?
EDIT
I've recreated the Function App and noticed that it creates a DefaultResourceGroup-XXX resource group with a Default Workspace in it which I remember deleting it when I first created the Function App. I've left it on and now I see the logs in Monitoring but cannot see any connections to the Function App itself. Does anyone know how does this workspace relate to the logs and is there a way I can create a more user-friendly workspace name and link it to the App?
Thank you sheldonzy. Posting your suggestions as answer to help other community members.
On your function App go to monitor there if application insights is enabled you will see an option of Run Query in Application Insights
Open run query and check exception tables in application insights
Related
I am following this tutorial: https://learn.microsoft.com/en-us/academic-services/graph/tutorial-azure-databricks-hindex
I have obtained access to the Microsoft Academic Graph data set and want to issue some basic pySpark code against it, precisely per the tutorial.
For example, this code:
# Get affiliations
Affiliations = MAG.getDataframe('Affiliations')
Affiliations = Affiliations.select(Affiliations.AffiliationId, Affiliations.DisplayName)
Affiliations.show(3)
When I run the code with 'Shift + Enter', it goes into a state of 'Running command' - and never seems to finish, even after half an hour. I have inserted a screen shot of this and attached to my post.
I have run these commands individually, and it's the last one (Affiliations.show(3)) that causes the slowness.
For example, when I run the command (Affiliations = MAG.getDataframe('Affiliations')) by itself, I actually get a result:
AffiliationId:long
Rank:integer
NormalizedName:string
DisplayName:string
GridId:string
OfficialPage:string
WikiPage:string
PaperCount:long
CitationCount:long
Latitude:float
Longitude:float
CreatedDate:date
Question: how can I debug this to find out what's causing the slowness?
Debugging a distributed application is still challenging in the notebook environment. Even though the web UI has the necessary information, there is a gap between web UIs and the development environment: it’s usually difficult to locate information in the web UI that is relevant to the code you are investigating; and there is no easy way to find historical runtime information.
Understanding how to debug with the Databricks Spark UI:
The Spark UI contains a wealth of information you can use for debugging your Spark jobs. There are a bunch of great visualizations, and we have a blog post here about those features.
For more details, click on Jobx View (Stages):
Reference: Tips to Debug Apache Spark UI with Databricks
Hope this helps.
This used to be done in the adminstrator console. Now, according to the docs, it's controlled by a setting in the application's configuration files.
I updated my app.yaml file to include these lines and redeployed it:
#
# Module Settings
# https://cloud.google.com/appengine/docs/python/modules/#Python_Configuration
#
module: default
instance_class: F2
However, I haven't noticed any improvement in my application's performance. Specifically, I have a (cpu-bound) script that was taking 4-5 secs to run and there has been no difference since the change.
So my question is: am I doing this correctly? And is there a way to confirm (for example, in the logs or elsewhere in the admin console) the level at which my application's servers are running?
I should note that I am testing this on an unbilled application. Although I couldn't find any information in the docs that indicated this feature was limited to billed applications, I know that some features are unavailable on unbilled apps.
The settings you have there look correct.
If you are using modules, and it looks like you are, you can confirm the frontend instance class is what you set it to by viewing the "Versions" page on the old app engine console at http://appengine.google.com/
If you aren't using modules you can view the instance type on the "Application Settings" page.
Unfortunately, there doesn't seem to be a way to check the frontend instance class using the new cloud console.
If you look under Instances in the application dashboard you can see which ones you currently have running.
Newbie on appengine and I really don't know how to phrase the question which sadly results in me not knowing what keywords to google and I hope that i really do get help other than the bashing that a lot of people do.
I'm confused between the behavior of appengine online and the appengine on the local server.
Background info:
Btw this is in Python
Initially i assumed that , when needed or as authored
an instance of the app or module will be created.
And that instance will be the one serving multiple requests from different clients.
In this behavior any initialization code will only be run once.
But in the local development server.
Every time i add something new, specially in the main.py,
the server is able to catch the new changes,
then on browser-refresh be able to run it.
This made me think, wait...
Does it run the entire script over and over again
on every request?
Question:
Does an instance/module run the entire code on every request or is this just an added behavior to the dev server to make development easier?
Both your assumptions - about behaviour in production and development - are wrong.
In production, GAE spins up instances as required. This may be in response to increased load, or the host may simply decide after a certain amount of time to recycle an instance by killing it and starting a new one. Initialization code will always be run whenever a new instance is started.
In development, you only get a single instance. However, the server watches your file system for changes. If it detects a change to the code itself, it will restart itself, and therefore re-run the initialization code. But if you don't make any code changes between requests, the existing process continues indefinitely, and init code will not be re-run.
This might not be bug, but feature. I'm having problem views expanded logs when searching logs in dashboard on app engine.
Search results show first couple of logs in full detail, but rest of log entries are obscured. Every new entry in log is shown in full details, but older ones get obscured over the time.
Same behavior is reflected if I try to download logs from app engine, only more log entries are not obscured.
Point is that I can't get full log of my app and would like to be able to run some tasks over data.
App Engine stores logging information in a set of circular buffers. When it runs out of space, it overwrites older log entries with the new data. What you're seeing is requests for which the detailed logs have been overwritten by newer requests.
I have Django running in Apache via mod_wsgi. I believe Django is caching my pages server-side, which is causing some of the functionality to not work correctly.
I have a countdown timer that works by getting the current server time, determining the remaining countdown time, and outputting that number to the HTML template. A javascript countdown timer then takes over and runs the countdown for the user.
The problem arises when the user refreshes the page, or navigates to a different page with the countdown timer. The timer appears to jump around to different times sporadically, usually going back to the same time over and over again on each refresh.
Using HTTPFox, the page is not being loaded from my browser cache, so it looks like either Django or Apache is caching the page. Is there any way to disable this functionality? I'm not going to have enough traffic to worry about caching the script output. Or am I completely wrong about why this is happening?
[Edit] From the posts below, it looks like caching is disabled in Django, which means it must be happening elsewhere, perhaps in Apache?
[Edit] I have a more thorough description of what is happening: For the first 7 (or so) requests made to the server, the pages are rendered by the script and returned, although each of those 7 pages seems to be cached as it shows up later. On the 8th request, the server serves up the first page. On the 9th request, it serves up the second page, and so on in a cycle. This lasts until I restart apache, when the process starts over again.
[Edit] I have configured mod_wsgi to run only one process at a time, which causes the timer to reset to the same value in every case. Interestingly though, there's another component on my page that displays a random image on each request, using order('?'), and that does refresh with different images each time, which would indicate the caching is happening in Django and not in Apache.
[Edit] In light of the previous edit, I went back and reviewed the relevant views.py file, finding that the countdown start variable was being set globally in the module, outside of the view functions. Moving that setting inside the view functions resolved the problem. So it turned out not to be a caching issue after all. Thanks everyone for your help on this.
From my experience with mod_wsgi in Apache, it is highly unlikely that they are causing caching. A couple of things to try:
It is possible that you have some proxy server between your computer and the web server that is appropriately or inappropriately caching pages. Sometimes ISPs run proxy servers to reduce bandwidth outside their network. Can you please provide the HTTP headers for a page that is getting cached (Firebug can give these to you). Headers that I would specifically be interested in include Cache-Control, Expires, Last-Modified, and ETag.
Can you post your MIDDLEWARE_CLASSES from your settings.py file. It possible that you have a Middleware that performs caching for you.
Can you grep your code for the following items "load cache", "django.core.cache", and "cache_page". A *grep -R "search" ** will work.
Does the settings.py (or anything it imports like "from localsettings import *") include CACHE_BACKEND?
What happens when you restart apache? (e.g. sudo services apache restart). If a restart clears the issue, then it might be apache doing caching (it is possible that this could also clear out a locmen Django cache backend)
Did you specifically setup Django caching? From the docs it seems you would clearly know if Django was caching as it requires work beforehand to get it working. Specifically, you need to define where the cached files are saved.
http://docs.djangoproject.com/en/dev/topics/cache/
Are you using a multiprocess configuration for Apache/mod_wsgi? If you are, that will account for why different responses can have a different value for the timer as likely that when timer is initialised will be different for each process handling requests. Thus why it can jump around.
Have a read of:
http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
Work out in what mode or configuration you are running Apache/mod_wsgi and perhaps post what that configuration is. Without knowing, there are too many unknowns.
I just came across this:
Support for Automatic Reloading To help deployment tools you can
activate support for automatic reloading. Whenever something changes
the .wsgi file, mod_wsgi will reload all the daemon processes for us.
For that, just add the following directive to your Directory section:
WSGIScriptReloading On