Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
Let's say I want to run some function once a day at 10 am.
Do I simply keep a script running in the background forever?
What if I don't want to keep my laptop open/on for many days at a time?
Will the process eat a lot of CPU?
Are the answers to these questions different if I use cron/launchd vs scheduling programmatically? Thanks!
The answer to this question will likely depend on your platform, the available facilities and your particular project needs.
First let me address system resources. If you want to use the fewest resources, just call time.sleep(NNN), where NNN is the number of seconds until the next instance of 10AM. time.sleep will suspend execution of your program and should consume zero (or virtually zero resources). The python GC may periodically wake up and do maintenance, but it's work should be negligible.
If you're on Unix, cron is the typical facility for scheduling future tasks. It implements a fairly efficient Franta–Maly event list manager. It will determine based on the list of tasks which will occurr next and sleep until then.
On Windows, you have the Schedule Manager. It's a Frankenstein of complexity -- but it's incredibly flexible and can handle running missed events due to power outages and laptop hibernates, etc...
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I am very new to the concept of threading. I was going over the content in this site about threading and came across this claim that Tasks that spend much of their time waiting for external events are generally good candidates for threading. May I know why is this statement true.
Threading allows for efficient CPU usage. Tasks that spend a lot of time waiting for other events to finish can be put to sleep (this means temporarily stopped) with Threading.
By putting a thread to sleep, the CPU it was being executed with becomes free to execute other tasks while waiting for the thread to be woken up.
The ability to sleep and wake up allows:
(1) Faster computation without much overhead
(2) A reduction in wasted computational resources
Alternative viewpoint:
I don't know about Python specifically, but in many other programming languages/libraries, there will be some kind of "async I/O" mechanism, or "select" mechanism, or "event-driven" paradigm that enables a single-threaded program to actively do one thing while simultaneously waiting for multiple other things to happen.
The problem that threads solve comes from the fact that each independent source of input/events typically drives some independent "activity," and each of those activities has its own separate state. In an async/event-driven programming style, the state of each activity must be explicitly represented in some set of variables: When the program receives an event that drives activity X, it has to be able to examine those variables so that it can "pick up where it left off" last time it was working on activity X.
With threads, part or all of the state of activity X can be implicit in the X thread's context (that is, in the value of its program counter, in its registers, and in the contents of its call stack.) The code for a thread that handles one particular activity can look a lot like the pure-procedural code that that we all first learned to write when we were rank beginners—much more familiar looking than any system of "event handlers" and explicit state variables.
The down-side of using multiple threads, is that the familiar look and feel of the code can lull us into a false sense of security—we can easily overlook the possibility of deadlocks, and race conditions, and other hazards to which multi-threading exposes us. Multi-threaded code can be easy to read, but it can be much harder to write it without making subtle mistakes that are hard to catch in testing.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I want to run the program at the exact start of a minute (eg. 1:30:00, 1:20:00, 1:19:00 [Hour,minute,seconds]) There are many similar questions but they do not have the case in which the program is linked to a defined time, not just 60 second intervals. Other solutions are in other languages. Is there any python solution to this problem?
Then I guess the best you can do is to combine the answer here:
Python threading.timer - repeat function every 'n' seconds
but change the wait condition based on current time:
from datetime import datetime
now = datetime.now()
current_time = now.strftime("%H:%M:%S")
print("Current Time =", current_time)
when you create your thread compute how much time it has to wait before it gets executed.
Then it should within miliseconds precision start at the designated time.
Cheers
There are several options available. Here are three that I know of:
You can build your own solution using the sched package in the Python standard library: https://docs.python.org/3/library/sched.html
There is a third-party package called schedule, with the periodic functionality that you requested: https://schedule.readthedocs.io/en/stable/
As suggested in the comments, the Celery task queue library has a component called "Celery beat", which you can use to schedule periodic tasks: https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#guide-beat
Note that the first two libraries are not inherently multi-threaded. I'm not sure exactly what happens if a job blocks the main thread and overlaps the next scheduled run. You will need to consult the documentation for that, or ensure that all your jobs run in a non-blocking fashion.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
This may seem like a dumb question, but I haven't found an answer in regards to this. I was coding a loop in Python using Sublime Text, and I accidentally set the incorrect conditions that lead to running an infinite loop.
After multiple bad attempts, I've noticed that my OS is slower. I was wondering if running loops would negatively harm the RAM or processing even though if I force quit the application - or maybe it was a coincidental occurrence.
Don't worry. Vaguely speaking, the CPU in your computer is always running whether you're running your application code or not. And the RAM chip is always powered on as long as the computer is running.
Actually, a DDR memory has to be periodically refreshed in order to work (think of this as periodic read-write cycles, although they're carried out by the chip itself.
So no, an infinite loop will not wear out your cpu or ram, but it could prevent some parts of them from entering low power modes, depending on the actual hardware an OS.
After multiple bad attempts, I've noticed that my OS is slower. I was wondering if running loops would negatively harm the RAM or processing even though if I force quit the application - or maybe it was a coincidental occurrence.
No. Unless the application has left behind execution artifacts (such as other zombie processes which happen to loop too), nothing will happen (after the execution of the process stops, the operating system reclaims all the resources it held).
I was wondering if running loops would negatively harm the RAM or processing even though if I force quit the application
As far as the CPU is concerned, an infinite loop is just a set of never ending (conditional or unconditional) jumps, that may or may not be followed by other meaningful instructions. It's completely harmless.
As far as I know, yes. Your RAM will start getting used up. I've had this happen to me as well and sometimes I had no choice but to force-quit the application too.
In the case of Sublime Text, just use ctrl+break to stop the execution. It may happen in certain code you write where it isn't immediately obvious that this is happening. However, you can easily check the RAM usage and you'll see it spiking!
Once the process/thread in which the loop was executing is destroyed, then the loop execution is also terminated.
There may be cleanup of resources that might need to still occur, which could have an 'impact' on depending on how those resources were managed... As far as I know python performs all the necessary cleanup preventing memory leaks, etc
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
What I want to achieve is to run python some script which will collect data and insert it to DB in a background.
So basically, a person opens Django view, clicks on a button and then closes the browser and Django launches this script on a server, the script then collects data in background while everything else goes on its own.
What is the best library, framework, module or package to achieve such functionality?
Celery is the most used tool for such tasks.
Celery is a good suggestion, but its a bit heavy solution and there more simple and straightforward solution exist unless you need full power of celery.
So i suggest to use rq and django integration of rq.
RQ inspired by the good parts of Celery, Resque , and has been created as a lightweight alternative to the heaviness of Celery or other AMQP-based queuing implementations.
I'd humbly reccomend the standard library module multiprocessing for this. As long as the background process can run on the same server as the one processing the requests, you'll be fine.
Although i consider this to be the simplest solution, this wouldn't scale well at all, since you'd be running extra processess on your server. If you expect these things to only happen once in a while, and not to last that long, it's a good quick solution.
One thing to keep in mind though: In the newly started process ALWAYS close your database connection before doing anything - this is because the forked process shares the same connection to the SQL server and might enter into data races with your main django process.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions concerning problems with code you've written must describe the specific problem — and include valid code to reproduce it — in the question itself. See SSCCE.org for guidance.
Closed 9 years ago.
Improve this question
I made a server monitoring script that is monitoring mainly network drive usage and cluster's job status. It's really basic and mainly uses unix commands such as top, status, df and such.
I rely using subprocess which works well, but under heavy workload it starts to get really slow and use a lot of cpu capacity. Slowest part is where I grep users from status -a and they have thousands of jobs running.
Script runs over endless while loop.
So I'm searching for more effective solutions to do this and any help or hint will be appreciated. I'm using Python 2.7
I can suggest you to take a look to iotop, especially the source code as it is made in python.
The global philosophy behind this is to not use the unix tools (top, df...) but parse their source of informations that is /proc.
Opening a file (especially in a memory filesystem like the procfs) is much more faster than forking a process to launch an unix command.