How do I (efficiently) loop through huge ranges in Python? Whenever I try to run something like-
for x in range(105, 10000000000000):
#do stuff
it takes an eternity to complete. I've tried many things like iter, set, xrange but still no success. Is there any external module which can help me do that?
Given the range, I think it's safe to say that almost regardless of what you do or how you do it, it's going to take a while.
To be more specific, you're executing approximately 1E+13 iterations of your loop. With decent code and a reasonable fast processor, you might be able to execute around 4E+9 instructions per second (e.g., 2 instructions per clock cycle at 2 GHz). That obviously isn't exact, but for the moment let's just go with it, and see where we end up.
So, even if we assume each iteration of the loop requires executing only one, single-cycle instruction, it's going to take approximately: 1E+13/4E+9 = 2.5E3 seconds = ~42 minutes. Given that you're doing some work inside the loop, and there's some overhead for the loop itself, we're clearly going to be execution more than one machine-code instruction per iteration. If we have to execute, say, 1000 instructions per iteration (may be a reasonable first approximation, given that you haven't told us anything about what you're doing), then we're looking at something like 600-700 hours for the loop to execute.
Bottom line: changing how you represent your range may help, but if you need to iterate across a range this large, there's no getting away from the fact that it's going to take a while.
Assuming you have a 3GHz processor, and that you just need one cycle for each entry in the range, you would need ~3,333 seconds (~1 hour) to process it.
This leads me to think that there's somehting fundamentally wrong with what you're doing. Maybe you should restructure your problem.
Related
I'm trying to write code for this and I'm completely lost. I know the answer but am new to code and don't know my way around. Can someone point me in the right direction even if its just a similar example.
A cash drawer contains 160 bills, all 10s and 50s. The total value of the 10s and 50s is $1,760.
How many of each type of bill are in the drawer? You can figure this out by trial and error (or by doing algebra with pencil and paper), but try to use loops and conditionals to check a plausible possibilities and stop when you find the correct one.
I've tried to write it out in code but keep getting syntax errors. I tried looking up loops and conditionals on google but im still confused. I think the range function is what I'm looking for but I don't know how to key it in. I don't want the answer I just want resources for finding it.
print("Print integers within given start and stop number using range
(function")
for x in range (1,160)
print(x, end=',')
Indentation error: Expected an indented block
Its has to have loop, i would suggest you use while loops, and have a variable for the dollars and take one down and add one when it doesn't match.
Look up while loops, variables and if statements and you should get it.
I'm trying to calculate bits/bytes/kilobytes/megabytes per second in a python / bash script, I don't know which one is more sufficient.
I was thinking about writing a loop which checks wether the download duration is more than one second, and if so, count how many miliseconds its shy off the 'one second' and then apply this somehow to my downloaded size to recieve my MB/s result. I don't know if this is the best solution or if this is even close.
The first variable is the amount downloaded (file size)
The second variable is the duration in seconds.
1MB;0,127
2MB;0,563
3MB;0,655
Am I going in the right direction with my idea or is there a different (more sufficient) method?
I have a long running process that is mostly IO bound. It is basically just a loop uploading items somewhere, some of these items take more time than others, some days the whole process is slower so the time can't be hardcoded.
Is there a module that given the progress through the loop in terms of (current position, final position) could evaluate the first few iterations then give an estimate of the remaining time, but also update on every iteration?
I'm thinking something like the progress output you get from tools like wget and apt-get.
I guess I could write it myself but I wondered if something like this exists already.
I writing some code to get sensor readings from GPIO against time.
To make sure the measurements corresponds to a specific time, I want to know if python iterates at a constant speed (so that the gap between iterations is constant) - and what is its minimum time gap between iterations.
If they're not, can someone let me know how to make the time gap constant.
Thank you!
No, Python does not and can not iterate at constant speed.
Python is just another process on your Raspberry PI, and your OS is responsible for allocating it time to run on the CPU (called multi-tasking). Other processes also get allotted time. This means Python is never going to be running all the time and any processing times are going to be depend on what the other processes are doing.
Iteration itself is also delegated to specific types; how the next item is produced then varies widely, and even if Python was given constant access to the CPU iteration would still vary. Whatever you do in your loop body also takes time, and unless the inputs and outputs are always exactly the same, will almost certainly take a variable amount of time to do the work.
Instead of trying to time your loops, measure time with time.time() or timeit.default_timer (depending on how precise you need to be, on your Raspberry it'll be the same function) in a loop and adjust your actions based on that.
Sorted by total time, the second longest executing function is "{built-in method mainloop}" ? I looked at the same entry with pstats_viewer.py and clicked it and it says :
Function Exclusive time Inclusive time Primitive calls Total calls Exclusive per call Inclusive per call
Tkinter.py:359:mainloop 0.00s 561.03s (26.3%) 1 1 0.00s 561.03s
What does this mean?
Edit
Here's part of the cProfile output from a longer run of my code. The more ODE's I solve, the more time is devoted to mainloop. This is crazy! I thought that my runtime was getting killed by either branch divergence in my CUDA kernel or Host-GPU memory transfers. God, I'm a horrible programmer!
How have I made Tkinter take so much of my runtime?
mainloop is the event loop in Tkinter. It waits for events and processes them as they come in.
This is a recurring thing that you will see in all GUIs as well as any other event-driven frameworks like Twisted or Tornado.
First of all, it's a lot easier to see if you change tabs to spaces, as in:
Function Exclusive time Inclusive time Primitive calls Total calls Exclusive per call Inclusive per call
Tkinter.py:359:mainloop 0.00s 561.03s (26.3%) 1 1 0.00s 561.03s
Exclusive time means time that the program counter was in that routine. For a top-level routine you would expect this to be practically zero.
Inclusive time means including time in all routines that the routine calls. For a top-level routine you would expect this to be practically 100%.
(I don't understand what that 26.3% means.)
If you are trying to get more speed, what you need to do is find activity that 1) has a high percent inclusive time, and 2) that you can do something about.
This link shows the method I use.
After you speed something up, you will still find things that take a high percent inclusive time, but the overall elapsed time will be less.
Eventually you will get to a point where some things still take a high percent, but you can no longer figure out how to improve it.