We Keep Coding

How a for in loop in python ends when there is no update statement in it? [duplicate]

This question already has an answer here:
How does the Python for loop actually work?
(1 answer)
Closed 3 months ago.
For example:
#1
val = 5
for i in range(val) :
print(i)
When the range is exhausted i.e. last value reached how python knows for in loop ends . As in other languages
#2
for(i=0;i<=5;i++){
print(i)
}
As in this exp. when i's values becomes larger than 5 false condition leads to termination of loop .
I tried reading docs of python and browsed over google but no satisfying answer. So unable to get a picture of this .

So this is actually a complicated question, but the very rough version of the answer is "the compiler/interpreter can do what it wants".
It isn't actually running the human-readable text you write at all - instead it goes through a whole pipeline of transformations. At minimum, a lexer converts the text to a sequence of symbols, and then a parser turns that into a tree of language constructs; that may then be compiled into machine code or interpreted by a virtual machine.
So, the python interpreter creates a structure that handles the underlying logic. Depending on the optimizations performed (those are really a black box, it's hard to say what they do), this may be producing structures logically equivalent to what a Java-like for loop would make, or it could actually create a data structure of numbers (that's what the range() function does on its own) and then iterate over them.
Editing to give some more foundation for what this construct even means:
Python iteration-style loops are different in how they're defined from C-style i++ sorts of loops. Python loops are intended to iterate on each element of a list or other sequence data structure - you can say, for instance, for name in listOfNames, and then use name in the following block.
When you say for i in range(x), this is the pythonic way of doing something like the C-style loop. Think of it as the reverse of
for(int i = 0; i < arr.length(); i++){
foo(arr[i[)
}
In that code block you're accessing each element of an indexible sequence arr by going through each valid index. You don't actually care about i - it's just a means to an end, a way to make sure you visit each element.
Python assumes that's what you're trying to do: the python variant is
for elem in arr:
foo(elem)
Which most people would agree is simpler, clearer and more elegant.
However, there are times when you actually do want to explicitly go number by number. To do that with a python style, you create a list of all the numbers you'll want to visit - that's what the range function does. You'll mostly see it as part of a loop statement, but it can exist independently - you can say x = range(10), and x will hold a list that consists of the numbers 0-9 inclusive.
So, where before you were incrementing a number to visit each item of a list, now you're taking a list of numbers to get incrementing values.
"How it does this" is still explanation I gave above - the parser and interpreter know how to create the nitty-gritty logic that actually creates this sequence and step through it, or possibly transform it into some logically equivalent steps.

Why is one way more efficient than another?

I was trying out hackerrank, and I came across a problem which I tried to solve using python3.
The problem was
"A kidnapper wrote a ransom note but is worried it will be traced back to him. He found a magazine and wants to know if he can cut out whole words from it and use them to create an untraceable replica of his ransom note. The words in his note are case-sensitive and he must use whole words available in the magazine, meaning he cannot use substrings or concatenation to create the words he needs.
Given the words in the magazine and the words in the ransom note, print Yes if he can replicate his ransom note exactly using whole words from the magazine; otherwise, print No."
I tried using the following approach,
def ransom_note(magazine, ransom):
# comparing based on the number of times word occurred in the list
for word in set(ransom):
if ransom.count(word) > magazine.count(word):
return False
return True
This did work, I got 18 out of 20 test cases right.
But the other two cases were timing out, so I had to get the best cost effective way of doing this.
I tried to store the words as a dictionary by using the word as the key and count of the word as the value. Still not getting those two cases, when i looked into the cases there was 30000 words for both the inputs and the output expected was "Yes".
I saw the discussion's page and found a piece of code that got me through.
from collections import Counter
def ransom_note(magazine, ransom):
return not (Counter(ransom) - Counter(magazine))
Can someone explain why this was more efficient than my method?
Thanks in advance :)

As I understand it, in your second attempt at the problem, both ransom and magazine were dictionaries, so theoretically your code was as fast as it could be.
The Python Counter collection is designed specifically to work with simple integer counts, and optimized to perform common operations very quickly. It turns out that seeing if there are enough things in one list to satisfy the requests from another list is a really common operation. So they spent time optimizing Counter do that operation very quickly.

Most efficient way to check if any substrings in list are in another list of strings

I have two lists, one of words, and another of character combinations. What would be the fastest way to only return the combinations that don't match anything in the list?
I've tried to make it as streamlined as possible, but it's still very slow when it uses 3 characters for the combinations (goes up to 290 seconds for 4 characters, not even going to try 5)
Here's some example code, currently I'm converting all the words to a list, and then searching the string for each list value.
#Sample of stuff
allCombinations = ["a","aa","ab","ac","ad"]
allWords = ["testing", "accurate" ]
#Do the calculations
allWordsJoined = ",".join( allWords )
invalidCombinations = set( i for i in allCombinations if i not in allWordsJoined )
print invalidCombinations
#Result: set(['aa', 'ab', 'ad'])
I'm just curious if there's a better way to do this with sets? With a combination of 3 letters, there are 18278 list items to search for, and for 4 letters, that goes up to 475254, so currently my method isn't really fast enough, especially when the word list string is about 1 million characters.
Set.intersection seems like a very useful method if you need the whole string, so surely there must be something similar to search for a substring.

The first thing that comes to mind is that you can optimize lookup by checking current combination against combinations that are already "invalid". I.e. if ab is invalid, than ab.? will be invalid too and there's no point to check such.
And one more thing: try using
for i in allCombinations:
if i not in allWordsJoined:
invalidCombinations.add(i)
instead of
invalidCombinations = set(i for i in allCombinations if i not in allWordsJoined)
I'm not sure, but less memory allocations can be a small boost for real data run.

Seeing if a set contains an item is O(1). You would still have to iterate through your list of combinations (with some exceptions. If your word doesn't have "a" it's not going to have any other combinations that contain "a". You can use some tree-like data structure for this) to compare with your original set of words.
You shouldn't convert your wordlist to a string, but rather a set. You should get O(N) where N is the length of your combinations.
Also, I like Python, but it isn't the fastest of languages. If this is the only task you need to do, and it needs to be very fast, and you can't improve the algorithm, you might want to check out other languages. You should be able to very easily prototype something to get an idea of the difference in speed for different languages.

How to stop sqlite from return unicode symbol

I could be mistaken but since I backtracked to python 2.7 from 3.2 I've noticed something sqlite3 is doing that I don't like...
When I use to use my cursor.fetch method it would return (1, x, x, x) from my table
but in 2.7 its returning (1, u'x' ,u'x' ,u'x')
How do I get it to stop returning that 'u' in my queries? I need to store the return values in variables and it's kind of irritating having to problem solve this.
Edit: Through a little research I've found a way...
db.text_factory = str
if there is a better way please inform me as I really don't even know what this method does, hate to be changing the wrong thing

Sigh.
(1, u'x' ,u'x' ,u'x') is a tuple. That tuple contains four elements. One of them is an integer, the other three are unicode strings, as indicated by the u.
If you print the entire tuple at once, it will look like the above. But that's a silly thing to do, because presumably you actually want to do something with the contents of the tuple, not the tuple itself.
If you did print result[1], you would see the output is just x: the u is not shown when you actually print it. That is the correct behaviour.
If you use the code you post, as soon as you retrieve a non-ASCII value from your datastore - someone whose name contains an accented character, for example - things will go horribly wrong.

How to work with very long strings in Python?

I'm tackling project euler's problem 220 (looked easy, in comparison to some of the
others - thought I'd try a higher numbered one for a change!)
So far I have:
D = "Fa"
def iterate(D,num):
for i in range (0,num):
D = D.replace("a","A")
D = D.replace("b","B")
D = D.replace("A","aRbFR")
D = D.replace("B","LFaLb")
return D
instructions = iterate("Fa",50)
print instructions
Now, this works fine for low values, but when you put it to repeat higher then you just get a "Memory error". Can anyone suggest a way to overcome this? I really want a string/file that contains instructions for the next step.

The trick is in noticing which patterns emerge as you run the string through each iteration. Try evaluating iterate(D,n) for n between 1 and 10 and see if you can spot them. Also feed the string through a function that calculates the end position and the number of steps, and look for patterns there too.
You can then use this knowledge to simplify the algorithm to something that doesn't use these strings at all.

Python strings are not going to be the answer to this one. Strings are stored as immutable arrays, so each one of those replacements creates an entirely new string in memory. Not to mention, the set of instructions after 10^12 steps will be at least 1TB in size if you store them as characters (and that's with some minor compressions).
Ideally, there should be a way to mathematically (hint, there is) generate the answer on the fly, so that you never need to store the sequence.
Just use the string as a guide to determine a method which creates your path.

If you think about how many "a" and "b" characters there are in D(0), D(1), etc, you'll see that the string gets very long very quickly. Calculate how many characters there are in D(50), and then maybe think again about where you would store that much data. I make it 4.5*10^15 characters, which is 4500 TB at one byte per char.
Come to think of it, you don't have to calculate - the problem tells you there are 10^12 steps at least, which is a terabyte of data at one byte per character, or quarter of that if you use tricks to get down to 2 bits per character. I think this would cause problems with the one-minute time limit on any kind of storage medium I have access to :-)

Since you can't materialize the string, you must generate it. If you yield the individual characters instead of returning the whole string, you might get it to work.
def repl220( string ):
for c in string:
if c == 'a': yield "aRbFR"
elif c == 'b': yield "LFaLb"
else yield c
Something like that will do replacement without creating a new string.
Now, of course, you need to call it recursively, and to the appropriate depth. So, each yield isn't just a yield, it's something a bit more complex.
Trying not to solve this for you, so I'll leave it at that.

Just as a word of warning be careful when using the replace() function. If your strings are very large (in my case ~ 5e6 chars) the replace function would return a subset of the string (around ~ 4e6 chars) without throwing any errors.

You could treat D as a byte stream file.
Something like:-
seedfile = open('D1.txt', 'w');
seedfile.write("Fa");
seedfile.close();
n = 0
while (n
warning totally untested