python - High time consumption and low efficiency in programming challenge - python

I was trying to solve this problem.
Recently Oz has found a magical string consisting of single digit "1". After experimenting on the string, Oz found a weird magical property of the string that is whenever he touches the string then each digit "1" of string changed to digit "0" and each digit "0" of string changed to "01". Oz found this property interesting and immediately asked a question to RK : "How many 1's and 0's will be in the magical string if he touches the string M times ?"
I wrote the following code for it:
l = [] #List of values
for x in range(int(raw_input())):
l.append(int(raw_input()))
def after_touchs(n, string): #Main function finds the no. of 0's and 1's
for x in range(n):
string = string.replace('1', '2').replace('0', '01').replace('2', '0')
return map(str, [string.count('1'), string.count('0')])
for num in l:
print ' '.join(after_touchs(num, '1'))
I don't understand why this code is taking a lot of time. To me it seems perfectly normal and does not use much time. Since it didn't work on the site and ran the code with the interpreter on my computer and even an input of 50 seemed to large. Does the string.replace function take up too much time? So what alternatives can I use to it? Please help me reduce the time consumption and increase the efficiency of the code.

You just need to count the number of 1 and 0, string manipulation is always heavy so I guess that's what slow you down.
number1 = 1
number0 = 0
for i in xrange(M):
# 0 -> 01
newnumber1 = number0
# 1 -> 0 and 0 -> 01
number0 += number1
# we replace the number1 with the new number
number1 = newnumber1
print "%d %d"%(number0,number1)
EDIT
There is a more efficient solution, that I saw in Tim Stopfer comment.
In fact, the number of 0's and 1's follow a fibonnacci sequence after the first change.
1: 101123
0: 011235
M: 012345
Which mean an O(1) solution would be:
if M>0:
number1 = Fibo(M-1)
number0 = Fibo(M)
But you have to approximate the value of the Fibonnacci Sequence with a formula found in wikipedia

It might be because you edit the string each time.
You are basically implementing the fibonacci sequence and the 50th number is
50 : 12586269025 = 52 x 11 x 101 x 151 x 3001
So you have a string of this length and you apply several string operations to it.
This may cause the process to slow down.
I hope I could help.

according to the question the strings for the first 6 touches would look like this:
"1", "0", "01", "010", "01001", "01001010", "0100101001001"
and the counts will be
1 0, 0 1, 1 1, 1 2, 2 3, 3 5, 5 8
which reminds me of fibonacci series.
fibonacci numbers increase rapidly so you will end up with very long strings which take up a lot of memory and are slow to manipulate.
If you need speed then just calculate the fibonacci numbers.
You can also speed up the naive fibonacci by caching values that you already calculate so if you already know 4th and 5th elements of the series you can quickly calculate the 6th.

public class MagicalString {
public static void main(String[] args) {
String modifiedString = touchTheString();
countOneAndZero(modifiedString);
}
private static void countOneAndZero(String modifiedString) {
Map<Character,Integer> map = new HashMap<Character,Integer>();
char [] data = modifiedString.toCharArray();
for(char c : data) {
if(map.containsKey(c)) {
map.put(c, map.get(c)+1);
}else {
map.put(c, 1);
}
}
System.out.println(map);
System.out.println(map.get('0'));
System.out.println(map.get('1'));
}
static String touchTheString() {
String input = "1";
int touch = 5;
while (touch != 0) {
StringBuffer br = new StringBuffer();
char[] temp = input.toCharArray();
for (char c : temp) {
if (c == '0') {
input = br.append("01").toString();
} else if (c == '1') {
input = br.append("0").toString();
}
}
touch--;
}
System.out.println(input);
return input;
}
}

Related

Optimise a Python/ C++ algorithm

I was participating in a competitive programming contest, and faced a question where out of four test cases, my answer was correct in 3, but exceeded time limit in 4th.
I tried to get better results by converting my code from python to cpp (I know that time complexity remains same, but it was worth a shot :))
Following is the question:
A string is said to be using strong language if it contains at least K consecutive characters '*'.
You are given a string S with length N. Determine whether it uses strong language or not.
Input:
The first line of the input contains a single integer T denoting the number of test cases. The description of T test cases follows.
The first line of each test case contains two space-separated integers N and K.
The second line contains a single string S with length N.
Output:
Print a single line containing the string "YES" if the string contains strong language or "NO" if it does not
My python approach:
for _ in range(int(input())):
k = int(input().split()[1])
s = input()
s2 = "".join(["*"]*k)
if len(s.split(s2))>1:
print("YES")
else:
print("NO")
My converted Cpp code (converted it myself)
#include <iostream>
#include<string>
using namespace std;
int main() {
// your code goes here
int t;
std::cin >> t;
for (int i = 0; i < t; i++) {
/* code */
int n,k;
std::cin >> n >> k;
string str;
cin >> str;
string str2(k,'*');
size_t found = str.find(str2);
if (found != string::npos){
std::cout << "YES" << std::endl;
} else {
std::cout << "NO" << std::endl;
}
}
return 0;
}
Please guide me how can I reduce my time complexity?
Other approaches : "Using find() function instead of split or using for loop"
Edit:
Sample Input :
2
5 1
abd
5 2
*i**j
Output :
NO
YES
The bounds you posted suggest that linear time is OK in Python. You can simply keep a running track of how many asterisks you have seen in a row.
T = int(input())
for _ in range(T):
n, k = map(int, input())
s = input()
count, ans = 0, False
for c in s:
if c == "*":
count += 1
else:
count = 0
ans = ans or count >= k
if ans:
print("NO")
else:
print("YES")
I can also tell you why you are TLE'ing. Consider the case where n = 1e6, k = 5e5, and s is a string where the first k-1 characters are asterisks. The find method you have is going to check every position for matching the str2 you created. This will take O(n^2) time, giving you a TLE.

Translation from Python to R. Big difference in execution time

I am translating a piece of code from Python to R to locate the position of a series of numbers when the sum duplicates. The translation is to me one-to-one, but the python code takes less than a second to execute compared to the R code which takes around an hour. I wonder if I miss anything in the data structure of both languages. Why is there such a big time difference in executing the codes? What could be a better or efficient alternative when I write the following Python code in R?
Python:
def calculate(data):
found = set([0])
total = 0
while True:
for num in data:
total = total + num
if total in found:
return total
else:
found.add(total)
R:
calculate <- function(input) {
found = set(0)
total = 0
while (TRUE) {
for (num in input) {
total = total + num
if (total %in% found) {
return(total)
} else {
found <- c(found, total)
}
}
}
}
at a guess this is a combination of most things being immutable in R (hence the vector concatenation will be allocating new objects every time) and R's %in% operator being O(N), compared to in being O(1) in Python

How do I find all 32 bit binary numbers that have exactly six 1 and rest 0

I could do this in brute force, but I was hoping there was clever coding, or perhaps an existing function, or something I am not realising...
So some examples of numbers I want:
00000000001111110000
11111100000000000000
01010101010100000000
10101010101000000000
00100100100100100100
The full permutation. Except with results that have ONLY six 1's. Not more. Not less. 64 or 32 bits would be ideal. 16 bits if that provides an answer.
I think what you need here is using the itertools module.
BAD SOLUTION
But you need to be careful, for instance, using something like permutations would just work for very small inputs. ie:
Something like the below would give you a binary representation:
>>> ["".join(v) for v in set(itertools.permutations(["1"]*2+["0"]*3))]
['11000', '01001', '00101', '00011', '10010', '01100', '01010', '10001', '00110', '10100']
then just getting decimal representation of those number:
>>> [int("".join(v), 16) for v in set(itertools.permutations(["1"]*2+["0"]*3))]
[69632, 4097, 257, 17, 65552, 4352, 4112, 65537, 272, 65792]
if you wanted 32bits with 6 ones and 26 zeroes, you'd use:
>>> [int("".join(v), 16) for v in set(itertools.permutations(["1"]*6+["0"]*26))]
but this computation would take a supercomputer to deal with (32! = 263130836933693530167218012160000000 )
DECENT SOLUTION
So a more clever way to do it is using combinations, maybe something like this:
import itertools
num_bits = 32
num_ones = 6
lst = [
f"{sum([2**vv for vv in v]):b}".zfill(num_bits)
for v in list(itertools.combinations(range(num_bits), num_ones))
]
print(len(lst))
this would tell us there is 906192 numbers with 6 ones in the whole spectrum of 32bits numbers.
CREDITS:
Credits for this answer go to #Mark Dickinson who pointed out using permutations was unfeasible and suggested the usage of combinations
Well I am not a Python coder so I can not post a valid code for you. Instead I can do a C++ one...
If you look at your problem you set 6 bits and many zeros ... so I would approach this by 6 nested for loops computing all the possible 1s position and set the bits...
Something like:
for (i0= 0;i0<32-5;i0++)
for (i1=i0+1;i1<32-4;i1++)
for (i2=i1+1;i2<32-3;i2++)
for (i3=i2+1;i3<32-2;i3++)
for (i4=i3+1;i4<32-1;i4++)
for (i5=i4+1;i5<32-0;i5++)
// here i0,...,i5 marks the set bits positions
So the O(2^32) become to less than `~O(26.25.24.23.22.21/16) and you can not go faster than that as that would mean you miss valid solutions...
I assume you want to print the number so for speed up you can compute the number as a binary number string from the start to avoid slow conversion between string and number...
The nested for loops can be encoded as increment operation of an array (similar to bignum arithmetics)
When I put all together I got this C++ code:
int generate()
{
const int n1=6; // number of set bits
const int n=32; // number of bits
char x[n+2]; // output number string
int i[n1],j,cnt; // nested for loops iterator variables and found solutions count
for (j=0;j<n;j++) x[j]='0'; x[j]='b'; j++; x[j]=0; // x = 0
for (j=0;j<n1;j++){ i[j]=j; x[i[j]]='1'; } // first solution
for (cnt=0;;)
{
// Form1->mm_log->Lines->Add(x); // here x is the valid answer to print
cnt++;
for (j=n1-1;j>=0;j--) // this emulates n1 nested for loops
{
x[i[j]]='0'; i[j]++;
if (i[j]<n-n1+j+1){ x[i[j]]='1'; break; }
}
if (j<0) break;
for (j++;j<n1;j++){ i[j]=i[j-1]+1; x[i[j]]='1'; }
}
return cnt; // found valid answers
};
When I use this with n1=6,n=32 I got this output (without printing the numbers):
cnt = 906192
and it was finished in 4.246 ms on AMD A8-5500 3.2GHz (win7 x64 32bit app no threads) which is fast enough for me...
Beware once you start outputing the numbers somewhere the speed will drop drastically. Especially if you output to console or what ever ... it might be better to buffer the output somehow like outputting 1024 string numbers at once etc... But as I mentioned before I am no Python coder so it might be already handled by the environment...
On top of all this once you will play with variable n1,n you can do the same for zeros instead of ones and use faster approach (if there is less zeros then ones use nested for loops to mark zeros instead of ones)
If the wanted solution numbers are wanted as a number (not a string) then its possible to rewrite this so the i[] or i0,..i5 holds the bitmask instead of bit positions ... instead of inc/dec you just shift left/right ... and no need for x array anymore as the number would be x = i0|...|i5 ...
You could create a counter array for positions of 1s in the number and assemble it by shifting the bits in their respective positions. I created an example below. It runs pretty fast (less than a second for 32 bits on my laptop):
bitCount = 32
oneCount = 6
maxBit = 1<<(bitCount-1)
ones = [1<<b for b in reversed(range(oneCount)) ] # start with bits on low end
ones[0] >>= 1 # shift back 1st one because it will be incremented at start of loop
index = 0
result = []
while index < len(ones):
ones[index] <<= 1 # shift one at current position
if index == 0:
number = sum(ones) # build output number
result.append(number)
if ones[index] == maxBit:
index += 1 # go to next position when bit reaches max
elif index > 0:
index -= 1 # return to previous position
ones[index] = ones[index+1] # and prepare it to move up (relative to next)
64 bits takes about a minute, roughly proportional to the number of values that are output. O(n)
The same approach can be expressed more concisely in a recursive generator function which will allow more efficient use of the bit patterns:
def genOneBits(bitcount=32,onecount=6):
for bitPos in range(onecount-1,bitcount):
value = 1<<bitPos
if onecount == 1: yield value; continue
for otherBits in genOneBits(bitPos,onecount-1):
yield value + otherBits
result = [ n for n in genOneBits(32,6) ]
This is not faster when you get all the numbers but it allows partial access to the list without going through all values.
If you need direct access to the Nth bit pattern (e.g. to get a random one-bits pattern), you can use the following function. It works like indexing a list but without having to generate the list of patterns.
def numOneBits(bitcount=32,onecount=6):
def factorial(X): return 1 if X < 2 else X * factorial(X-1)
return factorial(bitcount)//factorial(onecount)//factorial(bitcount-onecount)
def nthOneBits(N,bitcount=32,onecount=6):
if onecount == 1: return 1<<N
bitPos = 0
while bitPos<=bitcount-onecount:
group = numOneBits(bitcount-bitPos-1,onecount-1)
if N < group: break
N -= group
bitPos += 1
if bitPos>bitcount-onecount: return None
result = 1<<bitPos
result |= nthOneBits(N,bitcount-bitPos-1,onecount-1)<<(bitPos+1)
return result
# bit pattern at position 1000:
nthOneBit(1000) # --> 10485799 (00000000101000000000000000100111)
This allows you to get the bit patterns on very large integers that would be impossible to generate completely:
nthOneBits(10000, bitcount=256, onecount=9)
# 77371252457588066994880639
# 100000000000000000000000000000000001000000000000000000000000000000000000000000001111111
It is worth noting that the pattern order does not follow the numerical order of the corresponding numbers
Although nthOneBits() can produce any pattern instantly, it is much slower than the other functions when mass producing patterns. If you need to manipulate them sequentially, you should go for the generator function instead of looping on nthOneBits().
Also, it should be fairly easy to tweak the generator to have it start at a specific pattern so you could get the best of both approaches.
Finally, it may be useful to obtain then next bit pattern given a known pattern. This is what the following function does:
def nextOneBits(N=0,bitcount=32,onecount=6):
if N == 0: return (1<<onecount)-1
bitPositions = []
for pos in range(bitcount):
bit = N%2
N //= 2
if bit==1: bitPositions.insert(0,pos)
index = 0
result = None
while index < onecount:
bitPositions[index] += 1
if bitPositions[index] == bitcount:
index += 1
continue
if index == 0:
result = sum( 1<<bp for bp in bitPositions )
break
if index > 0:
index -= 1
bitPositions[index] = bitPositions[index+1]
return result
nthOneBits(12) #--> 131103 00000000000000100000000000011111
nextOneBits(131103) #--> 262175 00000000000001000000000000011111 5.7ns
nthOneBits(13) #--> 262175 00000000000001000000000000011111 49.2ns
Like nthOneBits(), this one does not need any setup time. It could be used in combination with nthOneBits() to get subsequent patterns after getting an initial one at a given position. nextOneBits() is much faster than nthOneBits(i+1) but is still slower than the generator function.
For very large integers, using nthOneBits() and nextOneBits() may be the only practical options.
You are dealing with permutations of multisets. There are many ways to achieve this and as #BPL points out, doing this efficiently is non-trivial. There are many great methods mentioned here: permutations with unique values. The cleanest (not sure if it's the most efficient), is to use the multiset_permutations from the sympy module.
import time
from sympy.utilities.iterables import multiset_permutations
t = time.process_time()
## Credit to #BPL for the general setup
multiPerms = ["".join(v) for v in multiset_permutations(["1"]*6+["0"]*26)]
elapsed_time = time.process_time() - t
print(elapsed_time)
On my machine, the above computes in just over 8 seconds. It generates just under a million results as well:
len(multiPerms)
906192

Possible optimizations for Project Euler #4 algorithm

Find the largest palindrome made from the product of two 3-digit numbers.
Even though the algorithm is fast enough for the problem at hand, I'd like to know if I missed any obvious optimizations.
from __future__ import division
from math import sqrt
def createPalindrome(m):
m = str(m) + str(m)[::-1]
return int(m)
def problem4():
for x in xrange(999,99,-1):
a = createPalindrome(x)
for i in xrange(999,int(sqrt(a)),-1):
j = a/i
if (j < 1000) and (j % 1 == 0):
c = int(i * j)
return c
It seems the biggest slowdown in my code is converting an integer to a string, adding its reverse and converting the result back to an integer.
I looked up more information on palindromes and stumbled upon this formula, which allows me to convert a 3-digit number "n" into a 6-digit palindrome "p" (can be adapted for other digits but I'm not concerned about that).
p = 1100*n−990*⌊n/10⌋−99*⌊n/100⌋
My original code runs in about 0.75 ms and the new one takes practically the same amount of time (not to mention the formula would have to be adapted depending on the number of digits "n" has), so I guess there weren't many optimizations left to perform.
Look here for Ideas
In C++ I do it like this:
int euler004()
{
// A palindromic number reads the same both ways. The largest palindrome
// made from the product of two 2-digit numbers is 9009 = 91 99.
// Find the largest palindrome made from the product of two 3-digit numbers.
const int N=3;
const int N2=N<<1;
int min,max,a,b,c,i,j,s[N2],aa=0,bb=0,cc=0;
for (min=1,a=1;a<N;a++) min*=10; max=(min*10)-1;
i=-1;
for (a=max;a>=min;a--)
for (b=a;b>=min;b--)
{
c=a*b; if (c<cc) continue;
for (j=c,i=0;i<N2;i++) { s[i]=j%10; j/=10; }
for (i=0,j=N2-1;i<j;i++,j--)
if (s[i]!=s[j]) { i=-1; break; }
if (i>=0) { aa=a; bb=b; cc=c; }
}
return cc; // cc is the output
}
no need for sqrt ...
the subcall to createPalindrome can slow things down due to heap/stack trashing
string manipulation m = str(m) + str(m)[::-1] is slow
string to int conversion can be faster if you do it your self on fixed size array
mine implementation runs around 1.7ms but big portion of that time is the App output and formating (AMD 3.2GHz 32bit app on W7 x64)...
[edit1] implementing your formula
int euler004()
{
int i,c,cc,c0,a,b;
for (cc=0,i=999,c0=1100*i;i>=100;i--,c0-=1100)
{
c=c0-(990*int(i/10))-(99*int(i/100));
for(a=999;a>=300;a--)
if (c%a==0)
{
b=c/a;
if ((b>=100)&&(b<1000)) { cc=c; i=0; break; }
}
}
return cc;
}
this takes ~0.4 ms
[edit2] further optimizations
//---------------------------------------------------------------------------
int euler004()
{
// A palindromic number reads the same both ways. The largest palindrome
// made from the product of two 2-digit numbers is 9009 = 91 99.
// Find the largest palindrome made from the product of two 3-digit numbers.
int i0,i1,i2,c0,c1,c,cc=0,a,b,da;
for (c0= 900009,i0=9;i0>=1;i0--,c0-=100001) // first digit must be non zero so <1,9>
for (c1=c0+90090,i1=9;i1>=0;i1--,c1-= 10010) // all the rest <0,9>
for (c =c1+ 9900,i2=9;i2>=0;i2--,c -= 1100) // c is palindrome from 999999 to 100001
for(a=999;a>=948;a-- )
if (c%a==0)
{
// biggest palindrome is starting with 9
// so smallest valid result is 900009
// it is odd and sqrt(900009)=948 so test in range <948,999>
b=c/a;
if ((b>=100)&&(b<1000)) { cc=c; i0=0; i1=0; i2=0; break; }
}
return cc;
}
//---------------------------------------------------------------------------
this is too fast for me to properly measure the time (raw time is around 0.037 ms)
removed the divisions and multiplications from palindrome generation
changed the ranges after some numeric analysis and thinking while waiting for bus
the first loop can be eliminated (result starts with 9)
I wrote this a while back when I just started learning python, but here it is:
for i in range (999, 800, -1):
for j in range (999,800, -1):
number = i*j
str_number = str(number)
rev_str_number = str_number[::-1]
if str_number == rev_str_number:
print("%s a palendrome") % number
I did not check all the numbers you did, but I still got the correct answer. What I really learned in this exercise is the "::" and how it works. You can check that out here.
Good luck with Euler!

Find num of overlapping and non-overlapping substrings in a string

PS: This is not a duplicate of How to find the overlap between 2 sequences, and return it
[Although I ask for solutions in above approach if it could be applied to the following problem]
Q: Although I got it right, it is still not a scalable solution and is definitely not optimized (low on score). Read the following description of the problem and kindly offer better solution.
Question:
For simplicity, we require prefixes and suffixes to be non-empty and shorter than the whole string S. A border of a string S is any string that is both a prefix and a suffix. For example, "cut" is a border of a string "cutletcut", and a string "barbararhubarb" has two borders: "b" and "barb".
class Solution { public int solution(String S); }
that, given a string S consisting of N characters, returns the length of its longest border that has at least three non-overlapping occurrences in the given string. If there is no such border in S, the function should return 0.
For example,
if S = "barbararhubarb" the function should return 1, as explained above;
if S = "ababab" the function should return 2, as "ab" and "abab" are both borders of S, but only "ab" has three non-overlapping occurrences;
if S = "baaab" the function should return 0, as its only border "b" occurs only twice.
Assume that:
N is an integer within the range [0..1,000,000];
string S consists only of lower-case letters (a−z).
Complexity:
expected worst-case time complexity is O(N);
expected worst-case space complexity is O(N) (not counting the storage required for input arguments).
def solution(S):
S = S.lower()
presuf = []
f = l = str()
rank = []
wordlen = len(S)
for i, j in enumerate(S):
y = -i-1
f += S[i]
l = S[y] + l
if f==l and f != S:
#print f,l
new=S[i+1:-i-1]
mindex = new.find(f)
if mindex != -1:
mid = f #new[mindex]
#print mid
else:
mid = None
presuf.append((f,mid,l,(i,y)))
#print presuf
for i,j,k,o in presuf:
if o[0]<wordlen+o[-1]: #non overlapping
if i==j:
rank.append(len(i))
else:
rank.append(0)
if len(rank)==0:
return 0
else:
return max(rank)
My solutions time complexity is: O(N2) or O(N4)
Help greatly appreciated.
My solution is combination between Rabin-Karp and Knuth–Morris–Pratt algorithms.
http://codility.com/cert/view/certB6J4FV-W89WX4ZABTDRVAG6/details
I have a (Java) solution that performs O(N) or O(N**3), for a resulting 90/100 overall, but I can't figure out how to make it go though 2 different testcases:
almost_all_same_letters
aaaaa...aa??aaaa??....aaaaaaa 2.150 s. TIMEOUT ERROR
running time: >2.15 sec., time limit: 1.20 sec.
same_letters_on_both_ends 2.120 s. TIMEOUT ERROR
running time: >2.12 sec., time limit: 1.24 sec.
Edit: Nailed it!
Now I have a solution that perform in O(N) and passes all the checks for a 100/100 result :)
I didn't know Codility, but it's a nice tool!
I have a solution with suffix arrays (there actually is algorithm for constructing SA and LCP in linear time or something bit worse than that, but surely not quadratic).
Still not sure if I can go without RMQs ( O(log n) with SegmentTree) which I couldn't make pass my own cases and seems quite complicated, but with RMQs it can (not mentioning approach with for loop instead of RMQ, that would make it quadratic anyway).
Solution is performing quite fast and passing my 21 test cases with various perks I've managed to craft, but still failing on some of their cases. Not sure if that helped you or gave you idea how to approach the problem, but I am sure that naive solution, like #Vicenco said in some of his comments, can't get you better than Silver.
EDIT:
managed to fix it all problems, but still to slow. I had to enforce some conditions but had to increase complexity with this, still not sure how to optimize that. Will keep you posted. Good luck!
protected int calcBorder(String input) {
if (null != input) {
int mean = (input.length() / 3);
while (mean >= 1) {
if (input.substring(0, mean).equals(
input.substring(input.length() - mean))) {
String reference = input.substring(0, mean);
String temp = input
.substring(mean, (input.length() - mean));
int startIndex = 0;
int endIndex = mean;
int count = 2;
while (endIndex <= temp.length()) {
if (reference.equals(temp.substring(startIndex,
endIndex))) {
count++;
if (count >= 3) {
return reference.length();
}
}
startIndex++;
endIndex++;
}
}
mean--;
}
}
return 0;
}
The Z-Algorithm would be a good solution.

Categories