We Keep Coding

Why does .items() work for the multiplication between sequence and `float`

I got two dicts self.theta & self.beta whose values are all fp and a fp value eta, when I tried to modify it as below:
self.theta[0]=self.theta[0]-eta*self.beta[0]
It returned TypeError: can't multiply sequence by non-int of type 'float'.
If I put it into a for loop of the dict, it worked, but I cannot separately manipulate with a single key such as [0] above, have to finish the modification in the for loop in this case.
for key, value in self.theta.items():
self.theta[key]=self.theta[key]-eta*self.beta[key]
I wonder is it possible to directly modify the self.theta[0] without getting the error? The numpy array seems like a workaround but I need to change the code a lot so I am thinking about solely torch methods.
p.s. I know the list cannot multiply with fp, but how come the for loop iterates the .items() works? Full scripts below:
beta = defaultdict(list)
theta = defaultdict(list)
layer = [500,300,100,3]
eta = 0.1
i = 0
for l in layer[0:-1]:
# print(l)
theta[l] = (torch.randn(l, layer[i+1]))/math.sqrt(l)
beta[l] = torch.Tensor(l, layer[i+1])
i += 1
theta[0] = theta[0] - eta * beta[0]
# TypeError: can't multiply sequence by non-int of type 'float'
for key, value in theta.items():
theta[key] = theta[key] - eta * beta[key]
print('theta:', theta[key])
# This is okay

How to increment floating point numbers by 1?

I am trying to create a python code which can help to increment the version values below by 1,expected output shows the result?I am thinking of splitting the version saving each digit ,increment the last digit by 1 and reconstruct the version,is there a way to this simply in python?
version1 = 1151.1
version2 = 4.1.1
version3 = 275.1.2.99
version4 = 379
next_version1 = version1 + 1
print next_version1
next_version2 = version2 + 1
print next_version2
next_version3 = version3 + 1
print next_version3
next_version4 = version4 + 1
print next_version4
EXPECTED OUTPUT:-
1151.2
4.1.2
275.1.2.100
380

Actually not all the numbers are floats in this case.
You should treat it as strings and update the last element.
version1 = '275.1.2.3'
version2 = '279'
version3 = '275.2.3.10'
def updateVersion(version):
if '.' in version:
version = version.split('.')
version[-1] = str(int(version[-1]) + 1)
version = '.'.join(version)
else:
version = str(int(version)+1)
return version
updateVersion(version1)
updateVersion(version2)
Output:
275.1.2.4
280
275.2.3.11

First and foremost please read about Floating Point Arithmetic: Issues and Limitations
Maybe that was the reason why you ask, not clear.
However, I suggest to save each part as an integer, e.g.
main_version_number = 1151
minor_version_number = 1
sub_version_number = 0
You could maybe have a data structure with those fields (a Version class maybe?) with appropriate methods.
Do not rely on floating point arithmetic.

First off, the code you outline would most certainly give a syntax error.
A number of the form 2 is an integer; 2.2, a floating point; but a 2.2.2, meaningless.
You are looking for tuples here. For instance,
>>> version3 = (275,1,2,3)
Then you would get
>>> version3
(275, 1, 2, 3)
To dirty-update only the last bit of such a tuple, you could do
>>> version3 = version3[:-1] + (version3[-1] + 1,)
>>> version3
(275, 1, 2, 4)
The reason I call this dirty updating is that it will not take care of carrying over into the next significant bit.
Here's a relatively simple script to do just that that I could put together in a couple of minutes. Assuming you have stored your version number as a tuple object called version, attempt the following:
new_version = version
for bit in range(len(version)):
new_version = new_version[:-1-bit] + ((new_version[-1-bit] + 1) % 10,) + new_version[-bit:]
if -2-bit >=0:
new_version = new_version[-2-bit:] + (new_version[-2-bit] + (version[-2-bit] + 1) // 10,) + new_version[-1-bit:]
elif (version[-2-bit] + 1) // 10:
new_version = (1,) + new_version
Alternatively, take a look at bumpversion, a tool that lets you take care of version-numbering within your project, with git integration.

The variables 'version2' and 'version3' will result in a syntax error. This syntax error is caused by the fact that Python does not know of any (number) type that has several points in its value. In essence you are trying to use certain types in a way that they are not meant to be used. More specifically the floating point number is not suitable for your goals. As the name suggests a floating point number, only contains one point and that point can be placed anywhere between its digits (floating).
My advice would be to create your own type/class. This would enable you to store the version number in a way that allows for easy modification of its values and better separation of concerns in your code (i.e. that each part of your code is only concerned with one thing).
Example
class VersionNumber:
"""Represents a version number that can contain parts (major, minor, etc)."""
def __init__(self, *argv):
"""This is the constructor, i.e. a function that is called when you create a new VersionNumber.
The '*argv' allows the user of this class to give a variable amount of arguments. This is why
you can have a version number with only 1 number, and one with 4. The '*argv' is iterable."""
#Create a class specific field, that stores all the version number parts in a list.
self.Parts = []
#Fill it with the supplied arguments.
for part in argv:
self.Parts.append(part)
def __repr__(self):
"""This function is called when the VersionNumber needs to be displayed in the console"""
return str(self)
def __str__(self):
"""This function is called when the VersionNumber is parsed to a string"""
return '.'.join(map(str,self.Parts))
def incrementVersion(self, position, incrementAmount):
"""This function allows you to increment the version number. It does this by adjusting the list
we have set in the constructor."""
self.Parts[position] += incrementAmount
version1 = VersionNumber(1, 23, 45, 0)
print(version1)
#Position -1, takes the last (most right) version number part from the version number.
version1.incrementVersion(-1, 1)
print(version1)
version2 = VersionNumber(346)
print(version2)
version2.incrementVersion(-1, 2)
print(version2)

Unexpected Python Arithmetic Behavior

I'm working on a huffman encoder/decoder in Python, and am experiencing some unexpected (at least for me) behavior in my code. Encoding the file is fine, the problem occurs when decoding the file. Below is the associated code:
def decode(cfile):
with open(cfile,"rb") as f:
enc = f.read()
len_dkey = int(bin(ord(enc[0]))[2:].zfill(8) + bin(ord(enc[1]))[2:].zfill(8),2) # length of dictionary
pad = ord(enc[2]) # number of padding zeros at end of message
dkey = { int(k): v for k,v in json.loads(enc[3:len_dkey+3]).items() } # dictionary
enc = enc[len_dkey+3:] # actual message in bytes
com = []
for b in enc:
com.extend([ bit=="1" for bit in bin(ord(b))[2:].zfill(8)]) # actual encoded message in bits (True/False)
cnode = 0 # current node for tree traversal
dec = "" # decoded message
for b in com:
cnode = 2 * cnode + b + 1 # array implementation of tree
if cnode in dkey:
dec += dkey[cnode]
cnode = 0
with codecs.open("uncompressed_"+cfile,"w","ISO-8859-1") as f:
f.write(dec)
The first with open(cfile,"rb") as f call runs very quickly for all file sizes (tested sizes are 1.2MB, 679KB, and 87KB), but the part that slows down the code significantly is the for b in com loop. I've done some timing and I honestly don't know what's going on.
I've timed the whole decode function on each file, as shown below:
87KB 1.5 sec
679KB 6.0 sec
1.2MB 384.7 sec
first of all, I don't even know how to assign this complexity. Next, I've timed a single run through of the problematic loop, and got that the line cnode = 2*cnode + b + 1 takes 2e-6 seconds while the if cnode in dkey line takes 0.0 seconds (according to time.clock() on OSX). So it seems as if the arithmetic is slowing down my program significantly...? Which I feel like doesn't make sense.
I actually have no idea what is going on, and any help at all would be super welcome

I found a solution to my problem, but I am still left with confusion afterwards. I solved the problem by changing the dec from "" to [], and then changing the dec += dkey[cnode] line to dec.append(dkey[cnode]). This resulted in the following times:
87KB 0.11 sec
679KB 0.21 sec
1.2MB 1.01 sec
As you can see, this has immensely cut down the time, so in that aspect, this was a success. However, I am still confused as to why python's string concatenation seems to be the problem here.

Binary To String Conversion

I'm relatively new to python and I'm trying to design a program that takes an input of Binary Data and Converts it to a String of Text. I have the following so far but keep getting the following error: TypeError: unsupported operand type(s) for &: 'str' and 'int' Can anyone see where I'm going wrong? And, if so, advise how to fix it?
a = int(input("please enter the binary you want to convert: "))
for str in a:
g = [~(chr(str)&~240)|(e&~240)]
f = 86
e = 231
d = bin(chr(str))
b = (str)
j=(b)
print(j)

There is quite a lot wrong with what you're doing; I'm not sure how you get the error you claim to have, given the other errors in the posted code. In order of reading the function:
Don't call your own variables str, it prevents you from accessing the built-in of the same name. Also, that variable either isn't a str, or causes a TypeError on chr(str).
You can't iterate over an integer for x in y:; this is also a TypeError.
(The error you report) chr(str) returns a string of length one. This is not a valid operand type for &, hence TypeError.
Another operand, e, has not yet been defined, so that will be a NameError.
Irrespective of that, you never use g again anyway.
Or f - what's that even for?
Now e is defined!
bin(chr(str)) will never work - again, chr returns a string, and bin takes a number as an argument.
b = (str) works, but the parentheses are redundant.
Same with j = (b), which is also not indented far enough to be in the loop.
Neither is print(j).
It is not clear what you are trying to achieve, exactly. If you gave example inputs (what format is the "Binary Data"?) and outputs (and what "String of Text" do you want to get?) along with your actual code and the full error traceback this might be easier.
Edit
With the information provided in your comment, it appears that you are trying to reverse these operations:
a = input("please enter the text you want to hide: ")
for ch in a:
## b = (ch) # this still doesn't do anything!
c = ord(ch)
## d = bin(c) # and d is never used
## e = 231 # e is only used to calculate g
f = 86
## g = (((c&240) >> 4)|(e&240)) # which is also never used
h = (((c&15)|(f&240)))
j = bin(h)
print(j)
This produces, for example (a == 'foo'):
0b1010110
0b1011111
0b1011111
However, to convert the input '0b1010110' to an integer, you need to supply int with the appropriate base:
>>> int('0b1010110', 2)
86
and you can't iterate over the integer. I think you want something like:
data = input("please enter the binary you want to convert: ")
for j in data.split():
h = int(j, 2)
...
where the input would be e.g. '0b1010110 0b1011111 0b1011111', or just do one at a time:
h = int(input(...), 2)
Note that, as the function is reversed, you will have to define f before trying to go back through the bitwise operation.

Efficient string to hex function

I'm using an old version of python on an embedded platform ( Python 1.5.2+ on Telit platform ). The problem that I have is my function for converting a string to hex. It is very slow. Here is the function:
def StringToHexString(s):
strHex=''
for c in s:
strHex = strHex + hexLoookup[ord(c)]
return strHex
hexLookup is a lookup table (a python list) containing all the hex representation of each character.
I am willing to try everything (a more compact function, some language tricks I don't know about). To be more clear here are the benchmarks (resolution is 1 second on that platform):
N is the number of input characters to be converted to hex and the time is in seconds.
N | Time (seconds)
50 | 1
150 | 3
300 | 4
500 | 8
1000 | 15
1500 | 23
2000 | 31
Yes, I know, it is very slow... but if I could gain something like 1 or 2 seconds it would be a progress.
So any solution is welcomed, especially from people who know about python performance.
Thanks,
Iulian
PS1: (after testing the suggestions offered - keeping the ord call):
def StringToHexString(s):
hexList=[]
hexListAppend=hexList.append
for c in s:
hexListAppend(hexLoookup[ord(c)])
return ''.join(hexList)
With this function I obtained the following times: 1/2/3/5/12/19/27 (which is clearly better)
PS2 (can't explain but it's blazingly fast) A BIG thank you Sven Marnach for the idea !!!:
def StringToHexString(s):
return ''.join( map(lambda param:hexLoookup[param], map(ord,s) ) )
Times:1/1/2/3/6/10/12
Any other ideas/explanations are welcome!

Make your hexLoookup a dictionary indexed by the characters themselves, so you don't have to call ord each time.
Also, don't concatenate to build strings – that used to be slow. Use join on a list instead.
from string import join
def StringToHexString(s):
strHex = []
for c in s:
strHex.append(hexLoookup[c])
return join(strHex, '')

Building on Petr Viktorin's answer, you could further improve the performance by avoiding global vairable and attribute look-ups in favour of local variable look-ups. Local variables are optimized to avoid a dictionary look-up on each access. (They haven't always been, by I just double-checked this optimization was already in place in 1.5.2, released in 1999.)
from string import join
def StringToHexString(s):
strHex = []
strHexappend = strHex.append
_hexLookup = hexLoookup
for c in s:
strHexappend(_hexLoookup[c])
return join(strHex, '')

Constantly reassigning and adding strings together using the + operator is very slow. I guess that Python 1.5.2 isn't yet optimizing for this. So using string.join() would be preferable.
Try
import string
def StringToHexString(s):
listhex = []
for c in s:
listhex.append(hexLookup[ord(c)])
return string.join(listhex, '')
and see if that is any faster.

Try:
from string import join
def StringToHexString(s):
charlist = []
for c in s:
charlist.append(hexLoookup[ord(c)])
return join(charlist, '')
Each string addition takes time proportional to the length of the string so, while join will also take time proportional to the length of the entire string, but you only have to do it once.
You could also make hexLookup a dict mapping characters to hex values, so you don't have to call ord for every character. It's a micro-optimization, so probably won't be significant.

def StringToHexString(s):
return ''.join( map(lambda param:hexLoookup[param], map(ord,s) ) )
Seems like this is the fastest! Thank you Sven Marnach!