We Keep Coding

Removing Single Quotes from a String Stored in an Array

I wrote code to append a json response into a list for some API work I am doing, but it stores the single quotes around the alphanumerical value I desire. I would like to get rid of the single quotes. Here is what I have so far:
i = 0
deviceID = []
while i < deviceCount:
deviceID.append(devicesRanOn['resources'][i])
deviceID[i] = re.sub('[\W_]', '', deviceID[i])
i += 1
if i >= deviceCount:
break
if (deviceCount == 1):
print ('Device ID: ', deviceID)
elif (deviceCount > 1):
print ('Device IDs: ', deviceID)
the desired input should look like this:
input Device IDs:
['14*************************00b29', '58*************************c3df4']
Output:
['14*************************00b29', '58*************************c3df4']
Desired Output:
[14*************************00b29, 58*************************c3df4]
As you can see, I am trying to use RegEx to filter non Alphanumeric and replace those with nothing. It is not giving me an error nor is it preforming the actions I am looking for. Does anyone have a recommendation on how to fix this?
Thank you,
xOm3ga

You won't be able to use the default print. You'll need to use your own means of making a representation for the list. But this is easy with string formatting.
'[' + ', '.join(f'{id!s}' for id in ids) + ']'
The f'{id:!s} is an f-string which formats the variable id using it's __str__ method. If you're on a version pre-3.6 which doesn't use f-strings, you can also use
'%s' % id
'{!s}'.format(id)
PS:
You can simplify you're code significantly by using a list comprehension and custom formatting instead of regexes.
ids = [device for device in devicesRanOn['resources'][:deviceCount]]
if deviceCount == 1:
label = 'Device ID:'
elif deviceCount > 1:
label = 'Device IDs:'
print(label, '[' + ', '.join(f'{id!s}' for id in ids) + ']')

Read Null terminated string in python

I'm trying to read a null terminated string but i'm having issues when unpacking a char and putting it together with a string.
This is the code:
def readString(f):
str = ''
while True:
char = readChar(f)
str = str.join(char)
if (hex(ord(char))) == '0x0':
break
return str
def readChar(f):
char = unpack('c',f.read(1))[0]
return char
Now this is giving me this error:
TypeError: sequence item 0: expected str instance, int found
I'm also trying the following:
char = unpack('c',f.read(1)).decode("ascii")
But it throws me:
AttributeError: 'tuple' object has no attribute 'decode'
I don't even know how to read the chars and add it to the string, Is there any proper way to do this?

Here's a version that (ab)uses __iter__'s lesser-known "sentinel" argument:
with open('file.txt', 'rb') as f:
val = ''.join(iter(lambda: f.read(1).decode('ascii'), '\x00'))

How about:
myString = myNullTerminatedString.split("\x00")[0]
For example:
myNullTerminatedString = "hello world\x00\x00\x00\x00\x00\x00"
myString = myNullTerminatedString.split("\x00")[0]
print(myString) # "hello world"
This works by splitting the string on the null character. Since the string should terminate at the first null character, we simply grab the first item in the list after splitting. split will return a list of one item if the delimiter doesn't exist, so it still works even if there's no null terminator at all.
It also will work with byte strings:
myByteString = b'hello world\x00'
myStr = myByteString.split(b'\x00')[0].decode('ascii') # "hello world" as normal string
If you're reading from a file, you can do a relatively larger read - estimate how much you'll need to read to find your null string. This is a lot faster than reading byte-by-byte. For example:
resultingStr = ''
while True:
buf = f.read(512)
resultingStr += buf
if len(buf)==0: break
if (b"\x00" in resultingStr):
extraBytes = resultingStr.index(b"\x00")
resultingStr = resultingStr.split(b"\x00")[0]
break
# now "resultingStr" contains the string
f.seek(0 - extraBytes,1) # seek backwards by the number of bytes, now the pointer will be on the null byte in the file
# or f.seek(1 - extraBytes,1) to skip the null byte in the file

(edit version 2, added extra way at the end)
Maybe there are some libraries out there that can help you with this, but as I don't know about them lets attack the problem at hand with what we know.
In python 2 bytes and string are basically the same thing, that change in python 3 where string is what in py2 is unicode and bytes is its own separate type, which mean that you don't need to define a read char if you are in py2 as no extra work is required, so I don't think you need that unpack function for this particular case, with that in mind lets define the new readString
def readString(myfile):
chars = []
while True:
c = myfile.read(1)
if c == chr(0):
return "".join(chars)
chars.append(c)
just like with your code I read a character one at the time but I instead save them in a list, the reason is that string are immutable so doing str+=char result in unnecessary copies; and when I find the null character return the join string. And chr is the inverse of ord, it will give you the character given its ascii value. This will exclude the null character, if its needed just move the appending...
Now lets test it with your sample file
for instance lets try to read "Sword_Wea_Dummy" from it
with open("sword.blendscn","rb") as archi:
#lets simulate that some prior processing was made by
#moving the pointer of the file
archi.seek(6)
string=readString(archi)
print "string repr:", repr(string)
print "string:", string
print ""
#and the rest of the file is there waiting to be processed
print "rest of the file: ", repr(archi.read())
and this is the output
string repr: 'Sword_Wea_Dummy'
string: Sword_Wea_Dummy
rest of the file: '\xcd\xcc\xcc=p=\x8a4:\xa66\xbfJ\x15\xc6=\x00\x00\x00\x00\xeaQ8?\x9e\x8d\x874$-i\xb3\x00\x00\x00\x00\x9b\xc6\xaa2K\x15\xc6=;\xa66?\x00\x00\x00\x00\xb8\x88\xbf#\x0e\xf3\xb1#ITuB\x00\x00\x80?\xcd\xcc\xcc=\x00\x00\x00\x00\xcd\xccL>'
other tests
>>> with open("sword.blendscn","rb") as archi:
print readString(archi)
print readString(archi)
print readString(archi)
sword
Sword_Wea_Dummy
ÍÌÌ=p=Š4:¦6¿JÆ=
>>> with open("sword.blendscn","rb") as archi:
print repr(readString(archi))
print repr(readString(archi))
print repr(readString(archi))
'sword'
'Sword_Wea_Dummy'
'\xcd\xcc\xcc=p=\x8a4:\xa66\xbfJ\x15\xc6='
>>>
Now that I think about it, you mention that the data portion is of fixed size, if that is true for all files and the structure on all of them is as follow
[unknow size data][know size data]
then that is a pattern we can exploit, we only need to know the size of the file and we can get both part smoothly as follow
import os
def getDataPair(filename,knowSize):
size = os.path.getsize(filename)
with open(filename, "rb") as archi:
unknown = archi.read(size-knowSize)
know = archi.read()
return unknown, know
and by knowing the size of the data portion, its use is simple (which I get by playing with the prior example)
>>> strins_data, data = getDataPair("sword.blendscn", 80)
>>> string_data, data = getDataPair("sword.blendscn", 80)
>>> string_data
'sword\x00Sword_Wea_Dummy\x00'
>>> data
'\xcd\xcc\xcc=p=\x8a4:\xa66\xbfJ\x15\xc6=\x00\x00\x00\x00\xeaQ8?\x9e\x8d\x874$-i\xb3\x00\x00\x00\x00\x9b\xc6\xaa2K\x15\xc6=;\xa66?\x00\x00\x00\x00\xb8\x88\xbf#\x0e\xf3\xb1#ITuB\x00\x00\x80?\xcd\xcc\xcc=\x00\x00\x00\x00\xcd\xccL>'
>>> string_data.split(chr(0))
['sword', 'Sword_Wea_Dummy', '']
>>>
Now to get each string a simple split will suffice and you can pass the rest of the file contained in data to the appropriated function to be processed

Doing file I/O one character at a time is horribly slow.
Instead use readline0, now on pypi: https://pypi.org/project/readline0/ . Or something like it.
In 3.x, there's a "newline" argument to open, but it doesn't appear to be as flexible as readline0.

Here is my implementation:
import struct
def read_null_str(f):
r_str = ""
while 1:
back_offset = f.tell()
try:
r_char = struct.unpack("c", f.read(1))[0].decode("utf8")
except:
f.seek(back_offset)
temp_char = struct.unpack("<H", f.read(2))[0]
r_char = chr(temp_char)
if ord(r_char) == 0:
return r_str
else:
r_str += r_char

Find the differences between nested dictionaries which contain lists

I want to extract the differences between two nested dictionaries and I want the result to include the full dictionary keypath. I have installed Python2.7 and DeepDiff, which appears to be the best option for what I am trying to achieve. I am trying to determine how to change the output of DeepDiff so it provides the full dictionary path and values rather than a set which I cannot index. Is there a better way to alter the output (rather than converting the output back to a dictionary)?
Code:
from __future__ import print_function
from deepdiff import DeepDiff
knownAPs = {'WLC1': {'10.1.1.1': {'72.6': ['AP22', 'city'], '55.1': ['AP102', 'office']}}, 'WLC2': {'10.1.1.2': {}}}
discoveredAPs = {'WLC1': {'10.1.1.1': {}}, 'WLC2': {'10.1.1.2': {}}}
ddiff = DeepDiff(knownAPs, discoveredAPs)
if 'dic_item_added' in ddiff.keys():
print('Item added to known: ' + str((ddiff['dic_item_added'])))
if 'dic_item_removed' in ddiff.keys():
DisAssociatedAPs = (list(list(ddiff['dic_item_removed'])))
for i in DisAssociatedAPs:
fullkeypath = (str(i).strip('root'))
ControllerName = (fullkeypath[0])
ControllerIP = (fullkeypath[1])
AccessPointIndex = (fullkeypath[2])
print('AP: ' + str(knownAPs + fullkeypath) + ' on controller: ' + str(ControllerName) + ' was removed from the known database')
if 'values_changed' in ddiff.keys():
print('Item changed: ' + str((ddiff['values_changed'])))
Output
Traceback (most recent call last):
File "C:/xxx/testdic4.py", line 15, in <module>
print('AP: ' + str(knownAPs + fullkeypath) + ' on controller: ' + str(ControllerName) + ' was removed from the known database')
TypeError: unsupported operand type(s) for +: 'dict' and 'str'
Process finished with exit code 1
Preferred Output
AP: ['AP22', 'city'] on controller: ['WLC1'] was removed from the known database
AP: ['AP102', 'office'] on controller: ['WLC1'] was removed from the known database

The issue is exactly what the traceback tells you: you are trying to add a dictionary to a string, which is of course not what you want. Specifically, when you add knownAPs (type dict) to fullkeypath (type str) you get an error, because dict doesn't know how to add itself to a str.
But that doesn't answer your more general question of how to output the diffs in a way you want. Try this:
diffs = deepdiff.DeepDiff(knownAPs, discoveredAPs)
def get_value_from_string(d, s):
s = list(filter(None, (piece[2:-1] for piece in s.split(']'))))
for piece in s:
d = d[piece]
return d
if 'dic_item_removed' in diffs:
for item in diffs['dic_item_removed']:
item = item.strip('root')
base = item[2:item.find(']') - 1]
print('AP:', get_value_from_string(knownAPs, item),
'on controller: \'' + base + '\' was removed from the known '
'database')

Single quotes around list elements that should be floats

I am asked to "return a list of tuples containing the subset name (as a string) and a list of floating point data values".
My code is:
def load_data(filename):
fileopen = open(filename)
result_open=[]
for line in fileopen:
answer = (line.strip().split(","))
result_open.append((answer[0],(answer[1:])))
return result_open
However, when I run the code, the following appears:
[('Slow Loris', [' 21.72', ' 29.3', ' 20.08', ' 29.98', ' 29.85', ' 26.22', ' 19......)]
Is there anyway to change the tuple to appear without the apostrophes? I want it to look like:
[('Slow Loris', [21.72, 29.3, 20.08, 29.98, 29.85, 6.22, 19......)]

line is a string, and line.strip().split(",") is a list of strings. You need to convert the string values into float or Decimal values. One way would be:
result_open.append((answer[0], [float(val) for val in answer[1:]]))
That will raise an exception on values that can't be converted to a float, so you should think about how you want to handle such input.

Inserting unicode MYSQL results within Python

if results:
for line in results:
print line[0] + ' - ' + line[1]
I need to insert a '-' between line[0] and line[1], but when I enter the above code, I get the error message 'coercing to Unicode: need string or buffer, int found'. Any suggestions on a way around this?
Thanks in advance.

the problem is that line[0] or line[1] holding an int number.
by using "+" and 'some string' you are telling python please append to the string the next string, but you have an int so the interpreter is raising a problem.
try to use:
if results:
for line in results:
print str(line[0]) + ' - ' + str(line[1])
"+" note in python will change it's "action" when meeting different data types. it's a very powerful feature but as default it cannot add different datatypes.