We Keep Coding

Convert file to base64 string on Python 3

I need to convert image (or any file) to base64 string. I use different ways, but result is always byte, not string. Example:
import base64
file = open('test.png', 'rb')
file_content = file.read()
base64_one = base64.encodestring(file_content)
base64_two = base64.b64encode(file_content)
print(type(base64_one))
print(type(base64_two))
Returned
<class 'bytes'>
<class 'bytes'>
How do I get a string, not byte? Python 3.4.2.

Base64 is an ascii encoding so you can just decode with ascii
>>> import base64
>>> example = b'\x01'*10
>>> example
b'\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01'
>>> result = base64.b64encode(example).decode('ascii')
>>> print(repr(result))
'AQEBAQEBAQEBAQ=='

I need to write base64 text in file ...
So then stop worrying about strings and just do that instead.
with open('output.b64', 'wb'):
write(base64_one)

The following code worked for me:
import base64
file_text = open(file, 'rb')
file_read = file_text.read()
file_encode = base64.encodebytes(file_read)
I initially tried base64.encodestring() but that function has been deprecated as per this issue.

Reading from file into Python data structure

I have a file which has the following format:
[
["unique_id1", {"cc":15, "dd":30}], ["unique_id2", {"cc": 184,"dd":10}], ...
]
I want to directly read the file and put data in a Python data structure. For now, I'm processing using regular expressions. Is there a command that I'm missing to read it directly?

This file format is probably JSON from what you've shown us.
You can parse it by doing
import json
out = json.load(file_object)
Either that or its a literal
out = eval(file_object.read())
OR (Preferred)
import ast
out = ast.literal_eval(file_object.read())

You can use literal_eval
import ast
f = open('myfile.txt')
print ast.literal_eval(f.read())

You should use ast.literal_eval(), it turns strings that contain Python objects, into Python objects:
>>> import ast
>>> l = ast.literal_eval('[1, 2, 3]')
>>> type(l)
<class 'list'>
>>> l
[1, 2, 3]
So you would read the data from your file and turn it into a list:
with open('file.txt') as infile:
data = ast.literal_eval(infile.read())

How to write a number as text while writing in csv file in python

import csv
a = ['679L', 'Z60', '033U', '0003']
z = csv.writer(open("test1.csv", "wb"))
z.writerow(a)
Consider the code above
Output:
676L Z60 33U 3
I need to get it in the text format itself as
676L Z60 033U 0003
How to do that.

The Python csv module does not treat strings as numbers when writing the file:
>>> import csv
>>> from StringIO import StringIO
>>> a = ['679L', 'Z60', '033U', '0003']
>>> out = StringIO()
>>> z = csv.writer(out)
>>> z.writerow(a)
>>> out.getvalue()
'679L,Z60,033U,0003\r\n'
If you are seeing 3 in some other tool when reading you need to fix that tool; Python is not at fault here.
You can instruct the csv.writer() to put quotes around anything that is not a number; this could make it clearer to whatever reads your CSV that the column is not numeric. Set quoting to csv.QUOTE_NONNUMERIC:
>>> out = StringIO()
>>> z = csv.writer(out, quoting=csv.QUOTE_NONNUMERIC)
>>> z.writerow(a)
>>> out.getvalue()
'"679L","Z60","033U","0003"\r\n'
but this won't prevent Excel from treating the column as numeric anyway.
If you are loading this into Excel then don't use the Open feature. Instead create a new empty worksheet and use the Import feature instead. This will let you designate a column as Text rather than General.

writing data into file with binary packed format in python

I am reading some value for file and wants to write modified value into file. My file is .ktx format [binary packed format].
I am using struct.pack() but seems that something is going wrong with that:
bytes = file.read(4)
bytesAsInt = struct.unpack("l",bytes)
number=1+(bytesAsInt[0])
number=hex(number)
no=struct.pack("1",number)
outfile.write(no)
I want to write in both ways little-endian and big-endian.

no_little =struct.pack(">1",bytesAsInt)
no_big =struct.pack("<1",bytesAsInt) # i think this is default ...
again you can check the docs and see the format characters you need
https://docs.python.org/3/library/struct.html
>>> struct.unpack("l","\x05\x04\x03\03")
(50529285,)
>>> struct.pack("l",50529285)
'\x05\x04\x03\x03'
>>> struct.pack("<l",50529285)
'\x05\x04\x03\x03'
>>> struct.pack(">l",50529285)
'\x03\x03\x04\x05'
also note that it is a lowercase L , not a one (as also covered in the docs)

I haven't tested this but the following function should solve your problem. At the moment it reads the file contents completely, creates a buffer and then writes out the updated contents. You could also modify the file buffer directly using unpack_from and pack_into but it might be slower (again, not tested). I'm using the struct.Struct class since you seem to want to unpack the same number many times.
import os
import struct
from StringIO import StringIO
def modify_values(in_file, out_file, increment=1, num_code="i", endian="<"):
with open(in_file, "rb") as file_h:
content = file_h.read()
num = struct.Struct(endian + num_code)
buf = StringIO()
try:
while len(content) >= num.size:
value = num.unpack(content[:num.size])[0]
value += increment
buf.write(num.pack(value))
content = content[num.size:]
except Exception as err:
# handle
else:
buf.seek(0)
with open(out_file, "wb") as file_h:
file_h.write(buf.read())
An alternative is to use the array which makes it quite easy. I don't know how to implement endianess with an array.
def modify_values(filename, increment=1, num_code="i"):
with open(filename, "rb") as file_h:
arr = array("i", file_h.read())
for i in range(len(arr)):
arr[i] += increment
with open(filename, "wb") as file_h:
arr.tofile(file_h)

Storing a random byte string in Python

For my project, I need to be able to store random byte strings in a file and read the byte string again later. For example, I want to store randomByteString from the following code:
>>> from os import urandom
>>> randomByteString=urandom(8)
>>> randomByteString
b'zOZ\x84\xfb\xceM~'
What would be the proper way to do this?
Edit: Forgot to mention that I also want to store 'normal' string alongside the byte strings.

Code like:
>>> fh = open("e:\\test","wb")
>>> fh.write(randomByteString)
8
>>> fh.close()
Operate the file as binary mode. Also, you could do it in a better manner if the file operations are near one place (Thanks to #Blender):
>>> with open("e:\\test","wb") as fh:
fh.write(randomByteString)
Update: if you want to strong normal strings, you could encode it and then write it like:
>>> "test".encode()
b'test'
>>> fh.write("test".encode())
Here the fh means the same file handle opened previously.

Works just fine. You can't expect the output to make much sense though.
>>> import os
>>> with open("foo.txt", "wb") as fh:
... fh.write(os.urandom(8))
...
>>> fh.close()
>>> with open("foo.txt", "r") as fh:
... for line in fh.read():
... print line
...
^J^JM-/
^O
R
M-9
J
~G