We Keep Coding

zipfile module in python [duplicate]

I am creating an ZIP file with ZipFile in Python 2.5, it works OK so far:
import zipfile, os
locfile = "test.txt"
loczip = os.path.splitext (locfile)[0] + ".zip"
zip = zipfile.ZipFile (loczip, "w")
zip.write (locfile)
zip.close()
But I couldn't find how to encrypt the files in the ZIP file.
I could use system and call PKZIP -s, but I suppose there must be a more "Pythonic" way. I'm looking for an open source solution.

I created a simple library to create a password encrypted zip file in python. - here
import pyminizip
compression_level = 5 # 1-9
pyminizip.compress("src.txt", "dst.zip", "password", compression_level)
The library requires zlib.
I have checked that the file can be extracted in WINDOWS/MAC.

This thread is a little bit old, but for people looking for an answer to this question in 2020/2021.
Look at pyzipper
A 100% API compatible replacement for Python’s zipfile that can read and write AES encrypted zip files.
7-zip is also a good choice, but if you do not want to use subprocess, go with pyzipper...

The duplicate question: Code to create a password encrypted zip file? has an answer that recommends using 7z instead of zip. My experience bears this out.
Copy/pasting the answer by #jfs here too, for completeness:
To create encrypted zip archive (named 'myarchive.zip') using open-source 7-Zip utility:
rc = subprocess.call(['7z', 'a', '-mem=AES256', '-pP4$$W0rd', '-y', 'myarchive.zip'] +
['first_file.txt', 'second.file'])
To install 7-Zip, type:
$ sudo apt-get install p7zip-full
To unzip by hand (to demonstrate compatibility with zip utility), type:
$ unzip myarchive.zip
And enter P4$$W0rd at the prompt.
Or the same in Python 2.6+:
>>> zipfile.ZipFile('myarchive.zip').extractall(pwd='P4$$W0rd')

pyminizip works great in creating a password protected zip file. For unziping ,it fails at some situations. Tested on python 3.7.3
Here, i used pyminizip for encrypting the file.
import pyminizip
compression_level = 5 # 1-9
pyminizip.compress("src.txt",'src', "dst.zip", "password", compression_level)
For unzip, I used zip file module:
from zipfile import ZipFile
with ZipFile('/home/paulsteven/dst.zip') as zf:
zf.extractall(pwd=b'password')

You can use pyzipper for this task and it will work great when you want to encrypt a zip file or generate a protected zip file.
pip install pyzipper
import pyzipper
def encrypt_():
secret_password = b'your password'
with pyzipper.AESZipFile('new_test.zip',
'w',
compression=pyzipper.ZIP_LZMA,
encryption=pyzipper.WZ_AES) as zf:
zf.setpassword(secret_password)
zf.writestr('test.txt', "What ever you do, don't tell anyone!")
with pyzipper.AESZipFile('new_test.zip') as zf:
zf.setpassword(secret_password)
my_secrets = zf.read('test.txt')
The strength of the AES encryption can be configure to be 128, 192 or 256 bits. By default it is 256 bits. Use the setencryption() method to specify the encryption kwargs:
def encrypt_():
secret_password = b'your password'
with pyzipper.AESZipFile('new_test.zip',
'w',
compression=pyzipper.ZIP_LZMA) as zf:
zf.setpassword(secret_password)
zf.setencryption(pyzipper.WZ_AES, nbits=128)
zf.writestr('test.txt', "What ever you do, don't tell anyone!")
with pyzipper.AESZipFile('new_test.zip') as zf:
zf.setpassword(secret_password)
my_secrets = zf.read('test.txt')
Official Python ZipFile documentation is available here: https://docs.python.org/3/library/zipfile.html

#tripleee's answer helped me, see my test below.
This code works for me on python 3.5.2 on Windows 8.1 ( 7z path added to system).
rc = subprocess.call(['7z', 'a', output_filename + '.zip', '-mx9', '-pSecret^)'] + [src_folder + '/'])
With two parameters:
-mx9 means max compression
-pSecret^) means password is Secret^). ^ is escape for ) for Windows OS, but when you unzip, it will need type in the ^.
Without ^ Windows OS will not apply the password when 7z.exe creating the zip file.
Also, if you want to use -mhe switch, you'll need the file format to be in 7z instead of zip.
I hope that may help.

2022 answer:
I believe this is an utterly mundane task and therefore should be oneliner. I abstracted away all the frevolous details in a library that is as powerfull as a bash terminal.
from crocodile.toolbox import Path
file = Path(r'my_string_path')
result_file = file.zip(pwd="lol", use_7z=True)
when the 7z flag is raised, it gets called behind the scenes.
You don't need to learn 7z command line syntax.
You don't need to worry about installing 7z, does that automatically if it's not installed. (tested on windows so far)

You can use the Chilkat library. It's commercial, but has a free evaluation and seems pretty nice.
Here's an example I got from here:
import chilkat
# Demonstrates how to create a WinZip-compatible 128-bit AES strong encrypted zip
zip = chilkat.CkZip()
zip.UnlockComponent("anything for 30-day trial")
zip.NewZip("strongEncrypted.zip")
# Set the Encryption property = 4, which indicates WinZip compatible AES encryption.
zip.put_Encryption(4)
# The key length can be 128, 192, or 256.
zip.put_EncryptKeyLength(128)
zip.SetPassword("secret")
zip.AppendFiles("exampleData/*",True)
zip.WriteZip()

Generate TeX/LaTeX file and compile both in Python

I am preparing a data processing code and the results from these are to be fed to a TeX/LaTeX file and the file is to be compiled by Python (i.e. Python should send the command to compile the TeX/LaTeX file)
At present I plan to generate a text file (with .tex extension) with the necessary syntax for TeX/LaTeX and then call external system command using os.system. Is there any easier way or modules that can do this?

No such Python modules exist but you could try generating text files with the required syntax of the text file with .tex extension and compile it by running system commands with Python:
import os
os.system("pdflatex filename")

You could use PyLaTeX for this. This is exactly one of the things that it is meant to be used for. https://github.com/JelteF/PyLaTeX

Make sure, you have installed Perl and MikTeX (for latexmk and pdflatex compilers support) in your system.
If not, you can download Perl from https://www.perl.org/get.html
and MikTeX from https://miktex.org/download.
Also do not forget to check http://mg.readthedocs.io/latexmk.html#installation as it guides nicely about Latex compilers.
I have document.tex with following content.
\documentclass{article}%
\usepackage[T1]{fontenc}%
\usepackage[utf8]{inputenc}%
\usepackage{lmodern}%
\usepackage{textcomp}%
\usepackage{lastpage}%
%
%
%
\begin{document}%
\normalsize%
\section{A section}%
\label{sec:A section}%
Some regular text and some %
\textit{italic text. }%
\subsection{A subsection}%
\label{subsec:A subsection}%
Also some crazy characters: \$\&\#\{\}
%
\end{document}
Finally create any python file and paste any one of the below code and run.
1st way
# 1st way
import os
tex_file_name = "document.tex";
os.system("latexmk " + tex_file_name + " -pdf");
2nd way
# 2nd way
import os
tex_file_name = "document.tex";
os.system("pdflatex " + tex_file_name);
For compiling complex latex files, you need to look for extra options passed with latexmk or pdflatex commands.
Thanks.

Is there a version of ConfigParser that deals with files with no section headers?

I have a config file which is mainly used in shell scripts, and therefore has the following format:
# Database parameters (MySQL only for now)
DBHOST=localhost
DATABASE=stuff
DBUSER=mypkguser
DBPASS=zbxhsxhg
# Storage locations
STUFFDIR=/var/mypkg/stuff
GIZMODIR=/var/mypkg/gizmo
Now I need to read its values from a Python (2.6) script. I would like not to reinvent the wheel and parse it with descriptor.readlines() and looking for equal signs and skipping lines beginning with '#' and dealing with quoted values and blah blah blah boring. I tried using ConfigParser but it doesn't like files that don't have section headers. Do I have any options or will I have to do the boring thing?
Oh, by the way, wrapping a shell script around the Python script is not an option. It has to run within Apache.

I'm not aware of such a module, but as a quick and dirty hack - just add the [section] before the file-content and you can use ConfigParser as intended!
from io import StringIO
filename = 'ham.egg'
vfile = StringIO(u'[Pseudo-Sectio]\n%s' % open(filename).read())

Python - SCP vs SSH for multiple computers and multiple files

Python 2.4.x (cannot install any non-stock modules).
Question for you all. (assuming use of subprocess.popen)
Say you had 20 - 30 machines - each with 6 - 10 files on them that you needed to read into a variable.
Would you prefer to scp into each machine, once for each file (120 - 300 SCP commands total), reading each file after it's SCP'd down into a variable - then discarding the file.
Or - SSH into each machine, once for each file - reading the file into memory. (120 - 300 ssh commands total).
?
Unless there's some other way to grab all desired files in one shot per machine (files are named YYYYMMDD.HH.blah - range would be given 20111023.00 - 20111023.23). - reading them into memory that I cannot think of?

Depending on the size of the file, you can possibly do something like:
...
files= "file1 file2 ..."
myvar = ""
for tm in machine_list
myvar = myvar+ subprocess.check_output(["ssh", "user#" + tm, "/bin/cat " + files]);
...
file1 file2 etc are space delimited. Assuming all are unix boxes you can /bin/cat them all in one shot from each machine. (This is assuming that you are simply loading the ENTIRE content in one variable) variations of above.. SSH will be simpler to diagnose.
At least that's my thought.
UPDATE
use something like
myvar = myvar+Popen(["ssh", "user#" +tm ... ], stdout=PIPE).communicate()[0]
Hope this helps.

scp lets you:
Copy entire directories using the -r flag: scp -r g0:labgroup/ .
Specify a glob pattern: scp 'g0:labgroup/assignment*.hs' .
Specify multiple source files: scp 'g0:labgroup/assignment1*' 'g0:labgroup/assignment2*' .
Not sure what sort of globbing is supported, odds are it just uses the shell for this. I'm also not sure if it's smart enough to merge copies from the same server into one connection.

You could run a remote command via ssh that uses tar to tar the files you want together (allowing the result to go to standard out), capture the output into a Python variable, then use Python's tarfile module to split the files up again. I'm not actually sure how tarfile works; you may have to put the read output into a file-like StringIO object before accessing it with tarfile.
This will save you a bit of time, since you'll only have to connect to each machine once, reducing time spent in ssh negotiation. You also avoid having to use local disk storage, which could save a bit of time and/or energy — useful if you are running in laptop mode, or on a device with a limited file system.
If the network connection is relatively slow, you can speed things up further by using gzip or bzip compression; decompression is supported by tarfile.

As an extra to Inerdia's answer, yes you can get scp to transfer multiple files in one connection, by using brace patterns:
scp "host:{path/to/file1,path/to/file2}" local_destination"
And you can use the normal goodies of brace patterns if your files have common prefixes or suffixes:
scp "host:path/to/{file1,file2}.thing" local_destination"
Note that the patterns are inside quotes, so they're not expanded by the shell before calling scp. I have a host with a noticeable connection delay, on which I created two empty files. Then executing the copy like the above (with the brace pattern quoted) resulted in a delay and then both files quickly transferred. When I left out the quotes, so the local shell expanded the braces into two separate host:file arguments to scp, then there was a noticeable delay before the first file and between the two files.
This suggests to me that Inerdia's suggestion of specifying multiple host:file arguments will not reuse the connection to transfer all the files, but using quoted brace patterns will.

Checking folder/file ntfs permissions using python

As the question title might suggest, I would very much like to know of the way to check the ntfs permissions of the given file or folder (hint: those are the ones you see in the "security" tab). Basically, what I need is to take a path to a file or directory (on a local machine, or, preferrably, on a share on a remote machine) and get the list of users/groups and the corresponding permissions for this file/folder. Ultimately, the application is going to traverse a directory tree, reading permissions for each object and processing them accordingly.
Now, I can think of a number of ways to do that:
parse cacls.exe output -- easily done, BUT, unless im missing something, cacls.exe only gives the permissions in the form of R|W|C|F (read/write/change/full), which is insufficient (I need to get the permissions like "List folder contents", extended permissions too)
xcacls.exe or xcacls.vbs output -- yes, they give me all the permissions I need, but they work dreadfully slow, it takes xcacls.vbs about ONE SECOND to get permissions on a local system file. Such speed is unacceptable
win32security (it wraps around winapi, right?) -- I am sure it can be handled like this, but I'd rather not reinvent the wheel
Is there anything else I am missing here?

Unless you fancy rolling your own, win32security is the way to go. There's the beginnings of an example here:
http://timgolden.me.uk/python/win32_how_do_i/get-the-owner-of-a-file.html
If you want to live slightly dangerously (!) my in-progress winsys package is designed to do exactly what you're after. You can get an MSI of the dev version here:
http://timgolden.me.uk/python/downloads/WinSys-0.4.win32-py2.6.msi
or you can just checkout the svn trunk:
svn co http://winsys.googlecode.com/svn/trunk winsys
To do what you describe (guessing slightly at the exact requirements) you could do this:
import codecs
from winsys import fs
base = "c:/temp"
with codecs.open ("permissions.log", "wb", encoding="utf8") as log:
for f in fs.flat (base):
log.write ("\n" + f.filepath.relative_to (base) + "\n")
for ace in f.security ().dacl:
access_flags = fs.FILE_ACCESS.names_from_value (ace.access)
log.write (u" %s => %s\n" % (ace.trustee, ", ".join (access_flags)))
TJG