Generate an output csv file outside the exe file made using Pyinstaller - python

Problem
I'm using PyInstaller on Windows to make an .exe file of my project, basically my projects generates a csv file as output and the name of the csv file is dependent on the current time so the program generates unique file each time it is ran
I couldn't find any resource online that could help me with this problem
PyInstaller Command that I used: (data.csv file added is supposed to be bundled with exe so no issue there)
pyinstaller src\main.py -F --name "Attendance_System" --add-data "src\data.csv;data" --add-data "C:\Users\Darshit Shah\OneDrive\Desktop\TCET\Att_Sys\att_sys\Lib\site-packages\customtkinter;customtkinter" --clean
code block where the file is generated:
except KeyboardInterrupt:
timer.cancel()
endTime = str(dt.datetime.now().time())
op_file = f"{app.currdate}_{app.startTime[0:-7]}_{endTime[0:-7]}.csv".replace(":","-")
app.getList().to_csv(f"{op_file}")
print("O/P File generated")
sys.exit()
Basically the code generates the file in the folder where my main.py is located but after bundling it with PyInstaller i cant seem to achieve that
Project Structure
my_proj
|
|--build
|
|--dist <--- "This is Where i want my output file to generate"
| `--my_proj.exe
|
|--proj_venv
| |--Include
| |--Lib
| |--Scripts
| `--pyvenv.cfg
|
`--src <--- "Folder where my output file would normally generate without .exe"
|--classes.py
|--interface.py
|--main.py
`--data.csv

Explanation of Problem
I echo #mrblue6's statement, but through past coding, believe that the line
app.getList().to_csv(f"{op_file}")
is the problem here. This would appear to generate the file in (most probably) the %TEMP%\_MEIXXXX folder (under the local AppData folder). This is because a compiled program uses a subdirectory of %TEMP% as its working directory (on Windows at least)
EDIT:
After posting, I remembered that as long as the output csv is in the same folder as your exe, you could do something like the following:
op_path = os.path.join(os.path.dirname(sys.executable), op_file)
As sys.executable holds the full path to the exe when compiled. This seems like a more robust solution than what I previously suggested. This would make:
import os
import sys # if you haven't already
op_file = f"{app.currdate}_{app.startTime[0:-7]}_{endTime[0:-7]}.csv".replace(":","-")
op_path = os.path.join(os.path.dirname(sys.executable), op_file)
app.getList().to_csv(f"{op_path}")
print("O/P File generated")
OLD:
I would try to change the name output to have an absolute path instead of just a file name, something like:
app.getList().to_csv(f"C:\users\dcs_2002\path\to\my_proj\dist\{op_file}")
for basic usage. If you want this is work elsewhere, it depends on the actual location of your my_proj folder, but I would do something like the following (assuming my_proj is in your home directory:
op_path = os.path.expanduser(f"~\\path\\to\\my_proj\\dist\\{op_file}")
# OR
op_path = os.path.join(os.path.expanduser("~"), "path", "to", "my_proj", "dist", op_file)
Both of these would work as os.path.expanduser() expands any leading ~'s to the path of your home directory. You could also use os.path.expandvars() to expand more complex percent-enclosed variables (ie `os.path.expandvars("%LOCALAPPDATA%\rest\of\path"). Obviously modify the paths to suit your needs, just make sure to replace any backslashes with a double backslash (to escape Python). All together this would be:
import os # if you haven't already
op_file = f"{app.currdate}_{app.startTime[0:-7]}_{endTime[0:-7]}.csv".replace(":","-")
op_path = os.path.expanduser(f"~\\path\\to\\my_proj\\dist\\{op_file}")
app.getList().to_csv(f"{op_path}")
print("O/P File generated")

Related

PYTHON AND BATCH SCRIPT: Run file if it exists and create if it doesn't

Full Disclaimer: I DO NOT KNOW PYTHON.
Hi Guys,
I have made an AutoHotKey Script for my volume keys. I would like to create a batch file which runs a python file (so if I change computers, I can easily create this scripts) which would do the following
Check if volume_keys.ahk exists in the D Drive
If it exists, run that;
If it doesn't exist, then create a file named volume_keys.ahk and add my script to it.
My script is:
^!NumpadMult::Send {Volume_Mute}
^!NumpadAdd::Send {Volume_Up}
^!NumpadSub::Send {Volume_Down}
I know how to code the .bat file and just need help for the python point-of-view, but I request the community to check it:
#ECHO OFF
ECHO This script will run an AHK Script. If you want to stop this process from happening, then cross this window off.If you want to continye:
pause
cd d:
D:\run_volume_keys_ahk_script.py
I really appreciate any help by the community.
Thanks in advance
You can use the os library for this. Here's what the python program would look like.
import os
if os.path.isfile('D:\\volume_keys.ahk'): # check if it exists
os.system('D:\\volume_keys.ahk') # execute it
else:
with open('D:\\volume_keys.ahk', 'w') as f: # open it in w (write) mode
f.write('^!NumpadMult::Send {Volume_Mute} \
^!NumpadAdd::Send {Volume_Up} \
^!NumpadSub::Send {Volume_Down}') # Write to file
os.system('D:\\volume_keys.ahk') # execute
To activate the ahk script, you might want to use the subprocess module, of which I took the example from here
import subprocess
subprocess.call(["path/to/ahk.exe", "script.ahk"])
Note that you'll have to find the ahk executable on a computer before you can use the script, maybe you want to automatically check that too.
You can set the path you want to check for scripts in one string, and then add the filenames of your scripts as strings to a list. You can use listdir() from the os module to see any files and directories at a given path, then iterate over your scriptnames and check if it exists in that list of files. If it does, run it.
In this example I copy-pasted your script into a string as value for the key 'scriptname' in a dictionary, so that python can actually create the script file. This isn't really a neat way to do it though, you might want to have your scripts prepared in a directory next to your python script and copy them from there. See an example of how here
from os import listdir
from os.path import isfile, join
CHECK_PATH = "D:"
AHK_EXECUTABLE_PATH = "path/to/ahk.exe"
SCRIPTS_TO_CHECK = {'script1.ahk':"""^!NumpadMult::Send {Volume_Mute}
^!NumpadAdd::Send {Volume_Up}
^!NumpadSub::Send {Volume_Down} """, 'script2.ahk':" some other script here"}
files_to_check = set(listdir(CHECK_PATH)) # using a set for fast lookup later
for scriptname, script in SCRIPTS_TO_CHECK.items():
if not scriptname in files_to_check:
print(f"script {scriptname} not found, creating it.")
with open(scriptname, 'w') as file:
file.write(script)
# else
subprocess.call(AHK_EXECUTABLE_PATH, scriptname)

Why can't I find files that have py extension?

The below codes can find files that have a particular file extension. But same codes can't find files that have py extension. What is the reason for that?
search.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import os
def search(dir_or_file):
if dir_or_file.startswith("*."):
return [
os.path.join(i, m)
for i, j, k in os.walk("/")
for m in k
if m.endswith("." + dir_or_file.split(".")[1])
]
else:
return [
os.path.join(i, dir_or_file)
for i, j, k in os.walk("/")
if dir_or_file in k or dir_or_file in j
]
if __name__ == "__main__":
import sys
if len(sys.argv) == 2:
for i in search(sys.argv[1]):
print(i)
When I run the search.py script to search for files that have *.txt extension, I am getting the below output:
tanberk#kutlu:~$ ./search.py *.txt
/media/tanberk/Data/Belgeler/İş Başvurusu/a.txt
/media/tanberk/Data/Projects/astrology/a.txt
/home/tanberk/a.txt
/home/tanberk/Projects/astrology/a.txt
tanberk#kutlu:~$
Note: By the way there are many files that have *.txt extension. The program can not find all of them.
When I run the search.py script to search for files that have *.py extension, I am getting the below output:
tanberk#kutlu:~$ ./search.py *.py
tanberk#kutlu:~$
Your shell is expanding the pattern before your script sees it. You find all the a.txt files because *.txt expands to a.txt from your home directory (and you apparently have no other .txt files in your home directory), so the real command you ran was ./search.py a.txt, and all you see are files named a.txt.
At a guess, the reason you see nothing for *.py is because you have multiple .py files in your home directory, and your script explicitly does nothing if len(sys.argv) is anything other than 2. When the shell expands *.py, you end up with a too long sys.argv (because it actually ran something like ./search.py foo.py search.py ...) and do nothing.
To run your script as you expect (with the script interpreting the wildcards), quote your inputs to prevent shell expansion, e.g.:
$ ./search.py '*.txt'
I also suggest you change your argument processing to produce a usage message when it receives the wrong number of arguments; usage errors should never pass silently.

Python for Data Analysis, Chapter 2, first example

I'm following along with the examples in a translated version of Wes McKinney's "Python for Data Analysis" and I was blocked in first example of Chapter 2
I think my problem arose because I saved a data file in a wrong path. is that right?
I stored a file, usagov_bitly_data2012-03-16-1331923249.txt, in C:\Users\HRR
and also stored folder, pydata-book-mater, that can be downloaded from http://github.com/pydata-book in C:\Users\HRR\Anaconda2\Library\bin.
Depends.
You might change the location you save your File or eddit the path you give to your code in Line 10. Since you're yousing relativ Paths i guess your script runs in C:\Users\HRR\Anaconda2\Library\bin, which means you have to go back to C:\Users\HRR or use an absolute Path ... or move the File, but hell you don't want to move a file every time you want to open it, like moving word files into msoffice file to open it, so try to change the Path.
And allways try harder ;)
In python open() will open from the current directory down unless given a full path (in linux that starts with / and windows <C>://). In your case the command is open the folder ch02 in the directory the script is running from and then open usagov_bitly_data2012-03-16-1331923249.txt in that folder.
Since you are storing the text file in C:\Users\HRR\usagov_bitly_data2012-03-16-1331923249.txt and you did not specify the directory of the script. I recommend the following command instead open(C:\\Users\\HRR\\usagov_bitly_data2012-03-16-1331923249.txt)
Note: the double \ is to escape the characters and avoid tabs and newlines showing up in the path.

Including sound files in a pygame script using pyinstaller

I am new to programming, so I made myself the challenge to create Pong, and so I did. Now I want to share it with a couple of friends, so I decided to try using pyinstaller (have tried cx_Freeze).
In this Pong game I have 3 sound effects, located in the folder "sfx". So I've looked into including files using pyinstaller, so my .spec file says:
added_files = [
('E:\Game Development Stuff\Python 3\Games\Pong\sfx\hitOutline.ogg', 'sfx'),
('E:\Game Development Stuff\Python 3\Games\Pong\sfx\hitPaddle.ogg', 'sfx'),
('E:\Game Development Stuff\Python 3\Games\Pong\sfx/score.ogg', 'sfx')
]
a = Analysis(['pong.py'],
pathex=['E:\\Game Development Stuff\\Python 3\\Games\\Pong'],
binaries=None,
datas=added_files,
and in the Pong program itself, I use this code to get the path:
def resource_path(relative):
if hasattr(sys, "_MEIPASS"):
return os.path.join(sys._MEIPASS, relative)
return os.path.join(relative)
fileDir = os.path.dirname(os.path.realpath('__file__'))
hitPaddle = resource_path(os.path.join(fileDir, "sfx", "hitPaddle.ogg"))
hitOutline = resource_path(os.path.join(fileDir, "sfx", "hitOutline.ogg"))
score = resource_path(os.path.join(fileDir, "sfx", "score.ogg"))
hitPaddleSound=pygame.mixer.Sound(hitPaddle)
hitOutlineSound=pygame.mixer.Sound(hitOutline)
scoreSound=pygame.mixer.Sound(score)
So I make the exe file using pyinstaller (with the command pyinstaller pong.spec)
but when I open the pong.exe file the command window says:
Traceback "<string>", Unable to open file 'E:\\Game Development Stuff\\Python 3\\Games\\Pong\\dist\\pong\\sfx\\hitPaddle.ogg'
but in that exact same path is hitPaddle.ogg.
It seems to me that pygame isn't able to found it for some weird reason?
I believe the issue is in this line. You are not refrencing the files correctly. You wrote:
hitPaddle = resource_path(os.path.join(fileDir, "sfx", "hitPaddle.ogg"))
Instead you should of just:
hitpaddle = resource_path("sfx\hitPaddle.ogg")
This is because when you added the files in the spec file, you stated that they should be in "root\sfx" folder. When the .exe is run in onefile mode, all files are actually located in a temp folder called MEIXXXX, with XXXX being some integers. When you run the .exe, if you open this folder you should be able to see your files there.
Solved it for me after struggling with the same issue for hours. Conclusions:
The problem is not in not being able to find it, then it would say something with 'could not find'. It is really an issue with opening the file. Somehow, the .ogg format is giving issues. I changed all my .ogg files to .wav files and my game is running without problems now as executable.
I have no idea why though, because two weeks ago for a previous version I did manage to make a working .exe with the exact same .ogg files. And I don't see how the changes I made should have any effect on this. Anyway, it works now, and perhaps this can also solve this problem for others.

py2exe - generate single executable file

I thought I heard that py2exe was able to do this, but I never figured it out. Has anyone successfully done this? Can I see your setup.py file, and what command line options you used?
Basically I'm thinking of it giving me a single executable file that does something like unzips itself to maybe /temp and runs.
The way to do this using py2exe is to use the bundle_files option in your setup.py file. For a single file you will want to set bundle_files to 1, compressed to True, and set the zipfile option to None. That way it creates one compressed file for easy distribution.
Here is a more complete description of the bundle_file option quoted directly from the py2exe site*
Using "bundle_files" and "zipfile"
An easier (and better) way to create
single-file executables is to set
bundle_files to 1 or 2, and to set
zipfile to None. This approach does
not require extracting files to a
temporary location, which provides
much faster program startup.
Valid values for bundle_files are:
3 (default) don't bundle
2 bundle everything but the Python interpreter
1 bundle everything, including the Python interpreter
If zipfile is set to None, the files will be bundle
within the executable instead of library.zip.
Here is a sample setup.py:
from distutils.core import setup
import py2exe, sys, os
sys.argv.append('py2exe')
setup(
options = {'py2exe': {'bundle_files': 1, 'compressed': True}},
windows = [{'script': "single.py"}],
zipfile = None,
)
PyInstaller will create a single .exe file with no dependencies; use the --onefile option. It does this by packing all the needed shared libs into the executable, and unpacking them before it runs, just as you describe (EDIT: py2exe also has this feature, see minty's answer)
I use the version of PyInstaller from svn, since the latest release (1.3) is somewhat outdated. It's been working really well for an app which depends on PyQt, PyQwt, numpy, scipy and a few more.
As the other poster mention, py2exe, will generate an executable + some libraries to load. You can also have some data to add to your program.
Next step is to use an installer, to package all this into one easy-to-use installable/unistallable program.
I have used InnoSetup with delight for several years and for commercial programs, so I heartily recommend it.
I've been able to create a single exe file with all resources embeded into the exe.
I'm building on windows. so that will explain some of the os.system calls i'm using.
First I tried converting all my images into bitmats and then all my data files into text strings.
but this caused the final exe to be very very large.
After googleing for a week i figured out how to alter py2exe script to meet my needs.
here is the patch link on sourceforge i submitted, please post comments so we can get it included in
the next distribution.
http://sourceforge.net/tracker/index.php?func=detail&aid=3334760&group_id=15583&atid=315583
this explanes all the changes made, i've simply added a new option to the setup line.
here is my setup.py.
i'll try to comment it as best I can.
Please know that my setup.py is complex do to the fact that i'm access the images by filename.
so I must store a list to keep track of them.
this is from a want-to-b screen saver I was trying to make.
I use exec to generate my setup at run time, its easyer to cut and paste like that.
exec "setup(console=[{'script': 'launcher.py', 'icon_resources': [(0, 'ICON.ico')],\
'file_resources': [%s], 'other_resources': [(u'INDEX', 1, resource_string[:-1])]}],\
options={'py2exe': py2exe_options},\
zipfile = None )" % (bitmap_string[:-1])
breakdown
script = py script i want to turn to an exe
icon_resources = the icon for the exe
file_resources = files I want to embed into the exe
other_resources = a string to embed into the exe, in this case a file list.
options = py2exe options for creating everything into one exe file
bitmap_strings = a list of files to include
Please note that file_resources is not a valid option untill you edit your py2exe.py file as described in the link above.
first time i've tried to post code on this site, if I get it wrong don't flame me.
from distutils.core import setup
import py2exe ##UnusedImport
import os
#delete the old build drive
os.system("rmdir /s /q dist")
#setup my option for single file output
py2exe_options = dict( ascii=True, # Exclude encodings
excludes=['_ssl', # Exclude _ssl
'pyreadline', 'difflib', 'doctest', 'locale',
'optparse', 'pickle', 'calendar', 'pbd', 'unittest', 'inspect'], # Exclude standard library
dll_excludes=['msvcr71.dll', 'w9xpopen.exe',
'API-MS-Win-Core-LocalRegistry-L1-1-0.dll',
'API-MS-Win-Core-ProcessThreads-L1-1-0.dll',
'API-MS-Win-Security-Base-L1-1-0.dll',
'KERNELBASE.dll',
'POWRPROF.dll',
],
#compressed=None, # Compress library.zip
bundle_files = 1,
optimize = 2
)
#storage for the images
bitmap_string = ''
resource_string = ''
index = 0
print "compile image list"
for image_name in os.listdir('images/'):
if image_name.endswith('.jpg'):
bitmap_string += "( " + str(index+1) + "," + "'" + 'images/' + image_name + "'),"
resource_string += image_name + " "
index += 1
print "Starting build\n"
exec "setup(console=[{'script': 'launcher.py', 'icon_resources': [(0, 'ICON.ico')],\
'file_resources': [%s], 'other_resources': [(u'INDEX', 1, resource_string[:-1])]}],\
options={'py2exe': py2exe_options},\
zipfile = None )" % (bitmap_string[:-1])
print "Removing Trash"
os.system("rmdir /s /q build")
os.system("del /q *.pyc")
print "Build Complete"
ok, thats it for the setup.py
now the magic needed access the images.
I developed this app without py2exe in mind then added it later.
so you'll see access for both situations. if the image folder can't be found
it tries to pull the images from the exe resources. the code will explain it.
this is part of my sprite class and it uses a directx. but you can use any api you want or just access the raw data.
doesn't matter.
def init(self):
frame = self.env.frame
use_resource_builtin = True
if os.path.isdir(SPRITES_FOLDER):
use_resource_builtin = False
else:
image_list = LoadResource(0, u'INDEX', 1).split(' ')
for (model, file) in SPRITES.items():
texture = POINTER(IDirect3DTexture9)()
if use_resource_builtin:
data = LoadResource(0, win32con.RT_RCDATA, image_list.index(file)+1) #windll.kernel32.FindResourceW(hmod,typersc,idrsc)
d3dxdll.D3DXCreateTextureFromFileInMemory(frame.device, #Pointer to an IDirect3DDevice9 interface
data, #Pointer to the file in memory
len(data), #Size of the file in memory
byref(texture)) #ppTexture
else:
d3dxdll.D3DXCreateTextureFromFileA(frame.device, ##UndefinedVariable
SPRITES_FOLDER + file,
byref(texture))
self.model_sprites[model] = texture
#else:
# raise Exception("'sprites' folder is not present!")
Any questions fell free to ask.
You should create an installer, as mentioned before. Even though it is also possible to let py2exe bundle everything into a single executable, by setting bundle_files option to 1 and the zipfile keyword argument to None, I don't recommend this for PyGTK applications.
That's because of GTK+ tries to load its data files (locals, themes, etc.) from the directory it was loaded from. So you have to make sure that the directory of your executable contains also the libraries used by GTK+ and the directories lib, share and etc from your installation of GTK+. Otherwise you will get problems running your application on a machine where GTK+ is not installed system-wide.
For more details read my guide to py2exe for PyGTK applications. It also explains how to bundle everything, but GTK+.
I'm told bbfreeze will create a single file .EXE, and is newer than py2exe.
I recently used py2exe to create an executable for post-review for sending reviews to ReviewBoard.
This was the setup.py I used
from distutils.core import setup
import py2exe
setup(console=['post-review'])
It created a directory containing the exe file and the libraries needed. I don't think it is possible to use py2exe to get just a single .exe file. If you need that you will need to first use py2exe and then use some form of installer to make the final executable.
One thing to take care of is that any egg files you use in your application need to be unzipped, otherwise py2exe can't include them. This is covered in the py2exe docs.
try
c_x freeze
it can create a good standalone

Categories