py2exe and Tableau Python API - python

First of all, please excuse me if I'm using some of the terminology incorrectly (accountant by trade ...)
I'm writing a piece of code that I was planning to pack as .exe product.
I've already included number of standard libraries (xlrd, csv, math, operator, os, shutil, time, datetime, and xlwings). Unfortunately, when I've added 'dataextract' library my program stopped working.
dataextract is an API written specifically for software called Tableau (one of the leading BI solutions on the market). Also Tableau website says it does not provide any maintenance support for it at the moment.
I've tested it on very basic setup:
from xlwings import Workbook, Sheet, Range
Workbook.set_mock_caller(r'X:\JAC Reporting\Tables\Pawel\Development\_DevXL\Test1.xlsx')
f = Workbook.caller()
s = raw_input('Type in anything: ')
Range(1, (2, 1)).value = s
This works perfectly fine. After adding:
import dataextract as tde
The Console (black box) will only flash on the screen and nothing happens.
Questions:
Does library (in this case 'dataextract') has to meet certain criteria to be compatible with py2exe?
As Tableau does not maintain the original code, does it mean I won't be able to pack it into .exe using py2exe?
Finally: I'm using 'dataextract' for almost 2 years now and as long as you will run the program through .py file it works like a charm :) I just decided to take it one step further.
Any comments/input would be greatly appreciated.
EDIT:
Not sure does it help or not, but when I tried to run the same script using cx_Freeze compiler got below error:

First of all massive thanks to #Andris as he pointed me at the correct direction.
It turned out dataextrac library dlls are not automatically copied while compiler is running. Therefore you need to copy them from 'site-package/dataextrac/bin' into 'dist' folder.
Also from 12 dlls you only need 9 of them (I tried running exe file for each of them). One you don't need are: icin44.dll, msvcp100.dll and msvcr100.dll.
To be on the safe side I will be coping them anyway though.
Hope this post will be any help to otheres :)

Related

Cannot find csv data file where progress is saved (Python/Pyinstaller base, possibly sys. MEIPASS or NSIS ?)

Qualifier quite new to Python.
I wrote some Python code importing pandas, selenium, sys, os, tkinter, and pillow, then put together with pyinstaller and NSIS.
The programme uses a csv file for its input and updates this based on user actions. The updates are to be saved internally so if the user quits they can continue where they left off.
It all saves properly, user progress saves and picks up correctly, and if I "download CSV" the file is up-to-date. It all works perfectly, functionally speaking.
However, when the programme is run, the csv that it starts with is not where it is saving the progress. It stays the same. The "progress" data is being saved somewhere else From a data security perspective, I need to know where it is saving. I could not find it after hours of looking.
Even if I uninstall the programme and reinstall, it still remembers the progress. Tested also on machines with no Python etc.
I am using:
if getattr(sys, 'frozen', False):
CurrentPath = sys._MEIPASS
else:
CurrentPath = os.path.dirname(__file__)
Could that be it?
The save line itself is rather standard:
df.to_csv('file_s.csv',encoding='utf-8', index=False)
My only other idea is it relates to NSIS installer and the uninstall script. There are painfully few tutorials for beginners on that. If the community thinks that is the issue, I think it best I post a new question with relevant info.
Apologies if this is too vague and happy to provide anymore needed info!
You are defining the CurrentPath variable using the standard "if frozen" approach, which looks OK. But, you don't then seem to use it when you save the file.
Try explicitly concatenating it with the filename to save
df.to_csv(os.path.join(CurrentPath,'file_s.csv'),encoding='utf-8', index=False)
Note that when you run as a executable, sys._MEIPASS will be a temporary folder created (on Windows atleast) somewhere like C:\Users\<you>\AppData\Local\Temp\MEIXXX

Creating Executable Zip Archives With Packages in the Project

I feel this question needs a better title and I will amend it if someone suggests something better. The problem is I'm not sure of the terminology of the feature that I'm using here.
The best way to describe my problem is to show what I've done. The project is here: https://github.com/jeffnyman/quendor
This project is setup so it can be executed as a module. For example, from the project root someone could do this:
python3 -m quendor
I also have a build script to generate an in-memory zip (if I'm using that terminology correctly):
https://github.com/jeffnyman/quendor/blob/master/build.py
That works in that if you run build.py it will generate a quendor.py file that executes the entire project. That worked fine up until I included other directories (like my utilities and zinterface).
With the project as it is in the repo right now, if you run the build (.\build.py) and then run the generated file:
./quendor.py
You get the following error:
File "./quendor.py/quendor/__main__.py", line 6, in <module>
ModuleNotFoundError: No module named 'quendor.zinterface'
So a key point: if all of my files are in the same directory (i.e., in quendor) this build script works fine in terms of producing an executable script file.
But once I include the subdirectories and files in those directories, things go south on me with the above error.
I'm sure all the files are being gathered. I handle that starting on line 18 (https://github.com/jeffnyman/quendor/blob/master/build.py#L18). And if you were to add to line 24 this statement:
print(f"* {file_path}")
You would see it outputs the following:
* quendor/__init__.py
* quendor/__version__.py
* quendor/zinterface/fileio.py
* quendor/utilities/messages.py
* quendor/__main__.py
So I'm suspecting it might have to do with the code where I write the string at line 28 (https://github.com/jeffnyman/quendor/blob/master/build.py#L28). I feel I have to do more to let the executable zipped script file know about the modules.
But I'm not sure if (1) I'm accurate and (2) even if I'm accurate, if that's possible. I'm finding I'm in a bit over my head here.
Any thoughts would be appreciated and I'm happy to update with any necessarily clarifications or terminology.
So it won't let me comment unless I have more reputation but I can post an answer. Even though I don't have an answer, but rather a comment. I think the above comment was not meant for your actual __main__.py file but rather the one that is getting generated in your quendor.py file. You might want to try adding the import statements to your packed string that you write.
For example, see what happens if on line 32 you add this: import quendor.zinterface.fileio as zio. (Don't replace the line that's there. Just put my line and then keep your others.) I'm not sure how the zip process works but if it tries to mirror the module process that should work. However, if it doesn't, that won't work. You might also just want to try doing import quendor.zinterface. By itself that won't work but it would be interesting to see if it gave you a different error.
Actually, it turns out I found a way to do this! It required using os.walk rather than os.listdir. This required taking a few ideas that people here discussed. Here is the script that does the trick:
https://github.com/jeffnyman/quendor/blob/master/build.py
You can compare that with my previous commit that was trying to handle this a different way.
Eldritch was right that I couldn't just flatten the directory nor could I just add imports to the string I was writing to the final zip file. Jean-François was correct that I had to focus on the __main__.py that was being generated. My contribution was figuring out os.walk() and then parameterizing the written string to handle the different directories.
Finally, this solution does require, as per HTF's suggestion, that I put an empty __init__.py file in each package.
With my solution in place, you can run build.py which then generates the quendor.py script. That script then executes correctly, in terms of recognizing the imports to various packages.
Playing around with just about every variation of import and file gathering that I can think of with your repo, there's a good news / bad news thing.
The bad news is that the answer is this: it isn't possible.
The good news is this: you do have a working implementation if you just keep all files in the quendor directory rather than having subdirectories.
The other good news is you stumbled on something, and posed a problem, that Python gurus aren't able to answer. And there's a certain pleasure to be found in that! I guarantee you will not get an answer to this that works (except for the "all files in one directory" solution).
A refinement to the answer is that if you're setting up the program to run as a module anyway, just use a pip configuration. That basically does the same thing that you want but without having to go through the contortions. (Unless there's a reason you were doing the build the way you were rather than using pip.)

Extremely new user to Python. "No module named request" error while trying code to detect image subdomains in a website to extract them to a folder

I may sound rather uninformed writing this, and unfortunately, my current issue may require a very articulate answer to fix. Therefore, I will try to be specific as possible as to ensure that my problem can be concisely understood.
My apologizes for that- as this Python code was merely obtained from a friend of mine who wrote it for me in order to complete a certain task. I myself had had extremely minimal programming knowledge.
Essentially, I am running Python 3.6 on a Mac. I am trying to work out a code that allows Python to scan through a bulk of a particular website's potentially existent subdomains in order to find possibly-existent JPG images files contained within said subdomains, and download any and all of the resulting found files to a distinct folder on my Desktop.
The Setup-
The code itself, named "download.py" on my computer, is written as follows:
import urllib.request
start = int(input("Start range:100000"))
stop = int(input("End range:199999"))
for i in range(start, stop + 1):
filename = str(i).rjust(6, '0') + ".jpg"
url = "http://website.com/Image_" + filename
urllib.request.urlretrieve(url, filename)
print(url)
(Note that the words "website" and "Image" have been substituted for the actual text included in my code).
Before I proceed, perhaps some explanation would be necessary.
Basically, the website in question contains several subdomains that include .JPG images, however, the majority of the exact URLs that allow the user to access these sub-domains are unknown and are a hidden component of the internal website itself. The format is "website.com/Image_xxxxxx.jpg", wherein x indicates a particular digit, and there are 6 total numerical digits by which only when combined to make a valid code pertain to each of the existent images on the site.
So as you can see, I have calibrated the code so that Python will initially search through number values in the aforementioned URL format from 100000 to 199999, and upon discovering any .JPG images attributed to any of the thousands of link combinations, will directly download all existent uncovered images to a specific folder that resides within my Desktop. The aim would be to start from that specific portion of number values, and upon running the code and fetching any images (or not), continually renumbering the code to work my way through all of the possible 6-digit combos until the operation is ultimately a success.
(Possible Side-Issue- Although I am fairly confident that my friend's code is written in a manner so that Python will only download .JPG files to my computer from images that actually do exist on that particular URL, rather than swarming my folder with blank/bare files from every single one of URL attempts regardless of whether that URL happens to be successful or not, I am admittedly not completely certain. If the latter is the case, informing me of a more suitable edit to my code would be tremendously appreciated.)
The Execution-
Right off the bat, the code experienced a large error. I'll list through the series of steps that led to the creation of said error.
#1- Of course, I first copy-pasted the code into a text document, and saved it as "download.py". I saved it inside of a folder named "Images" where I sought the images to be directly downloaded to. I used BBEdit.
#2- I proceeded, in Terminal, to input the commands "cd Desktop/Images" (to account for the file being held within the "Images" folder on my Desktop), followed by the command "Python download.py" (to actually run the code).
As you can see, the error which I obtained following my attempt to run the code was the ImportError: No module named request. Despite me guessing that the answer to solving this is simple, I can legitimately say I have got such minimal knowledge regarding Python that I've absolutely no idea how to solve this.
Hint: Prior to making the download.py file, the folder, and typing the Terminal code the only interactions I made with Python were downloading the program (3.6) and placing it in my toolbar. I'm not even quite sure if I am required to create any additional scripts/text files, or make any additional downloads before a script like this would work and successfully download the resulting images into my "Images" folder as is my desired goal. If I sincerely missed something integral at any point during this long read, hopefully, someone in here can provide a thoroughly detailed explanation as to how to solve my issue.
Finishing statements for those who've managed to stick along this far:
Thank you. I know this is one hell of a read, and I'm getting more tired as I go along. What I hope to get out of this question is
1.) Obviously, what would constitute a direct solution to the "No module named request" Input Error in Terminal. In other words, what I did wrong there or am missing.
2.) Any other helpful information that you know would assist this code, for example, if there is any integral step or condition I've missed or failed to meet that would ultimately cause the entirety of my code to cease to work. If you do see a fault in this, I only ask of you to be specific, as I've not got much experience in the programming world. After all, I know there is a lot of developers out here that are far more informed and experienced than am I. Thanks.
urllib.request is in Python 3 only. When running 'python' on a Mac, you're running Python 2 by default. Try running executing with python3.
python --version
might need to
brew install python3
urllib.request is a Python 3 construct. Most systems run Python 2 as default and this is what you get when you run simply python.
To install Python 3, go to https://brew.sh/ and follow the instructions to install the Hombrew package manager. Then run
brew install python3
python3 download.py

error with Xyce Gnuplot.py

I am using Xyce which is a circuit simulator. I am using it to export a .CSV file and a .prn file. I found Xycegnuplot.py "https://github.com/OpenXyce/Xyce/blob/master/utils/gnuplotXyce.py". I am trying to use it to plot my output variables from Xyce, howver, every time I run gnuplotXyce.py as mentioned by its author I get an error " Import Error" at the "from finblock import findblock" line and I don't know what is that error.
Please help.
Thanks
If you are going to use Xyce, you should probably get the official version from Sandia National Laboratories instead of from the OpenXyce site on github. This version was forked by an anonymous github user, and has not been updated since last fall. Since that update, Sandia released Xyce 6.2 and the OpenXyce creator did not import the new release.
You should also probably join the xyce-users group on googlegroups, where the Xyce developers monitor all questions and try to answer them promptly. It is only by happenstance that I found your question here on stackoverflow.
The "gnuplotXyce.py" script is not really maintained, and might not have been kept working with all the changes that have been made to Xyce since its release. That said, the python script depends on a number of python modules including gnuplot-py which should be available from http://gnuplot-py.sourceforge.net. The "findblock.py" module that you say cannot be found is also present in the "utils" directory of the Xyce source code, alongside gnuplotXyce.py. If you have the whole utils directory downloaded, this error should go away.
I just tried gnuplotXyce.py on a simple netlist with csv output and it didn't work, so my assumption is that the script was not maintained and will need to be fixed.
The script does sort of work if you use the native Xyce standard (.prn) format (i.e. don't specify "format=csv" on your .print line). Unfortunately, it does not leave the window open after it finishes plotting, so it is rather useless. If you use the "--ps" option, though, a correct postscript file will be created that can be viewed in any postscript viewer, or printed on a postscript printer (or through a properly set-up Linux CUPS printer that understands postscript).
The CSV format in Xyce was primarily created in order to allow import into spreadsheets such as Excel or OpenOffice-scalc, which programs have their own plotting utilities.
The ".prn" standard format works well in gnuplot. There is an example of how to use gnuplot to do this display in the document "Using Open Source Schematic Capture Tools With Xyce" on the Sandia Labs Xyce web site (in the documentation and tutorials section).
The official Xyce web site is http://xyce.sandia.gov/

Can python read the value of a cell in a spreadsheet?

All,
Can python read the value of a cell in a spreadsheet?
From a mapping/GIS/analysis standpoint: the simplest example would be a script that ran a buffer (proximity) tool on a given shapefile (GIS dataset).
For the buffer distance parameter, instead of just using a number like '1000' feet, the script would point to a value in a cell of a spreadsheet (libre or open office preferred).
If there was then a way to trigger the script from the spreadsheet by way of a button, that would be the next step (then the next step would be to have a map control inside the spreadsheet to see the updated results!)
Just to give some insight into where I'm going with this: I'd like to use a spreadsheet as an analysis 'dashboard' where users could run analysis with different parameters - what would proximity around parks (grocery stores, etc.) be at 1/2 mi vs 1/4 mi...then another sheet in the spreadsheet would have a breakdown of the demographics within that proximity.
Thank you!!!
(also posted here: https://gis.stackexchange.com/questions/49288/can-python-read-the-value-of-a-cell-in-a-spreadsheet)
-mb
pyoo is a simple package. Install
python-uno
python3-uno
Then install pyoo
python setup.py install
python3 setup.py install
run soffice (LibreOffice or OpenOffice)
soffice --accept="socket,host=localhost,port=2002;urp;" --norestore --nologo --nodefault # --headless
The following script shows how it works in python
desktop = pyoo.Desktop('localhost', 2002)
doc = desktop.open_spreadsheet(path)
sheet = doc.sheets[0]
for i in range(0,14):
for j in range(0,4):
print(sheet[i,j].value)
There are a few great Python-Excel tools available: http://www.python-excel.org
It's all great but soo vague. Where do you execute the install command? And even if you somehow figure out you have to go to the pip3.6.exe excript and run it from there, it won't find python3-uno. It will fin python-uno but even then you won't run any script with import uno in the beginning because of missing elements, which are somehow impossible to get. It is stupidly hard to do even the simplest thing this way in LibreOffice.

Categories