I have a python code for doing "advanced analytics". Right now python reads input from an excel file (multiple sheets) and stores output to a new excel file.
Since python uses an external solver and custom modules I'm planning on putting everything in a docker container for convenience.
Now I'd like to be able to "click a button" in an excel workbook to send the contents of multiple sheets to my analytics engine and get the result back to a new sheet. I'm thinking about communicating with the docker container using API, but not sure if this is the most efficient approach. Is there a better way of solving this problem?
Related
I am working for a company which is currently storing PDF files into a remote drive and subsequently manually inserting values found within these files into an Excel document. I would like to automate the process using Zapier, and make the process scalable (we receive a large amount of PDF files). Would anyone know any applications useful and possibly free for converting PDFs into Excel docs and which integrate with Zapier? Alternatively, would it be possible to create a Python script in Zapier to access the information and store it into an Excel file?
This option came to mind. I'm using google drive as an example, you didn't say what you where using as storage, but Zapier should have an option for it.
Use cloud convert, doc parser (depends on what you want to pay, cloud convert at least gives you some free time per month, so that may be the closest you can get).
Create a zap with this step:
Trigger on new file in drive (Name: Convert new Google Drive files with CloudConvert)
Convert file with CloudConvert
Those are two options by Zapier that I can find. But you could also do it in python from your desktop by following something like this idea. Then set an event controller in windows event manager to trigger an upload/download.
Unfortunately it doesn't seem that you can import JS/Python libraries into zapier, however I may be wrong on that. If you could, or find a way to do so, then just use PDFminer and "Code by Zapier". A technician might have to confirm this though, I've never gotten libraries to work in zaps.
Hope that helps!
When using Google Data Prep, I am able to create automated schedules to run jobs that update my BigQuery tables.
However, this seems pointless when considering that the data used in Prep is updated by manually dragging and dropping CSVs (or JSON, xlsx, whatever) into the data storage bucket.
I have attempted to search for a definitive way of updating this bucket automatically with files that are regularly updated on my PC, but there seems to be no best-practice solution that I can find.
How should one go about doing this efficiently and effectively?
So, in order to upload files from your computer to Google Cloud Storage, there are a few possibilities. If you just run an daemon process which handles any change in that shared directory, you can code an automatic upload in this different languages: C#, Go, Java, Node.JS, PHP, Python or Ruby.
You have here some code examples for uploading objects but, be aware that there is also a detailed Cloud Storage Client Libraries references and you can also find the GitHub links in "Additional Resources".
I have an Excel sheet with a worksheet named test_sheet. I have a Google sheet named G_sheet having 5 tabs. I would like to automate the process of copying the contents of Excel worksheet test_sheet and paste it in a specific tab, say 'sheet1', of the Google sheet. How can be done this using excel VBA?
I am using Excel 2013.
Your going to need
Pandas
Watchdog
Pygsheets
Pandas read xlsx/csv files, convert them to dictionaries, so that you can apply a batch update using pygsheets.
Watchdog, to watch you xlsx files for modifications. When a modification is made, you re-read the files with pandas, and update the spreadsheet on google using pygsheets.
I may be misinterpreting the question, but if you just want to copy a file from Excel to Google Sheets you can just upload it with their web interface and it will be automatically converted.
If you need the file to be synchronized, you can use the Google Drive sync client, or save the Excel sheet to OneDrive and use Zapier to synchronize the file.
If, for whatever reason, you need to upload the file programmatically, it will be easier to just automate the user inputs with a program like AutoHotkey than to create a Python program to do this.
If you really need the program in Python (which is after all one of the most useful languages for data processing), you can use xlwings to read the sheet and the Google Sheets API for output.
Another thing you could do to speed up the file read-in would be to add some simple VBA code to the Excel file to output the entire contents to a format more easily readable by python, or exporting the whole sheet as csv and then using the python csv library to read it in.
Another question is why you need it in Google Sheets format anyway. If you just need the file to be accessible via a link, you could upload the sheet to OneDrive and make it accessible. If you need it to be accessible by other programs over the internet, you could transfer it directly or make the file accessible over FTP.
If you have Google's OAuth2 credentials,:
http://gspread.readthedocs.io/en/latest/oauth2.html
just use:
https://github.com/burnash/gspread
You could use googles program called Backup and Sync to easily save any folder on your computer to your google drive.
You just have to set up the sync settings to where it only saves that one excel sheet to a place on your computer. And then every time you save the doc it will automatically be synced with the copy of the excel sheet on your google drive.
Using python, I want to continuously record some Excel calculations. The python/excel functions I have used will only read excel data from a saved spreadsheet. Is there a way to read unsaved data?
If not, is there a way to save excel and read the saved data periodically? I tried saving my sheets periodically (with a macro) but this is problematic if I am interacting with the spreadsheet during a save. Instead of saving my spreadsheet, excel saves a copy with a random name. If there is a way to remedy this (maybe some kind of vba error handling) that may solve the problem. Thanks.
Assuming you are using Excel on Windows and not OS X, you can try the win32com.client module, which you can get by installing the Python for Windows package:
http://sourceforge.net/projects/pywin32/
Unfortunately, the available documentation is pretty spotty at best... here's a start:
http://docs.activestate.com/activepython/3.3/pywin32/html/com/win32com/HTML/QuickStartClientCom.html
I should warn you that the COM API isn't very easy to use, so you are really better off sticking with VBA if your workflow requires Excel.
VBA error handling: http://www.cpearson.com/excel/errorhandling.htm
Is it possible in python by which I can write a simple .py script to update my access database records or insert new one if any i have on behalf of me? new records are to be pulled from Excel and pushed to be in the database.
MS-Access2010 i am using.
Thanks,
It's definitely possible. You'll probably want to do it with the comtypes module, which allows communication between Windows processes using the Component Object Model (COM).
Here's an example of a script that does that posted in another question.
Getting the information out of Microsoft Excel can be done with a lot of modules, but one I've had success with is openpyxl. Some examples of reading Excel workbooks with it can be found here.