We Keep Coding

Convert multiple multipage PDFs to JPGs in subfolders

Simple use case:
A folder with many (mostly multipage) PDF files.
A script should convert each PDF page to JPG and store it in a subfolder named after the PDF filename. (e.g. #33.pdf to folder #33)
Single JPG files should also have this filename plus a counter mirroring the sequential page number in the PDF. (e.g. #33_001.jpg)
I found a bounch of related questions, but nothing that quite does what I want, e.g.
How do I convert multiple PDFs into images from the same folder in Python?
A python script would work fine, but also any other way to do this in Win10 (imagemagick, e.g.) is cool with me.

Your comment requests how a batch can do as required, for simplicity the following only processes a single file so Python will need to loop through a folder and call with each name in turn. That could be done by adding a "for loop" in batch but first see where problems arise, as many of my single test files threw differing errors.
I have tried to cover several fails in this batch file, in my system, but there can still be issues such as a file that has no valid fonts to display
For most recent poppler windows 64bit utils see https://github.com/oschwartz10612/poppler-windows/releases/ for 32 bit use xpdf latest version http://www.xpdfreader.com/download.html but that has direct pdftopng.exe so needs a few edits.
pdf2dir.bat
#echo off
set "bin=C:\Apps\PDF\poppler\22.11.0\Library\bin"
set "res=200"
REM for type use one of 3 i.e. png jpeg jpegcmyk (PNG is best for documents)
set "type=png"
if exist "%~dpn1\*.%type%" echo: &echo Files already exist in "%~dpn1" skipping overwrite&goto pause
if not exist "%~dpn1.pdf" echo: &echo "%~dpn0" File "%~dpn1.pdf" not found&goto pause
if not exist "%~dpn1\*.*" md "%~dpn1"
REM following line deliberately opens folder to show progress delete it or prefix with REM for blind running
explorer "%~dpn1"
"%bin%\pdftoppm.exe" -%type% -r %res% "%~dpn1.pdf" "%~dpn1\%~n1"
if %errorlevel%==1 echo: &echo Build of %type% files failed&goto pause
if not exist "%~dpn1\*.%type%" echo: &echo Build of %type% files failed&goto pause
:pause
echo:
pause
:end
It requires Poppler binaries path to pdftoppm be correctly set in the second line
It can be placed wherever desired i.e. work folder or desktop
It allows for drag and drop of one pdf on top will (should work) without need to run in console
Can be run in a command console and place a space character after, you can drag and drop a single filename but any spaces in name must be "double quoted"
can be run from any shell or OS command as "path to/batchfile.bat" "c:\path to\file.pdf"

Using binary file stored in storage account from python script

I am trying to run a python script (the script resides on a directory in \wwwroot directory in my app file system) from my web app using java, the script uses a binary file (g.bin), the binary file size is huge (3.5+ GB), so I stored it in a storage account (file service), now I want to use the data found in the bin file, How can I open the file using my script? What is the path that the python script should use and how to open it?

You need to use azure-storage python SDK, and the module you need is azure.storage.file.fileservice
First initialize the FileService
f = FileService(account_name=None, account_
key=None, sas_token=None,
protocol=’https’, endpoint_
suffix=’core.windows.net’,
request_session=None, connection_
string=None)
Then you need to download the file to a stream using fileservice.get_file_to_stream method.
f.get_file_to_stream(share_name, directory_name, file_name, stream, start_range=None, end_range=None, validate_content=False, progress_callback=None, max_connections=2, timeout=None, snapshot=None)
Give that a try. If you can't get it to work, update your question with what you tried, and I'll be happy to help.

How to get files written by the micro:bit to the PC?

I am new to BBC micro:bit, so I have the following Problem:
I want to read movements and write it to a file on the m:b and after all I want to download it from there to the pc to work on it.
I wrote the file like that:
from microbit import *
with open('FileName.txt', 'w') as my_file:
my_file.write('text to write down')
I couldn't see the file, when I used the m:b as USB- device.
But when I programmed the m:b to list all files it wrote short before the file was on it.
I know, m:b has no disk operating System, so I tried to pull it with python code, I started the following python code on a Windows pc:
(see: Docs to microfs)
import microfs
print microfs.ls()
But I got the error IOError: Could not find micro:bit.
The m:b is not found, I suppose.
What am I doing wrong? What else could I try?

Sometimes if the micro:bit cannot be found by scripts like uFlash or MicroFs it helps if you unplug the USB cable, wait a few seconds and plug it again (an additional note for Linux users, although I am aware that this is not the case for you, on Linux is also helpful to wait until the micro:bit drive has been mounted).
You are in the right track using MicroFs to access the MicroPython files, as they are in the microcontroller flash, and not accessible through the USB mass storage interface. Remember that writing a new program into the micro:bit does erase all of the flash contents, including any files your previous program might have created.
For easy of use I would recommend using the Mu editor (https://codewith.mu), as it offers you a GUI to move files in and out of the micro:bit. It's worth noting that Mu uses uFlash and MicroFs internally, so it will give you the same results as using those individual command line tools.

I, too, have spent several hours searching for the answer to getting microfs and mu to read a file from the microbit, after receiving the 'can't find it' message.
I have just found a solution: update the microbit firmware.
I have flashed an empty Python file from mu to the microbit and then used the command line ufs put <path to file>\main.py to copy over code that creates a text file and displays a heart on the microbit.
Now the Files option in mu correctly displays main.py and the created text file on my microbit.
I hope this helps.

As mentioned, you can use the command line ufs to put, get, list and delete files on the microbit.
pip install microfs --user
Then use ufs to list, delete, put and get files on to and off the microbit.
github site: https://github.com/ntoll/microfs.
The commands are:
ufs ls #list files on the card
ufs rm <filename> # remove a file
ufs put <filename> # write a file to the micro:bit
ufs get <filename> # get a file from the micro:bit
First of all put a blank .py file on to the micro:bit. In Linux you can create this using:
touch empty.py
Connect with microbit by double clicking on it using your file browser (e.g. Nautilus in Linux).
Use mu to flash empty.py onto the microbit.
Then write your Micropython code and call it main.py. Use ufs to write main.py to the micro:bit.
ufs main.py
This file will run at reset and restart. You can have your main.py file import and use other Micropython files on the micro:bit. Put these onto the micro:bit using ufs.
ufs <file to import to main.py>.py
e.g.
ufs put utilities.py
The files can be overwritten using ufs put.
You can't use ufs and a repl at the same time.
Now you are in a position to write and read a text file. Please find two example functions I have used for this.
def read_file(filename):
with open(filename, 'r') as my_file:
read_value = my_file.read()
return read_value
def write_file(filename, value):
with open(filename, 'w') as my_file:
my_file.write(str(value))
my_file.close()
The files are written to the microbit and the data remains intact after repowering the device. The data file can be seen using:
ufs ls
Then copied to your local machine using:
ufs get <filename>

Allen Brain Institute - brain observatory example

I'm trying to follow the example of brain observatory ipython notebook.
However, I became stuck loading the nwb file like below.
from allensdk.core.brain_observatory_cache import BrainObservatoryCache
boc = BrainObservatoryCache(manifest_file='boc/manifest.json')
data_set = boc.get_ophys_experiment_data(501940850) # problem here
So, I opened the nwb file by HDFview.
All of the brain observatory nwb files were not opened except for 502376461.nwb.
It threw the following error:
IOError: Unable to open file (Truncated file: eof = 82280448, sblock->base_addr = 0, stored_eoa = 204046519)
When I tried to open the 502376461.nwb in the ipython notebook example from allen, it worked!! But the others (501940850, 503820068...) failed like above.

Summarizing the thread from github:
https://github.com/AllenInstitute/AllenSDK/issues/22
The files were partially downloaded or corrupted somehow. No exceptions were reported during the download, so urllib must not have noticed a problem.
AllenSDK developers are investigating some sort of file consistency check and/or a different HTTP library.
https://github.com/AllenInstitute/AllenSDK/issues/28
If others run into this, you can delete the bad file and re-run the download function (BrainObservatoryCache.get_ophys_experiment_data). Files are downloaded into a subdirectory of the BrainObservatoryCache manifest file, which defaults to the current working directory if unspecified.

AppDailySales: Works, but the downloaded gzip file is corrupted

I am trying to use the appdailysales.py module to download daily our iPhone apps. I am a .NET developer, so I tried running this using IronPython in a C# solution using the following code:
using IronPython.Hosting;
var ipy = Python.CreateRuntime();
dynamic appSales = ipy.UseFile("appdailysales.py");
appSales.main();
Because I didn't have gzip, I took out the references to that module. I was going to use the GZipStream C# class to decompress the file (Apple, provides their downloads as .gz files). So, I commented out lines 75 and 429-435.
I have tried executing appdailysales.py in my C# solution, directly from IronPython and using Python 2.7 (installed ActivePython last night); all with the same results: When I try to open the .gz file using 7zip, I get the following error:
CRC Failed ... file is broken
When I try using the GZipStream class I get:
The CRC in GZip footer does not match the CRC calculated from the decompressed data
If I download the .gz file manually, I can decompress the file just fine using 7Zip or GZipStream.
I am fluent in C#, but new to Python. Any help you can provide would be much appreciated.
Thanks for your time.

Looks like line 444 is the problem. Here are lines 444-446:
downloadFile = open(filename, 'w')
downloadFile.write(filebuffer)
downloadFile.close()
At this stage, IF you have deleted lines 429-435 OR selected not to unzip, then filebuffer refers to the raw gzipped stream that you got from the web. The output file is opened in TEXT mode, and you are on Windows, so every \n in the BINARY gzipped stream will be converted to \r\n -- CORRUPTION, like the error message said.
So: for the module to be used portably on both Windows and other platforms, the open mode must be "wb" (b for binary). If the gunzipped result file is also a binary file, "wb" can be hardcoded in the open call. However if the gunzipped file is a text file (meant to be capable of being opened in a text editor), then you need just "w" for that purpose, and you should set a variable mode to either "wb" or "w" as appropriate, and use mode in the open call.
Big question: I understand why you removed the gzip references for IronPython usage. Did you remove those lines for Python 2.7? Or did you run it under Python 2.7 with those lines still in, but set options.unzipFile to False?