Using binary file stored in storage account from python script - python

I am trying to run a python script (the script resides on a directory in \wwwroot directory in my app file system) from my web app using java, the script uses a binary file (g.bin), the binary file size is huge (3.5+ GB), so I stored it in a storage account (file service), now I want to use the data found in the bin file, How can I open the file using my script? What is the path that the python script should use and how to open it?

You need to use azure-storage python SDK, and the module you need is azure.storage.file.fileservice
First initialize the FileService
f = FileService(account_name=None, account_
key=None, sas_token=None,
protocol=’https’, endpoint_
suffix=’core.windows.net’,
request_session=None, connection_
string=None)
Then you need to download the file to a stream using fileservice.get_file_to_stream method.
f.get_file_to_stream(share_name, directory_name, file_name, stream, start_range=None, end_range=None, validate_content=False, progress_callback=None, max_connections=2, timeout=None, snapshot=None)
Give that a try. If you can't get it to work, update your question with what you tried, and I'll be happy to help.

Related

Decided: How in Python 3 encode file, for open this file with PHP zlib_decode function?

EDIT:
I found a fix, must use wb+ option for save binary files and dont use str()
Code in PHP that i cant change:
file_get_contents('data.pak');
$s_st = unserialize(zlib_decode($s_st));
I try this Python code:
import zlib
from phpserialize import serialize
file = zlib.compress(serialize(data), 9)
with open('data.pak', 'w+') as f_out:
f_out.write(str(file))
f_out.close()
But PHP cant open this file.
I also install PHP to check what file create zlib_encode (zlib_encode(serialize($data),15)) with same data, and these files are different.
Python file starts from:
b'\xa5\xbd\xcb\xb2,)\xaeh\xdb\x8e\xf3\x19\xd5O3\xe7\rY_s\x9b\xbb\xbd\x9b\xd7\xce\xbf\
Looks like file content in bytes
PHP file starts from:
њҐќ[Ом(і¦ЇіҐћDЯ—dО°{X=©‰ њ пЗRХ_»цЄ§Н1€гяыw]!‡ящ?ЖеЏsсъ„l?—фЧяю_ќuз!Фz¤СЙ`\ши
So my question is: How create file in Python, that I can then open in PHP, using code at the beginning.

How to get files written by the micro:bit to the PC?

I am new to BBC micro:bit, so I have the following Problem:
I want to read movements and write it to a file on the m:b and after all I want to download it from there to the pc to work on it.
I wrote the file like that:
from microbit import *
with open('FileName.txt', 'w') as my_file:
my_file.write('text to write down')
I couldn't see the file, when I used the m:b as USB- device.
But when I programmed the m:b to list all files it wrote short before the file was on it.
I know, m:b has no disk operating System, so I tried to pull it with python code, I started the following python code on a Windows pc:
(see: Docs to microfs)
import microfs
print microfs.ls()
But I got the error IOError: Could not find micro:bit.
The m:b is not found, I suppose.
What am I doing wrong? What else could I try?
Sometimes if the micro:bit cannot be found by scripts like uFlash or MicroFs it helps if you unplug the USB cable, wait a few seconds and plug it again (an additional note for Linux users, although I am aware that this is not the case for you, on Linux is also helpful to wait until the micro:bit drive has been mounted).
You are in the right track using MicroFs to access the MicroPython files, as they are in the microcontroller flash, and not accessible through the USB mass storage interface. Remember that writing a new program into the micro:bit does erase all of the flash contents, including any files your previous program might have created.
For easy of use I would recommend using the Mu editor (https://codewith.mu), as it offers you a GUI to move files in and out of the micro:bit. It's worth noting that Mu uses uFlash and MicroFs internally, so it will give you the same results as using those individual command line tools.
I, too, have spent several hours searching for the answer to getting microfs and mu to read a file from the microbit, after receiving the 'can't find it' message.
I have just found a solution: update the microbit firmware.
I have flashed an empty Python file from mu to the microbit and then used the command line ufs put <path to file>\main.py to copy over code that creates a text file and displays a heart on the microbit.
Now the Files option in mu correctly displays main.py and the created text file on my microbit.
I hope this helps.
As mentioned, you can use the command line ufs to put, get, list and delete files on the microbit.
pip install microfs --user
Then use ufs to list, delete, put and get files on to and off the microbit.
github site: https://github.com/ntoll/microfs.
The commands are:
ufs ls #list files on the card
ufs rm <filename> # remove a file
ufs put <filename> # write a file to the micro:bit
ufs get <filename> # get a file from the micro:bit
First of all put a blank .py file on to the micro:bit. In Linux you can create this using:
touch empty.py
Connect with microbit by double clicking on it using your file browser (e.g. Nautilus in Linux).
Use mu to flash empty.py onto the microbit.
Then write your Micropython code and call it main.py. Use ufs to write main.py to the micro:bit.
ufs main.py
This file will run at reset and restart. You can have your main.py file import and use other Micropython files on the micro:bit. Put these onto the micro:bit using ufs.
ufs <file to import to main.py>.py
e.g.
ufs put utilities.py
The files can be overwritten using ufs put.
You can't use ufs and a repl at the same time.
Now you are in a position to write and read a text file. Please find two example functions I have used for this.
def read_file(filename):
with open(filename, 'r') as my_file:
read_value = my_file.read()
return read_value
def write_file(filename, value):
with open(filename, 'w') as my_file:
my_file.write(str(value))
my_file.close()
The files are written to the microbit and the data remains intact after repowering the device. The data file can be seen using:
ufs ls
Then copied to your local machine using:
ufs get <filename>

Python - Download file over HTTP and detect filetype automatically

I want download a file via HTTP, but all the examples online involve fetching the data and then putting it in a local file. The problem with this is that you need to explicitly set the filetype of the local file.
I want to download a file but I won't know the filetype of what I'm downloading.
This is what I currently have:
urllib.urlretrieve(fetch_url,output.csv)
But if I download, say a XML file it will be CSV. Is there anyway to get python to detect the file that I get sent from a URL like: http://asassaassa.com/assaas?abc=123
Say the above URL gives me an XML I want python to detect that.
You can use python-magic to detect file type. It can be installed via "pip install python-magic".
I assume you are using python 2.7 since you are calling urlretreieve. The example is geared to 2.7, but it is easily adapted.
This is a working example:
import mimetypes # Detects mimetype
import magic # Uses magic numbers to detect file type, and does so much better than the built in mimetypes
import urllib # Your library
import os # for renaming your file
mime = magic.Magic(mime=True)
output = "output" # Your file name without extension
urllib.urlretrieve("https://docs.python.org/3.0/library/mimetypes.html", output) # This is just an example url
mimes = mime.from_file(output) # Get mime type
ext = mimetypes.guess_all_extensions(mimes)[0] # Guess extension
os.rename(output, output+ext) # Rename file

Issues in copying files on iOS using NSStreams

I am trying to copy image and media files using NSStreams. I can not use NSFileManager copyItemAtPath, as I have to copy the file using streams.
The data is transferred over the network and the stream is read by a Python script that writes the data to a file. This worked fine on Mac OSX but when I tried in iOS,the file was not saved in the proper format.
I am able to copy all the files, but some of the metadata like dimensions (for image and media files), and duration (for media files) is missing in the copied file, and the kind is always Document. The other metadata is fine.
When I try to read the file attributes using the NSFileManager
[[NSFileManager defaultManager] attributesOfItemAtPath:#"filePath" error:&error];
It shows an error in the console:
The operation couldn't be completed. No such file or directory
I also observed that all the copied files, irrespective of the file extension (.png,.jpeg,.mov, .zip), has a kind of Document
How do I copy the source image metadata into the copied file?
Are there any Xcode optimizations I need to turn off?
OS : Mac OSX 10.8.4, iOS 6
Xcode : 4.6.3
This works for me for any type of file:
if (![NSFileManager.defaultManager copyItemAtPath: sourceFileName toPath: targetFileName error: &error]) {
NSAlert *alert = [NSAlert alertWithError: error];
[alert runModal];
return;
}
I found out it is an issue with file extension.Some junk characters appended after the file extension( something like 1.png\\\)

AppDailySales: Works, but the downloaded gzip file is corrupted

I am trying to use the appdailysales.py module to download daily our iPhone apps. I am a .NET developer, so I tried running this using IronPython in a C# solution using the following code:
using IronPython.Hosting;
var ipy = Python.CreateRuntime();
dynamic appSales = ipy.UseFile("appdailysales.py");
appSales.main();
Because I didn't have gzip, I took out the references to that module. I was going to use the GZipStream C# class to decompress the file (Apple, provides their downloads as .gz files). So, I commented out lines 75 and 429-435.
I have tried executing appdailysales.py in my C# solution, directly from IronPython and using Python 2.7 (installed ActivePython last night); all with the same results: When I try to open the .gz file using 7zip, I get the following error:
CRC Failed ... file is broken
When I try using the GZipStream class I get:
The CRC in GZip footer does not match the CRC calculated from the decompressed data
If I download the .gz file manually, I can decompress the file just fine using 7Zip or GZipStream.
I am fluent in C#, but new to Python. Any help you can provide would be much appreciated.
Thanks for your time.
Looks like line 444 is the problem. Here are lines 444-446:
downloadFile = open(filename, 'w')
downloadFile.write(filebuffer)
downloadFile.close()
At this stage, IF you have deleted lines 429-435 OR selected not to unzip, then filebuffer refers to the raw gzipped stream that you got from the web. The output file is opened in TEXT mode, and you are on Windows, so every \n in the BINARY gzipped stream will be converted to \r\n -- CORRUPTION, like the error message said.
So: for the module to be used portably on both Windows and other platforms, the open mode must be "wb" (b for binary). If the gunzipped result file is also a binary file, "wb" can be hardcoded in the open call. However if the gunzipped file is a text file (meant to be capable of being opened in a text editor), then you need just "w" for that purpose, and you should set a variable mode to either "wb" or "w" as appropriate, and use mode in the open call.
Big question: I understand why you removed the gzip references for IronPython usage. Did you remove those lines for Python 2.7? Or did you run it under Python 2.7 with those lines still in, but set options.unzipFile to False?

Categories