We Keep Coding

Using Shareplum to download, alter and then upload a file to sharepoint

Exactly as the title says, I have this code
from shareplum import Site
from shareplum import Office365
from shareplum.site import Version
authcookie = Office365('https://mysite.sharepoint.com/', username='username', password='password').GetCookies()
site = Site('https://mysite.sharepoint.com/sites/mysite/', version=Version.v2016, authcookie=authcookie)
folder = site.Folder('Shared Documents/Beta Testing')
file = folder.get_file('practice.xlsx')
with open("practice.xlsx", "wb") as fh:
fh.write(file)
print('---')
folder.upload_file('xlsx', 'practice.xlsx')
Currently it downloads the file just fine which is fantastic, however I do not know how to reverse what I did with opening and downloading the file. Basically I need to be able to upload the file with the exact same name as the one I downloaded in the exact same format (in this case xlsx) as to overwrite the one in the sharepoint with the updated document.

Your post indicates that you want to modify the file so you will need some file handling for the downloaded file once it is saved after modification. Once the file modification has been done you need to open the file in 'rb' and then read that to a variable which will be the content when calling folder_obj.upload_file(content, name).
#this is your step to modify the file.
with open("practice.xlsx", "wb") as fh:
#file modification stuff... pyxlsx?
fh.write(file)
#open the file and read it into a variable as binary
with open("practice.xlsx", "rb") as file_obj:
file_as_string = file_obj.read()
#upload the file including the file name and the variable (file_as_string)
folder.upload_file(file_as_string, 'practice.xlsx')
This has been working for me. If you want to change the name of the file to include a version, delete the old file by calling folder.delete_file("practice.xlsx").

Can you try the below and see if it works?
with open("practice.xlsx", "wb") as fh:
file_content = fh.write(file)
folder.upload_file(file_content,'practice.xlsx')

Azure - Files in file share with given extension

I'm using this to connect to Azure File Share and upload a file. I would like to chose what extension file will have, but I can't. I got an error shown below. If I remove .txt everything works fine. Is there a way to specify file extension while uploading it?
Error:
Exception: ResourceNotFoundError: The specified parent path does not exist.
Code:
def main(blobin: func.InputStream):
file_client = ShareFileClient.from_connection_string(conn_str="<con_string>",
share_name="data-storage",
file_path="outgoing/file.txt")
f = open('/home/temp.txt', 'w+')
f.write(blobin.read().decode('utf-8'))
f.close()
# Operation on file here
f = open('/home/temp.txt', 'rb')
string_to_upload = f.read()
f.close()
file_client.upload_file(string_to_upload)

I believe the reason you're getting this error is because outgoing folder doesn't exist in your file service share. I took your code and ran it with and without extension and in both situation I got the same error.
Then I created a folder and tried to upload the file and I was able to successfully do so.
Here's the final code I used:
from azure.storage.fileshare import ShareFileClient, ShareDirectoryClient
conn_string = "DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=myaccountkey;EndpointSuffix=core.windows.net"
share_directory_client = ShareDirectoryClient.from_connection_string(conn_str=conn_string,
share_name="data-storage",
directory_path="outgoing")
file_client = ShareFileClient.from_connection_string(conn_str=conn_string,
share_name="data-storage",
file_path="outgoing/file.txt")
# Create folder first.
# This operation will fail if the directory already exists.
print "creating directory..."
share_directory_client.create_directory()
print "directory created successfully..."
# Operation on file here
f = open('D:\\temp\\test.txt', 'rb')
string_to_upload = f.read()
f.close()
#Upload file
print "uploading file..."
file_client.upload_file(string_to_upload)
print "file uploaded successfully..."

Creating an on-the-fly zip file from string content for AWS Lambda in Python

I have a Python script that creates a Lambda script in AWS along with all the policies and triggers. I use python boto3 library for that. I create the zip file for the lambda as on-the-fly rather than uploading a static zip file from the hard drive. I use this simple code from below to create my zip file. It creates the zip file without any problems and my python code uploads this zip file as a lambda script and I can view my lambda script in the AWS without any problems. But when I run my lambda script it gives me the module not found error even though I can clearly see that both the module name and the file name does exist and is view-able.
Unable to import module 'xxxx': No module named xxxx
In the file system I double click that zip file that was created by this code and see that the content is created and everything looks normal.
If I bypass zipping on the fly and create the zip statically using WinZip and let the rest of the Python & boto3 script upload this file then it works just fine.
def CreateLambdaZip(self, fileName, fileContent):
with zipfile.ZipFile('Lambda/' + fileName + '.zip', 'w') as myzipc:
myzipc.writestr( fileName + '.py', fileContent)
myzipc.close()
It kinda looks like for the zip file I'm skipping some special headers that is needed by Aws Lambda. Is there such thing? Because in the file system the zip file that is created by Python code and the other one that is created by WinZip are exactly the same. So I know there's nothing wrong with the lambda script.
Update: I'm uploading the zip file using the below code that reads the zip file which was created using the above snippet.
with open('Lambda/'+ fileName +'.zip', 'rb') as zipFile:
func = boto3.client("Lambda").create_function(
FunctionName=lambdaFunction,
Runtime='python2.7',
Role=role['Role']['Arn'],
Handler= fileName + "." + functionName,
Description=description,
Timeout=10,
MemorySize=256,
Publish=True,
Code={'ZipFile': zipFile.read()},
)
When I use zipFile.read() I get 2 different headers for the same content when I zip it using WinZip and when I zip it using Python's module.
Zip file that's created programmatically using Python
b'PK\x03\x04\x14\x00\x00\x00\x00\x00\xe4~\x01IO\x96J=Z\x07\x00\x00Z\x07\x00\x00\x19\x00\x00\x00schedule-ec2-snapshots.pyimport json\nimport boto3\nimport time\nfrom datetime import date, timedelta\n\nprint(\'Loading scheduled EC2 backup actions\')\n\ndef create_snapshots(event, context):\n """\n Lambda function that executes daily snapshots for the instances that
and zipfile created by WinZip
b'PK\x03\x04\x14\x00\x02\x00\x08\x004X\xfcH\x88\x1f\xce\xb5&\x03\x00\x00b\x07\x00\x00\x19\x00\x00\x00schedule-ec2-snapshots.py\x8dU]k\xdb#\x10|7\xf4?,\nA\x12qL\xda\x06B\r~I\x93Bh\x9b\x87&\xf4E\x15\xe1\xac[\xdb\xd7HwBw2\t\xc1\xff\xbd{+\xeb\xcb.\xb4\n\xc4\xba\xdb\xd1\xec\xce\xdc\xae\xa4\x8a\xd2T\x0e~[\xa3\'\xaa\xb9_\x1ag>\xb6\x0b\xa7\n\x9c\xac*S\x80\x14\x0e\xfd\n\xf6\x11\xbf\x9er\\b\xee\xc4dRVJ\xbb(\xfcf\x84Tz\r6\xdb\xa0\xacs\x94p\xfb\xf9\x03,E\xf6\\\x97

With the info above I was able to start the in-memory solution. The deployment of that zip file worked but I could not use the resulting function. Got error:
Unable to import module '<function-name>': No module named <function-name>
I got it to work by specifying the file permissions.
I then use the in-mem-zip to create an AWS lambda function.
Setup:
file_map is a dictionary of full_path->file_bytes.
files is a list of full_paths
def create_lambda_function(function_name, desc, role, handler, file_map, files)
zip_contents = create_in_mem_zip_archive(file_map, files)
result = lambda_code.create_function(
FunctionName=function_name,
Runtime="python2.7",
Description=desc,
Role=role,
Handler=handler,
Code={'ZipFile': zip_contents},
)
return result
def create_in_mem_zip_archive(file_map, files):
buf = io.BytesIO()
logger.info("Building zip file: " + str(files))
with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as zfh:
for file_name in files:
file_blob = file_map.get(file_name)
if file_blob is None:
logger.error("Missing file {} from files".format(file_name))
continue
try:
info = zipfile.ZipInfo(file_name)
info.date_time = time.localtime()
info.compress_type = zipfile.ZIP_DEFLATED
info.external_attr = 0777 << 16L # give full access
# info.external_attr = 0644 << 16L # -r-wr--r--
# info.external_attr = 0755 << 16L # -rwxr-xr-x
zfh.writestr(info, file_blob)
except Exception as ex:
logger.info("Error reading file: " + file_name + ", error: " + ex.message)
buf.seek(0)
return buf.read()

I have experienced the exactly same problem you have. My solution is do NOT use on the fly zip file. Create a real zip file and add real file into it, and it just works. You can do that even in the lambda environment, by create filepath like "/tmp/yourfile.txt" you can create temp real file when lambda execute.

Python FTPLib Won't Upload Complete File

I am using the following code to upload a SQLITE3 Database file. For some reason, the script does not completely upload the file (the uploaded filesize is less than the original)
FTP = ftplib.FTP('HOST','USERNAME','PASSWORD')
FTP.cwd('/public_html/')
FILE = 'Database.db';
FTP.storbinary("STOR " + FILE, open(FILE, 'r'))
FTP.quit()
When I go to open the uploaded file in SQLite Browser, it says it is an invalid file.
What am I doing incorrectly?

In the open() call, you need to specify that the file is a binary file, like so:
FTP.storbinary("STOR " + FILE, open(FILE, 'rb'))

django open a zip file for user to download

Django 1.7, Python 3.4, windows apache 2.4.12 + wsgi
In my program, I generate some csv files and put them in a zip file. I would like to let the user download the zip file, either force download (after file is generated and render response) or a click of button (after display result, user has to click a button to download.)
Currently I am forcing them to download once the zip file is generated on the server.
I have referenced the following links and come up with my code below. But it is always give me this error "'charmap' codec can't decode byte 0x8d in position 80: character maps to " I tried to set it with utf-8 and ascii, and similar errors will be given.
referenced links:
django download file from server to user's machine,or read online
how to serve downloadable zip file in django
https://djangosnippets.org/snippets/365/
Generating file to download with Django
Anyone know why am I getting this error and how to get this to work?
Thank you very much!
zip_filename = time.strftime("%Y%m%d") + ".zip"
with zipfile.ZipFile(zip_filename, mode='a', compression=zipfile.ZIP_DEFLATED) as zf:
for s in sql_request:
// generate csv files
zf.write(csv_file)
// close zf
zip_file = open(zip_filename, 'r')
response = HttpResponse(zip_file, content_type='application/force-download')
response['Content-Disposition'] = 'attachment; filename="%s"' % 'foo.zip'
return response

If you're on Windows, you might need to change your open line to include the b flag.
zip_file = open(zip_filename, 'rb')