We Keep Coding

Downloading encrpyted/compressed 7z file from slack url

Having an issue trying to download a encrypted/compressed 7zip file. I see the binary data being returned but still getting an error that the file is not downloaded. The information is being grabbed from a slack api by the 'url_private' property. The authentication is good, I see the byte data in my logger but still having issues.
import requests
import py7zr
url= "https://someurl/test_file.7z"
headers = {"auth": "token"}
response = requests.request("GET", url=file_url, headers=headers)
file_name = file_url.split("/")[-1]
if response.status_code == 200:
with py7zr.SevenZipFile(file_name, mode='r', password=pw) as file:
file.extractall(f"templates/files/")
# logger.debug(file_data)
file.close()
Error:
Failed to run listener function (error: [Errno 2] No such file or directory: 'test_file.7z')
Traceback (most recent call last):
File "/opt/venv/lib/python3.8/site-packages/slack_bolt/listener/thread_runner.py", line 103, in run_ack_function_asynchronously
listener.run_ack_function(request=request, response=response)
File "/opt/venv/lib/python3.8/site-packages/slack_bolt/listener/custom_listener.py", line 49, in run_ack_function
return self.ack_function(
File "/app/app/routes/events.py", line 62, in handle_file_shared
file = get_file_shared(file_url)
File "/app/app/lib/file_shared.py", line 28, in get_file_shared
with py7zr.SevenZipFile(file_name, mode='r', password=pw) as file:
File "/opt/venv/lib/python3.8/site-packages/py7zr/py7zr.py", line 324, in __init__
self.fp = open(file, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'test_file.7z'
I can't seem to find a solution, any help is appreciated.

You had most of it right. One of your problems might be that you didn't import the packages like requests. I found it by a simple google search.
import os
import requests
import py7zr
URL = "https://dl-alt1.winworldpc.com/Microsoft%20Windows%201.01%20Demo%20Slideshow%20(5.25).7z"
filename = os.path.basename(URL)
response = requests.get(URL, stream=True)
if response.status_code == 200:
with open(filename, 'wb') as out:
out.write(response.content)
with py7zr.SevenZipFile(filename, 'r') as archive:
archive.extractall(f"templates/files/")
else:
print('Request failed: %d' % response.status_code)

PermissionError: [Errno 13] Permission denied - Spotipy

I'm trying to call the Spotify API using the Spotipy Python library. I'm getting a 'PermissionError: [Errno 13] Permission denied' error.
I'm using a GCP Compute Engine with Debian as the OS.
Error is below
Traceback (most recent call last):<br>
File "<stdin>", line 1, in <module><br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/client.py", line 1307, in current_use
r_recently_played<br>
before=before,<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/client.py", line 291, in _get<br>
return self._internal_call("GET", url, payload, kwargs)<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/client.py", line 221, in _internal_ca
ll<br>
headers = self._auth_headers()<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/client.py", line 212, in _auth_header
s<br>
token = self.auth_manager.get_access_token(as_dict=False)<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/oauth2.py", line 484, in get_access_t
oken<br>
"code": code or self.get_auth_response(),<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/oauth2.py", line 439, in get_auth_res
ponse<br>
return self._get_auth_response_local_server(redirect_port)<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/oauth2.py", line 405, in _get_auth_re
sponse_local_server<br>
server = start_local_http_server(redirect_port)<br>
File "/home/#####/.local/lib/python3.7/site-packages/spotipy/oauth2.py", line 1227, in start_local
_http_server<br>
server = HTTPServer(("127.0.0.1", port), handler)<br>
File "/usr/lib/python3.7/socketserver.py", line 452, in __init__<br>
self.server_bind()<br>
File "/usr/lib/python3.7/http/server.py", line 137, in server_bind<br>
socketserver.TCPServer.server_bind(self)<br>
File "/usr/lib/python3.7/socketserver.py", line 466, in server_bind<br>
self.socket.bind(self.server_address)<br>
PermissionError: [Errno 13] Permission denied<br>
Code example is
import spotipy
from spotipy.oauth2 import SpotifyOAuth
scope = 'user-read-recently-played'
sp = spotipy.Spotify(auth_manager=SpotifyOAuth(scope=scope))
played = sp.current_user_recently_played()
I have done chmod 0775 -R on what I believe are the relevant directories but still getting stuck.

I would suggest trying using a different approach. Here's what I use for my Spotify pipeline:
import spotipy
import spotipy.util as util
import os
un = os.environ['un']
scope = 'user-read-recently-played'
cid = os.environ['cid']
csid = os.environ['csid']
redr = r'http://localhost:8888/callback/'
token = util.prompt_for_user_token(un,scope,cid,csid,redr)
sp = spotipy.Spotify(auth=token)
results = sp.current_user_recently_played()
Where un is my username, cid and csid are the key/secret key for my developer account, and redr is just a local webpage for redirects.

Connecting to a public FTP behind a corporate proxy

I'm trying to access a public FTP in my work, but I'm getting an error.
That's is the code that I used in my house.
from ftplib import FTP
ftp = FTP('ftp.cetip.com.br')
ftp.login()
ftp.cwd('/MediaCDI')
ftp.quit()
It's work fine in my home, but in my work I get this error:
Traceback (most recent call last):
File "C:\Users\TBMEPYG\Desktop\stack.py", line 3, in <module>
ftp = FTP('ftp.cetip.com.br')
File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\ftplib.py", line 117, in __init__
self.connect(host)
File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\ftplib.py", line 152, in connect
source_address=self.source_address)
File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\socket.py", line 722, in create_connection
raise err
File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\socket.py", line 713, in create_connection
sock.connect(sa)
[Finished in 21.3s]
When I was developing a scraper in a HTTP website, I solved this problem for http using HTTPProxyAuth and requests. Just to illustrate, that's was the code:
from requests.auth import HTTPProxyAuth
import requests
user = 'xxxx'
password = 'yyyy'
credenciais = HTTPProxyAuth(user, password)
params = {'Dt_Ref': data, 'TpInstFinanceiro': 'CRI', 'Tipo':'1','saida':'txt'}
proxy_access = {'http':'proxy.mywork/accelerated_pac_base.pac'}
url = 'http://www.anbima.com.br/reune/reune_down.asp'
r = requests.post(url, data = params, proxies = proxy_access , auth = credenciais)
Anyone has any idea about what can I do?
Thanks

GCS boto authentication on RPi

I have this simple script that spits out the buckets in my GCS:
import boto
import gcs_oauth2_boto_plugin
import os
import shutil
import StringIO
import tempfile
import time
# URI scheme for Cloud Storage.
GOOGLE_STORAGE = 'gs'
# URI scheme for accessing local files.
LOCAL_FILE = 'file'
header_values = {"x-goog-project-id": "xxxxxxxxxxxx"}
uri = boto.storage_uri('', GOOGLE_STORAGE)
for bucket in uri.get_all_buckets(headers=header_values):
print bucket.name
The top of my ~/.boto file has the following (with real values for everything inside brackets):
# Google OAuth2 service account credentials (for "gs://" URIs):
gs_service_key_file = /home/pi/dev/camera/cl-camera-<id>.json
gs_service_client_id = '<user>#<id>.iam.gserviceaccount.com'
Everything works fine when running without sudo, but once I add sudo (I need access to GPIO pins since this is on a RPi), I get the following error:
Traceback (most recent call last):
File "gcs-test.py", line 24, in <module>
for bucket in uri.get_all_buckets(headers=header_values):
File "/usr/local/lib/python2.7/dist-packages/boto/storage_uri.py", line 584, in get_all_buckets
conn = self.connect()
File "/usr/local/lib/python2.7/dist-packages/boto/storage_uri.py", line 140, in connect
**connection_args)
File "/usr/local/lib/python2.7/dist-packages/boto/gs/connection.py", line 47, in __init__
suppress_consec_slashes=suppress_consec_slashes)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 191, in __init__
validate_certs=validate_certs, profile_name=profile_name)
File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 569, in __init__
host, config, self.provider, self._required_auth_capability())
File "/usr/local/lib/python2.7/dist-packages/boto/auth.py", line 1021, in get_auth_handler
'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 3 handlers were checked. ['OAuth2Auth', 'OAuth2ServiceAccountAuth', 'HmacAuthV1Handler'] Check your credentials
Any ideas as to what's happening and why it's only when I run it with sudo?

I figure this one out. Since I'm running it as root now, it looks for the .boto file in a different place (/root/.boto instead of /home/pi/.boto), so I did the following to create a new config file and it worked:
$ sudo su
$ gsutil config -e

unable to dowload csv file from FTP server in app engine

Im trying to read CSV files from ftp server in AppEngine, and I am able to connect to the ftp server. But it returned error when I tried to retrieve files.
Here is my code to read the CSV files from server:
import ftplib
import cStringIO
import csv
session = ftplib.FTP('myftpserver.com')
session.login('username','pwd')
session.set_pasv(False)
output = cStringIO.StringIO()
session.retrbinary('RETR myfile.csv', output.write)
csvfile = csv.reader(output.getvalue().splitlines(), delimiter=',')
for i, row in enumerate(csvfile):
print row
And here is the traceback of the error I am getting:
Traceback (most recent call last):
File "/home/vikas/apps/myproj/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/vikas/apps/myproj/admin/admin_actions.py", line 3528, in get_ftp_file
session.retrbinary('RETR myfile.csv', output.write)
File "/usr/lib/python2.7/ftplib.py", line 414, in retrbinary
conn = self.transfercmd(cmd, rest)
File "/usr/lib/python2.7/ftplib.py", line 376, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/usr/lib/python2.7/ftplib.py", line 354, in ntransfercmd
sock = self.makeport()
File "/usr/lib/python2.7/ftplib.py", line 283, in makeport
for res in socket.getaddrinfo(None, 0, self.af, socket.SOCK_STREAM, 0, socket.AI_PASSIVE):
File "/home/vikas/gcloud/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 318, in getaddrinfo
raise gaierror(EAI_NONAME, 'nodename nor servname provided, or not known')
gaierror: [Errno 8] nodename nor servname provided, or not known
I have no idea what I have done wrong, none of the commands like dir() nlst() etc is working, and the above error occurred as soon as I added them .

Alas, the socket simulation currently offered by App Engine isn't quite good enough to cover all use cases.
Here's a sample accessing a well-known public anonymous FTP server and fetching a small text file from it, just so everybody can reproduce and experiment...: in file getit.py, we have:
import ftplib
import cStringIO
def getit():
session = ftplib.FTP('ftp.mozilla.org')
session.login('anonymous','')
# session.set_pasv(False)
session.cwd('/pub/mozilla.org')
output = cStringIO.StringIO()
session.retrbinary('RETR README', output.write)
return output.getvalue()
if __name__ == '__main__':
print(getit())
This runs fine as stand-alone, python getit.py, whether you leave the set_pasv commented as here, or remove the comment.
To embed this in a GAE app, e.g:
import getit
class GetitPage(webapp2.RequestHandler):
def get(self): # pylint:disable-msg=invalid-name
try: result = getit.getit()
except Exception as e:
result = 'Error {}: {}'.format(type(e), e)
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write(result)
this works fine with the set_pasv left commented, but if you de-comment it, you get essentially the same exception you received.
So doing FTP to a server that forces you to active mode (doesn't support passive mode) is not going to work this way. However, that is a rather broken server -- could you get it fixed so that it supports the popular, default passive mode? Then your GAE app could work happily with it...!