internet archive ( archive.org ) python package proxy / vpn issue - python

From behind my vpn, http proxy: 127.0.0.1:8118, I'm just trying to get the example from the quickstart of the ia package going as from the website:
https://archive.org/developers/tutorial-find-identifier-item.html
https://github.com/jjjake/internetarchive/tree/master/internetarchive
Standalone, this always works fine, ( so I believe the issue is in the package and not my computer ):
requests.get('http://google.com', proxies={'https'=127.0.0.1:8118'})
//ia uses HTTPS
I've tried setting the environment variable HTTPS_PROXY:
HTTPS_PROXY='http://127.0.0.1:8118'
I'm just trying to get the first working example from the website going (from behind a vpn), very simple:
from internetarchive import get_item
md = {'collection': 'test_collection', 'title': 'My New Item', 'mediatype': 'movies'}
r = item.upload('<identifier>', files=['film.txt', 'film.mov'], metadata=md, access_key='YoUrAcCEssKey', secret_key='youRSECRETKEY')
r[0].status_code
I've been digging around the ia package for places to enter the proxy variable in the requests.get(..., proxies={'http':127.0.0.1:8118' }) and running test with no luck...
I'm still having no luck here. Are there any suggestions?

Related

Push notification from python to android

I'm running some scripts and looking for an easy way to ping my phone once the script has finished running.
Doing some research on the web, I've seen ways of sending messages using Slack, Push bullet, twilio, email etc.
I am looking for recommendations for an easy way to send a ping/message from python to my phone.
Easy in the sense it dose not require considerable configuring of outside accounts or pay services.
I have found a much easier way, but it doesn't works on Linux. Here is a link for more details.
First you have to install notify_run:
pip install notify_run
Then you have to register:
notify-run register
It will give you a QR code (on windows the QR code doesn't works) and a link, which will take you to a website, there press the "Subscribe on this device" (Maybe you will have to refresh the site)
Then use this code:
from notify_run import Notify
notify = Notify()
notify.send('any message you want')
U can try cmd: pip install telegram-send and just send a message to your Telegram bot.
Create your telegram bot at BotFather, take a token from there, paste it to
cmd: telegram-send --configure
Usage:
import telegram_send
telegram_send.send(messages=["Hello world"])
I found this much better than any other push notification.
For more info:
Link1
Link2
I've tried a Twilio, but it's complicated and I don't think it can send you messages for free (anymore). Telegram seems an easy solution. To extend to the answer of #Nick PV, here are the steps I, as a Telegram beginner, took:
1) Get a Telegram account (free) using your phone number
Web: https://web.telegram.org
Also download the Telegram Andriod. Of course, you want the notifications on your phone after all
2) Go into settings (web or app) and set a username
This is needed to obtain an id which your bot will use to send messages to
3) Send a message to RawDataBot to get your id
Just search for RawDataBot and send any message (hi will do). Take a note of your id.
4) Create your bot (which you'll command with HTTP requests)
Now search for BotFather and send the message /start. Help is displayed. Send the message /newbot and follow the instructions. Take a note of your token to access the HTTP API
5) Send the API request using Python
You could install and use telegram-send, but if you are like me and you prefer the generic library requests which will give you the experience to handle any HTTP API, this is how to do it:
import requests
token = "token_from_step_4"
url = f"https://api.telegram.org/bot{token}"
params = {"chat_id": "id_from_step_3", "text": "Hello World"}
r = requests.get(url + "/sendMessage", params=params)
Links:
Telegram bots: https://core.telegram.org/bots
API Docs sendMessage https://core.telegram.org/bots/api#sendmessage
In the end, the easiest way I found was using Slack. It takes one Python function (about 6 lines) and an Slack account.
More details can be found here on medium
I tried pushsafer and notify-run without success on my computer (Linux Mint 19.3 tricia)
On my computer, only Slack works properly. This post comes from this Youtube video where you will see the full process in Video.
What you need first is Slack application on your device (iOs, Android, ...) and create an Slack account (if not done)
Tuto :
Create a workspace and an application (see details in this video. Sorry, I did not put the substeps here but I put this answer as a Community wiki in order to let you finish what I started :) )
Install the python library pip3 install sandesh
pip3 install sandesh
Get your webhook => https://hooks.slack.com/services/blablabla
Use this sample to send a message to Slack :
python code:
import sandesh
sandesh.send("This is my test", webhook="<put here the https link of your webhook>")
You can find the open source sandesh python module here in gitHub
You already mentioned it in your answer but Pushbullet is pretty easy:
Create a pushbullet account on their website using a gmail (I recommend not using your main one).
Install the android app and login with the same gmail. (Despite what their instructions say, you do not need to give any permissions to the Android app. Not even notifcations.)
From the website go to settings -> Account -> Create Access Token. Copy this token to use in your python script.
From python (yes you need the "py" at the end of the name)
pip install pushbullet.py (see project on pypi)
import pushbullet
pb = pushbullet.PushBullet(YOUR_ACCESS_TOKEN)
push = pb.push_note('Some Title', 'Some message you want to send')
This will pop up a notification on your Android phone with the message you specified. Works from Windows or Linux.
Using firebase fcm
One of the easiest and reliable way is to use Firebase Cloud Messaging for sending push notifications.
You can get server key from the firebase console after creating your project.
You can get device token by generating it in your device
see: https://firebase.google.com/docs/cloud-messaging/android/client (iOS, android and web)
## Install request module by running ->
# pip3 install requests
# Replace the deviceToken key with the device Token for which you want to send push notification.
# Replace serverToken key with your serverKey from Firebase Console
# Run script by ->
# python3 fcm_python.py
import requests
import json
serverToken = 'your server key here'
deviceToken = 'device token here'
headers = {
'Content-Type': 'application/json',
'Authorization': 'key=' + serverToken,
}
body = {
'notification': {'title': 'Sending push form python script',
'body': 'New Message'
},
'to':
deviceToken,
'priority': 'high',
# 'data': dataPayLoad,
}
response = requests.post("https://fcm.googleapis.com/fcm/send",headers = headers, data=json.dumps(body))
print(response.status_code)
print(response.json())
Hope you found helpful

Login to registry with Docker Python SDK (docker-py)

I am trying to use the Docker Python API to login to a Docker cloud:
https://docker-py.readthedocs.io/en/stable/client.html#creating-a-client1
What is the URL? What is the Port?
I have tried to get it to work with cloud.docker.com, but I am fine with any registry server, so long as it is free to use and I can use it to upload Docker images from one computer and run them on another.
I have already got everything running using my own locally hosted registry, but I can’t seem to figure out how to connect to a server. It’s kind of ridiculous that hosting my own registry is easier than using an existing registry server.
My code looks like this, but I am unsure what the args.* parameters should be:
client = docker.DockerClient(base_url=args.docker_registry)
client.login(username=args.docker_user, password=args.docker_password)
I’m not sure what the base_url needs to be so that I can log in, and the error messages are not helpful at all.
Can you give me an example that works?
The base_url parameter is the URL of the Docker server, not the Docker Registry.
Try something like:
from docker.errors import APIError, TLSParameterError
try:
client = docker.from_env()
client.login(username=args.docker_user, password=args.docker_password, registry=args.docker_registry)
except (APIError, TLSParameterError) as err:
# ...
Here's how I have logged in to Docker using Python:
import docker
client = docker.from_env()
client.login(username='USERNAME', password='PASSWORD', email='EMAIL',
registry='https://index.docker.io/v1/')
and here's what it returns:
{'IdentityToken': '', 'Status': 'Login Succeeded'}
So, that means it has been logged in successfully.
I still haven't figured out what the registry of cloud.docker.com is called, but I got it to work by switching to quay.io as my registry server, which works with the intuitive registry='quay.io'

TestLink xmlrpc API (via Python) 404 not found

I'm trying to connect to TestLink via the xmlrpc API. I've set the following in TestLink's config.inc.php:
$tlCfg->api->enabled = TRUE;
$tlCfg->exec_cfg->enabled_test_automation = ENABLED;
and restarted the apache sever. I tried to connect the TestLink server via the python package TestLink-API-Python-client (https://github.com/orenault/TestLink-API-Python-client)
from testlink import TestlinkAPIClient, TestLinkHelper
import sys
URL = 'http://MYSERVER/testlink/lib/api/xmlrpc.php'
DevKey = 'MYKEY'
tl_helper = TestLinkHelper()
myTestLink = tl_helper.connect(TestlinkAPIClient)
myTestLink.__init__(URL, DEVKEY)
myTestLink.checkDevKey()
And then I receive a TLConnectionError, stating my url, and 404 Not Found...
Does anyone have any idea?
Thanks.
I didn't solve it.
I reverted to working on the TestLink DB directly. I'm sure it's more fragile than using the API, but it works...
If you are still looking for help, this code worked for me:
set TESTLINK_API_PYTHON_SERVER_URL=http://[YOURSERVER]/testlink/lib/api/xmlrpc/v1/xmlrpc.php
set TESTLINK_API_PYTHON_DEVKEY=[Users devKey generated by TestLink]
python
import testlink
tls = testlink.TestLinkHelper().connect(testlink.TestlinkAPIClient)
tls.countProjects()
Check out TestLink API Documentation to learn more
In a glance your XML-RPC URL seems wrong. It should be
http://YOURSERVER/testlink/lib/api/xmlrpc/v1/xmlrpc.php

django views urllib2.py https error twilio api

I'm looking to send an SMS with the Twilio api, but I'm getting the following error:
"unknown url type: https"
I've recompiled python with Openssl, so my code runs fine from the python interpretor, but whenever I try to run it in one of my django views I get this error. Here is my code from my view:
def send_sms(request):
recipient = '1234567890'
account = twilio.Account(settings.TWILIO_ID, settings.TWILIO_TOKEN)
params = { 'From': settings.TWILIO_NUM, 'To': recipient, 'Body': 'This is a test message.', }
account.request('/%s/Accounts/%s/SMS/Messages' % (settings.TWILIO_API_VERSION, settings.TWILIO_ID), 'POST', params)
Edit- More info (thanks for bringing that up Stefan)
The project is hosted on dreamhost via Passenger wsgi. Django is using the same python install location and interp.
I appreciate any insight anyone may have, thanks!
Looks like it was just user error. My wsgi file was using a different interpreter but the paths were so similar I was just over looking it. Once I fixed that django was using the python version that I compiled with openssl and everything worked fine.
Always check if the tv is plugged in before you take it apart. Thanks stefanw!

Does urllib2 in Python 2.6.1 support proxy via https

Does urllib2 in Python 2.6.1 support proxy via https?
I've found the following at http://www.voidspace.org.uk/python/articles/urllib2.shtml:
NOTE
Currently urllib2 does not support
fetching of https locations through a
proxy. This can be a problem.
I'm trying automate login in to web site and downloading document, I have valid username/password.
proxy_info = {
'host':"axxx", # commented out the real data
'port':"1234" # commented out the real data
}
proxy_handler = urllib2.ProxyHandler(
{"http" : "http://%(host)s:%(port)s" % proxy_info})
opener = urllib2.build_opener(proxy_handler,
urllib2.HTTPHandler(debuglevel=1),urllib2.HTTPCookieProcessor())
urllib2.install_opener(opener)
fullurl = 'https://correct.url.to.login.page.com/user=a&pswd=b' # example
req1 = urllib2.Request(url=fullurl, headers=headers)
response = urllib2.urlopen(req1)
I've had it working for similar pages but not using HTTPS and I suspect it does not get through proxy - it just gets stuck in the same way as when I did not specify proxy. I need to go out through proxy.
I need to authenticate but not using basic authentication, will urllib2 figure out authentication when going via https site (I supply username/password to site via url)?
EDIT:
Nope, I tested with
proxies = {
"http" : "http://%(host)s:%(port)s" % proxy_info,
"https" : "https://%(host)s:%(port)s" % proxy_info
}
proxy_handler = urllib2.ProxyHandler(proxies)
And I get error:
urllib2.URLError: urlopen error
[Errno 8] _ssl.c:480: EOF occurred in
violation of protocol
Fixed in Python 2.6.3 and several other branches:
_bugs.python.org/issue1424152 (replace _ with http...)
http://www.python.org/download/releases/2.6.3/NEWS.txt
Issue #1424152: Fix for httplib, urllib2 to support SSL while working through
proxy. Original patch by Christopher Li, changes made by Senthil Kumaran.
I'm not sure Michael Foord's article, that you quote, is updated to Python 2.6.1 -- why not give it a try? Instead of telling ProxyHandler that the proxy is only good for http, as you're doing now, register it for https, too (of course you should format it into a variable just once before you call ProxyHandler and just repeatedly use that variable in the dict): that may or may not work, but, you're not even trying, and that's sure not to work!-)
Incase anyone else have this issue in the future I'd like to point out that it does support https proxying now, make sure the proxy supports it too or you risk running into a bug that puts the python library into an infinite loop (this happened to me).
See the unittest in the python source that is testing https proxying support for further information:
http://svn.python.org/view/python/branches/release26-maint/Lib/test/test_urllib2.py?r1=74203&r2=74202&pathrev=74203

Categories