I need to create and iOS app that is basically a mobile GitHub repository browser. A user passes their input and they get the search results for the repositories.
I have tried approaching it from the basic http point of view, but to na avail (check out this question):
How to print my http query results in Xcode?
And now with support from #HedgeHog I have found a piece of code which does exactly what the backend to my app should do; but is written in Python:
import requests, sys, webbrowser, bs4
print('Your GitHub repository search query:')
userInput = input()
results = requests.get('https://github.com/search?q=' + userInput + '&type=repositories'
+ ' '.join(sys.argv[1:]))
results.raise_for_status()
soup = bs4.BeautifulSoup(results.text, 'html.parser')
linkList = ['https://github.com/'+a['href'] for a in soup.select('.repo-list-item .f4 a[href]')]
As I have also learned, PythonKit doesn't support iOS apps.
Is there any way that I could put a *.py file in the Xcode project and that Xcode could run it as a function (pass an input from a Swift to the Python file, then receive an output to the Swift file)?
Related
I am trying to export a tableau view as an image/csv (doesn't matter) using Python. I googled and found that REST API would help here, so I created a Personal Access Token and wrote the following command to connect: -
import tableauserverclient as TSC
from tableau_api_lib import TableauServerConnection
from tableau_api_lib.utils.querying import get_views_dataframe, get_view_data_dataframe
server_url = 'https://tableau.mariadb.com'
site = ''
mytoken_name = 'Marine'
mytoken_secret = '$32mcyTOkmjSFqKBeVKEZYpMUexseV197l2MuvRlwHghMacCOa'
server = TSC.Server(server_url, use_server_version=True)
tableau_auth = TSC.PersonalAccessTokenAuth(token_name=mytoken_name, personal_access_token=mytoken_secret, site_id=site)
with server.auth.sign_in_with_personal_access_token(tableau_auth):
print('[Logged in successfully to {}]'.format(server_url))
It entered successfully and gave the message: -
[Logged in successfully to https://tableau.mariadb.com]
However, Iam at a loss now on how to access the tableau workbooks using Python. I searched here:-
https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref_workbooks_and_views.htm
but was unable to write these commands like GET or others in python.
Can anyone help?
I'm assuming you don't know the view_id of the view you're looking for
Adding this after the print in the with block will query all the views available on your site;
all_views, pagination_item = server.views.get()
print([view.name for view in all_views])
Then find the view you're looking for in the printed output and note the view_id for use like this;
view_item = server.view.get_by_id('d79634e1-6063-4ec9-95ff-50acbf609ff5')
From there, you can get the image like this;
server.views.populate_image(view_item)
with open('./view_image.png', 'wb') as f:
f.write(view_item.image)
The tableauserverclient-python docs should help you out a ton as well
https://tableau.github.io/server-client-python/docs/api-ref#views
I started learning web scraping with Python. Currently, I would like to download a video of the Japanese Diet. (https://www.shugiintv.go.jp/jp/index.php?ex=VL&deli_id=40124&media_type=)
The video seems to have a mechanism to call chunklist.m3u8 from playlist.m3u8 and then call the ts files described in chunklist.m3u8 in order.
I want to download the contents from the playlist.m3u8 URL first, then call chunklist.m3u8 to download the ts files in order and concat.
However, I tried to download Playlist.m3u8, but it didn't produce the text I expected.
Also, the sample URL of playlist.m3u8 is here↓
http://hlsvod.shugiintv.go.jp/vod/_definst_/amlst:2011/2011-1207-0900-12/playlist.m3u8
code:
import requests
url = "http://hlsvod.shugiintv.go.jp/vod/_definst_/amlst:2011/2011-1207-0900-12/playlist.m3u8"
res = requests.get(url)
print(res.text)
excepted text:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=564000,NAME="500k",RESOLUTION=640x360
chunklist_w60346572_b564000_t64NTAwaw==.m3u8
actual text:
<html><head><title>Wowza Streaming Engine 4 Perpetual Bundle Unlimited Edition 4.7.7 build20181108145350</title></head><body>Wowza Streaming Engine 4 Perpetual Bundle Unlimited Edition 4.7.7 build20181108145350</body></html>
I think there is a problem with the colon in the URL, but I don't have a clear solution. I would like to know how to avoid URL issues and successfully download the text in playlist.m3u8. Thanks.
Version:
Python 3.7.9
requests 2.25.1
Something is wrong with your url:
>>> url = "http://hlsvod.shugiintv.go.jp/vod/_definst_/amlst:2011/2011-1207-0900-12/playlist.m3u8"
>>> res = requests.get(url)
>>> res.request.url
'https://hlsvod.shugiintv.go.jp/vod/_definst_/amlst:2011/2011-1207-0900-12/playlist.m3u8%20'
See the "%20" in the end?
I am not really sure how you got it wrong, but copy-paste this should work:
url = 'https://hlsvod.shugiintv.go.jp/vod/_definst_/amlst:2011/2011-1207-0900-12/playlist.m3u8'
I am trying to use gdata within a Django app to create a directory in my google drive account. This is the code written within my Django view:
def root(request):
from req_info import email, password
from gdata.docs.service import DocsService
print "Creating folder........"
folder_name = '2015-Q1'
service_client = DocsService(source='spreadsheet create')
service_client.ClientLogin(email, password)
folder = service_client.CreateFolder(folder_name)
Authentication occurs without issue, but that last line of code triggers the following error:
Request Method: GET
Request URL: http://127.0.0.1:8000/
Django Version: 1.7.7
Exception Type: RequestError
Exception Value: {'status': 501, 'body': 'POST method does not support concurrency', 'reason': 'Not Implemented'}
I am using the following software:
Python 2.7.8
Django 1.7.7
PyCharm 4.0.5
gdata 2.0.18
google-api-python-client 1.4.0 (not sure if relevant)
[many other packages that I'm not sure are relevant]
What's frustrating is that the exact same code (see below) functions perfectly when I run it in its own, standalone file (not within a Django view).
from req_info import email, password
from gdata.docs.service import DocsService
print "Creating folder........"
folder_name = '2015-Q1'
service_client = DocsService(source='spreadsheet create')
service_client.ClientLogin(email, password)
folder = service_client.CreateFolder(folder_name)
I run this working code in the same virtual environment and the same PyCharm project as the code that produced the error. I have tried putting the code within a function in a separate file, and then having the Django view call that function, but the error persists.
I would like to get this code working within my Django app.
I don't recall if I got this to work within a Django view, but because Google has since required the use of Oauth 2.0, I had to rework this code anyways. I think the error had something to do with my simultaneous use of two different packages/clients to access Google Drive.
Here is how I ended up creating the folder using the google-api-python-client package:
from google_api import get_drive_service_obj, get_file_key_if_exists, insert_folder
def create_ss():
drive_client, credentials = get_drive_service_obj()
# creating folder if it does not exist
folder = get_file_key_if_exists(drive_client, 'foldername')
if folder: # if folder exists
print 'Folder "' + folder_name + '" already exists.'
else: # if folder doesn't exist
print 'Creating folder........"' + folder_name + '".'
folder = insert_folder(drive_client, folder_name)
After this code, I used a forked version (currently beta) of sheetsync to copy my template spreadsheet and populate the new file with my data. I then had to import sheetsync after the code above to avoid the "concurrency" error. (I could post the code involving sheetsync here too if folks want, but for now, I don't want to get too far off topic.)
I have checked Google Search API's and it seems that they have not released any API for searching "Images". So, I was wondering if there exists a python script/library through which I can automate the "search by image feature".
This was annoying enough to figure out that I thought I'd throw a comment on the first python-related stackoverflow result for "script google image search". The most annoying part of all this is setting up your proper application and custom search engine (CSE) in Google's web UI, but once you have your api key and CSE, define them in your environment and do something like:
#!/usr/bin/env python
# save top 10 google image search results to current directory
# https://developers.google.com/custom-search/json-api/v1/using_rest
import requests
import os
import sys
import re
import shutil
url = 'https://www.googleapis.com/customsearch/v1?key={}&cx={}&searchType=image&q={}'
apiKey = os.environ['GOOGLE_IMAGE_APIKEY']
cx = os.environ['GOOGLE_CSE_ID']
q = sys.argv[1]
i = 1
for result in requests.get(url.format(apiKey, cx, q)).json()['items']:
link = result['link']
image = requests.get(link, stream=True)
if image.status_code == 200:
m = re.search(r'[^\.]+$', link)
filename = './{}-{}.{}'.format(q, i, m.group())
with open(filename, 'wb') as f:
image.raw.decode_content = True
shutil.copyfileobj(image.raw, f)
i += 1
There is no API available but you are can parse the page and imitate the browser, but I don't know how much data you need to parse because google may limit or block access.
You can imitate the browser by simply using urllib and setting correct headers, but if you think parsing complex web-pages may be difficult from python, you can directly use a headless browser like phontomjs, inside a browser it is trivial to get correct elements using javascript/DOM
Note before trying all this check google's TOS
You can try this:
https://developers.google.com/image-search/v1/jsondevguide#json_snippets_python
It's deprecated, but seems to work.
The JSON syntax definition say that
html/xml tags (like the <script>...</script> part) are not part of
valid json, see the description at http://json.org.
A number of browsers and tools ignore these things silently, but python does
not.
I'd like to insert the javascript code (google analytics) to get info about the users using this service (place, browsers, OS ...).
What do you suggest to do?
I should solve the problem on [browser output][^1] or [python script][^2]?
thanks,
Antonio
[^1]: Browser output
<script>...</script>
[{"key": "value"}]
[^2]: python script
#!/usr/bin/env python
import urllib2, urllib, json
url="http://.........."
params = {}
url = url + '?' + urllib.urlencode(params, doseq=True)
req = urllib2.Request(url)
headers = {'Accept':'application/json;text/json'}
for key, val in headers.items():
req.add_header(key, val)
data = urllib2.urlopen(req)
print json.load(data)
These sound like two different kinds of services--one is a user-oriented web view of some data, with visualizations, formatting, etc., and one is a machine-oriented data service. I would keep these separate, and maybe build the user view as an extension to the data service.