I'm making an API REST request using Python. And I encountered the following html result that says "Service - Endpoint not found. Please see the service help page for constructing valid requests to the service"
How can I fix this issue?
Note: This API can help determine whether an individual address is up to date by inputting individual address, first name, last name, etc.
Python query
import requests
import json
url = 'https://smartmover.melissadata.net/v3/WEB/SmartMover/doSmartMover/'
payload = {'t': '1353', 'id': 'sw38hs47u', 'jobid': '1', 'act': 'NCOA, CCOA', 'cols': 'TransmissionResults,TransmissionReference, Version, TotalRecords,CASSReportLink,NCOAReportLink,Records,AddressExtras,AddressKey,AddressLine1,AddressLine2,AddressTypeCode,BaseMelissaAddressKey,CarrierRoute,City,CityAbbreviation,CompanyName,CountryCode,CountryName,DeliveryIndicator,DeliveryPointCheckDigit,DeliveryPointCode,MelissaAddressKey,MoveEffectiveDate,MoveTypeCode,PostalCode,RecordID,Results,State,StateName,Urbanization', 'opt': 'ProcessingType: Standard', 'List': 'test', 'full': 'PATEL MANISH', 'first':'MANISH','last':'PATEL', 'a1':'1600 S 5TH ST', 'a2':'1600 S 5TH ST', 'city':'Austin', 'state': 'TX', 'postal': '78704', 'ctry': 'USA'}
response = requests.get(url, params=payload)
print (response.text)
Result
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Service</title>
<style>BODY { color: #000000; background-color: white; font-family: Verdana; margin-left: 0px; margin-top: 0px; } #content { margin-left: 30px; font-size: .70em; padding-bottom: 2em; } A:link { color: #336699; font-weight: bold; text-decoration: underline; } A:visited { color: #6699cc; font-weight: bold; text-decoration: underline; } A:active { color: #336699; font-weight: bold; text-decoration: underline; } .heading1 { background-color: #003366; border-bottom: #336699 6px solid; color: #ffffff; font-family: Tahoma; font-size: 26px; font-weight: normal;margin: 0em 0em 10px -20px; padding-bottom: 8px; padding-left: 30px;padding-top: 16px;} pre { font-size:small; background-color: #e5e5cc; padding: 5px; font-family: Courier New; margin-top: 0px; border: 1px #f0f0e0 solid; white-space: pre-wrap; white-space: -pre-wrap; word-wrap: break-word; } table { border-collapse: collapse; border-spacing: 0px; font-family: Verdana;} table th { border-right: 2px white solid; border-bottom: 2px white solid; font-weight: bold; background-color: #cecf9c;} table td { border-right: 2px white solid; border-bottom: 2px white solid; background-color: #e5e5cc;}</style>
</head>
<body>
<div id="content">
<p class="heading1">Service</p>
<p xmlns="">Endpoint not found. Please see the <a rel="help-page" href="https://smartmover.melissadata.net/v3/WEB/SmartMover/help">service help page</a> for constructing valid requests to the service.</p>
</div>
</body>
</html>
[Finished in 0.9s]
Remove the / at the end of the url:
import requests
import json
url = 'https://smartmover.melissadata.net/v3/WEB/SmartMover/doSmartMover' # <<<
payload = {'t': '1353', 'id': 'sw38hs47u', 'jobid': '1', ...}
response = requests.get(
url, params=payload,
headers={'Content-Type': 'application/json'} # Using JSON here for readability in the response
)
print (response.text)
Output:
{
"CASSReportLink": "",
"NCOAReportLink": "",
"Records": [],
"TotalRecords": "",
"TransmissionReference": "1353",
"TransmissionResults": "SE20",
"Version": "4.0.4.48"
}
Related
Lets say there's a file that lives at the github repo:
https://github.com/someguy/brilliant/blob/master/somefile.txt
I'm trying to use requests to request this file, write the content of it to disk in the current working directory where it can be used later. Right now, I'm using the following code:
import requests
from os import getcwd
url = "https://github.com/someguy/brilliant/blob/master/somefile.txt"
directory = getcwd()
filename = directory + 'somefile.txt'
r = requests.get(url)
f = open(filename,'w')
f.write(r.content)
Undoubtedly ugly, and more importantly, not working. Instead of the expected text, I get:
<!DOCTYPE html>
<!--
Hello future GitHubber! I bet you're here to remove those nasty inline styles,
DRY up these templates and make 'em nice and re-usable, right?
Please, don't. https://github.com/styleguide/templates/2.0
-->
<html>
<head>
<meta http-equiv="Content-type" content="text/html; charset=utf-8">
<title>Page not found · GitHub</title>
<style type="text/css" media="screen">
body {
background: #f1f1f1;
font-family: "HelveticaNeue", Helvetica, Arial, sans-serif;
text-rendering: optimizeLegibility;
margin: 0; }
.container { margin: 50px auto 40px auto; width: 600px; text-align: center; }
a { color: #4183c4; text-decoration: none; }
a:visited { color: #4183c4 }
a:hover { text-decoration: none; }
h1 { letter-spacing: -1px; line-height: 60px; font-size: 60px; font-weight: 100; margin: 0px; text-shadow: 0 1px 0 #fff; }
p { color: rgba(0, 0, 0, 0.5); margin: 20px 0 40px; }
ul { list-style: none; margin: 25px 0; padding: 0; }
li { display: table-cell; font-weight: bold; width: 1%; }
#error-suggestions { font-size: 14px; }
#next-steps { margin: 25px 0 50px 0;}
#next-steps li { display: block; width: 100%; text-align: center; padding: 5px 0; font-weight: normal; color: rgba(0, 0, 0, 0.5); }
#next-steps a { font-weight: bold; }
.divider { border-top: 1px solid #d5d5d5; border-bottom: 1px solid #fafafa;}
#parallax_wrapper {
position: relative;
z-index: 0;
}
#parallax_field {
overflow: hidden;
position: absolute;
left: 0;
top: 0;
height: 370px;
width: 100%;
}
etc etc.
Content from Github, but not the content of the file. What am I doing wrong?
The content of the file in question is included in the returned data. You are getting the full GitHub view of that file, not just the contents.
If you want to download just the file, you need to use the Raw link at the top of the page, which will be (for your example):
https://raw.githubusercontent.com/someguy/brilliant/master/somefile.txt
Note the change in domain name, and the blob/ part of the path is gone.
To demonstrate this with the requests GitHub repository itself:
>>> import requests
>>> r = requests.get('https://github.com/kennethreitz/requests/blob/master/README.rst')
>>> 'Requests:' in r.text
True
>>> r.headers['Content-Type']
'text/html; charset=utf-8'
>>> r = requests.get('https://raw.githubusercontent.com/kennethreitz/requests/master/README.rst')
>>> 'Requests:' in r.text
True
>>> r.headers['Content-Type']
'text/plain; charset=utf-8'
>>> print r.text
Requests: HTTP for Humans
=========================
.. image:: https://travis-ci.org/kennethreitz/requests.png?branch=master
[... etc. ...]
You need to request the raw version of the file, from https://raw.githubusercontent.com.
See the difference:
https://raw.githubusercontent.com/django/django/master/setup.py vs. https://github.com/django/django/blob/master/setup.py
Also, you should probably add a / between your directory and the filename:
>>> getcwd()+'foo.txt'
'/Users/burhanfoo.txt'
>>> import os
>>> os.path.join(getcwd(),'foo.txt')
'/Users/burhan/foo.txt'
Just as an update, https://raw.github.com was migrated to https://raw.githubusercontent.com. So the general format is:
url = "https://raw.githubusercontent.com/user/repo/branch/[subfolders]/file"
E.g. https://raw.githubusercontent.com/earnestt1234/seedir/master/setup.py. Still use requests.get(url) as in Martijn's answer.
Adding a working example ready for copy+paste:
import requests
from requests.structures import CaseInsensitiveDict
url = "https://raw.githubusercontent.com/organization/repo/branch/folder/file"
# If repo is private - we need to add a token in header:
headers = CaseInsensitiveDict()
headers["Authorization"] = "token TOKEN"
resp = requests.get(url, headers=headers)
print(resp.status_code)
(*) If repo is not private - remove the headers part.
Bonus:
Check out this Curl < --> Python-requests online converter.
I am thinking to create a PyQt6 Application.
I want that application to be more beautiful and modern. So I found some good looking css buttons and chose this codepen button:
* {
margin: 0;
padding: 0;
}
html,
body {
box-sizing: border-box;
height: 100%;
width: 100%;
}
body {
background: #FFF;
font-family: 'Noto Sans JP', sans-serif;
font-weight: 400;
}
.buttons {
display: flex;
flex-direction: row;
flex-wrap: wrap;
justify-content: center;
text-align: center;
width: 100%;
height: 100%;
margin: 0 auto;
/* padding: 2em 0em; */
}
.container {
align-items: center;
display: flex;
flex-direction: column;
justify-content: center;
text-align: center;
background-color: #FFF;
padding: 40px 0px;
width: 240px;
}
h1 {
text-align: left;
color: #444;
letter-spacing: 0.05em;
margin: 0 0 0.4em;
font-size: 1em;
}
p {
text-align: left;
color: #444;
letter-spacing: 0.05em;
font-size: 0.8em;
margin: 0 0 2em;
}
.btn {
letter-spacing: 0.1em;
cursor: pointer;
font-size: 14px;
font-weight: 400;
line-height: 45px;
max-width: 160px;
position: relative;
text-decoration: none;
text-transform: uppercase;
width: 100%;
}
.btn:hover {
text-decoration: none;
}
/*btn_background*/
.effect01 {
color: #FFF;
border: 4px solid #000;
box-shadow:0px 0px 0px 1px #000 inset;
background-color: #000;
overflow: hidden;
position: relative;
transition: all 0.3s ease-in-out;
}
.effect01:hover {
border: 4px solid #666;
background-color: #FFF;
box-shadow:0px 0px 0px 4px #EEE inset;
}
/*btn_text*/
.effect01 span {
transition: all 0.2s ease-out;
z-index: 2;
}
.effect01:hover span{
letter-spacing: 0.13em;
color: #333;
}
/*highlight*/
.effect01:after {
background: #FFF;
border: 0px solid #000;
content: "";
height: 155px;
left: -75px;
opacity: .8;
position: absolute;
top: -50px;
-webkit-transform: rotate(35deg);
transform: rotate(35deg);
width: 50px;
transition: all 1s cubic-bezier(0.075, 0.82, 0.165, 1);/*easeOutCirc*/
z-index: 1;
}
.effect01:hover:after {
background: #FFF;
border: 20px solid #000;
opacity: 0;
left: 120%;
-webkit-transform: rotate(40deg);
transform: rotate(40deg);
}
<div class="buttons">
<div class="container">
<h1>光の反射</h1>
<p>Light reflection</p>
<span>Hover</span>
</div>
</div>
My question is: how can I use pyqt's functions to perform the :after.
I guess I cant just use a widget's stylesheet to perform.
Here is the Python Code that I have written so far:
from PyQt6.QtCore import QRect, Qt
from PyQt6.QtGui import QEnterEvent
from PyQt6.QtWidgets import QGraphicsOpacityEffect, QMainWindow, QPushButton
class QButton(QPushButton):
def __init__(self , main_window: QMainWindow):
super().__init__(main_window)
self.main_window = main_window
self.setText("Welcome")
self.render()
def render(self):
self.setStyleSheet("""
QPushButton{
background-color: #2E3440;
letter-spacing: 0.1px;
font-size: 14px;
font-weight: 400;
line-height: 45px;
text-decoration: none;
color: #FFF;
border: 4px solid #4C566A;
}
""")
self.setFlat(True)
self.setCursor(Qt.CursorShape.PointingHandCursor)
self.setGeometry(QRect(200 , 200 , 200 , 50))
def setText(self , text: str):
text = text.upper()
super().setText(text)
def enterEvent(self, event: QEnterEvent) -> None:
self.setStyleSheet("""
background-color: #D8DEE9;
letter-spacing: 0.13em;
color: #2E3440;
border: 0px;
""")
opacity = QGraphicsOpacityEffect()
opacity.setOpacity(.8)
self.setGraphicsEffect(opacity)
return super().enterEvent(event)
class Renderer:
def __init__(self, main_window: QMainWindow) -> None:
self.main_window = main_window
def render(self):
button = QButton(self.main_window)
if __name__ == "__main__":
app = QApplication([])
window = QMainWindow()
renderer = Renderer(window)
renderer.render()
window.show()
exit(app.exec())
Unfortunately, you can't. Qt Style Sheets are fundamentally based on CSS 2.1, and only support its basic implementation (see the syntax documentation).
If you want a more customized and advanced behavior, you have only two options:
subclass the widget and/or use QProxyStyle;
use QML;
I am facing the problem to creating a matrix in python which divides but instead it multiplies.
I have two dataframes:
df_in = pd.DataFrame([[77.279999], [80.099998]], index=[2019, 2020], columns=['Price'])
df_out = pd.DataFrame([[71.849998], [77.400002]], index=[2019, 2020], columns=['Price])
Now I will create the matrix:
df_matrix = pd.DataFrame(np.outer(df_in, df_out), df_in.index, df_out.index)
The output I get is:
<style type="text/css">
table.tableizer-table {
font-size: 12px;
border: 1px solid #CCC;
font-family: Arial, Helvetica, sans-serif;
}
.tableizer-table td {
padding: 4px;
margin: 3px;
border: 1px solid #CCC;
}
.tableizer-table th {
background-color: #104E8B;
color: #FFF;
font-weight: bold;
}
</style>
<table class="tableizer-table">
<thead><tr class="tableizer-firstrow"><th></th><th>2019</th><th>2020</th></tr></thead><tbody>
<tr><td>2019</td><td>5552.567794</td><td>5755.184768</td></tr>
<tr><td>2020</td><td>5981.472023</td><td>6199.740004</td></tr>
</tbody></table>
It is multiplying instead of dividing. The next problem I am facing is that if
df_in.index > df_out.index
then value should be 0.
The result that I would like to see is:
<style type="text/css">
table.tableizer-table {
font-size: 12px;
border: 1px solid #CCC;
font-family: Arial, Helvetica, sans-serif;
}
.tableizer-table td {
padding: 4px;
margin: 3px;
border: 1px solid #CCC;
}
.tableizer-table th {
background-color: #104E8B;
color: #FFF;
font-weight: bold;
}
</style>
<table class="tableizer-table">
<thead><tr class="tableizer-firstrow"><th></th><th>2019</th><th>2020</th></tr></thead><tbody>
<tr><td>2019</td><td>1,075574</td><td>1,114822</td></tr>
<tr><td>2020</td><td>0</td><td>1,034883</td></tr>
</tbody></table>
So thanks to all for your advices.
You can do divide.outer:
pd.DataFrame(np.divide.outer(df_in,df_out)[:,0,:,0], df_in.index, df_out.index)
Output:
2019 2020
2019 1.075574 0.998450
2020 1.114823 1.034884
I am using google maps which has a basic form for the user to fill in his details. On clicking submit the form details are suppose to get stored in the MySQL table that I have created.
I am using apache that is already pre-installed on my mac.
I have downloaded MySQL and its running fine. I have created the table with the required columns in it.
I am using PyCharm Community Edition for running my python scripts.
My map.html file (the one which has the google map with the form) is running fine on localhost.
However, on clicking submit I get the following error :
Not Found
The requested URL /Library/WebServer/CGI-Executables/processing.py was not found on this server.
My map.html file is in Library/WebServer/Documents/
#map.html file
<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
<meta charset="utf-8" />
<title>MAPS</title>
<style>
#map {
width: 100%;
height: 100%;
}
html, body {
height: 100%;
margin: 0;
padding: 0;
}
#description {
font-family: Calibri;
font-size: 15px;
font-weight: 300;
}
#infowindow-content .title {
display: none;
}
#map #infowindow-content {
display: none;
}
.pac-card {
margin: 10px 10px 0 0;
border-radius: 2px 0 0 2px;
box-sizing: border-box;
-moz-box-sizing: border-box;
outline: none;
box-shadow: 0 2px 6px rgba(0, 0, 0, 0.3);
background-color: #fff;
font-family: Calibri;
}
#pac-container {
padding-bottom: 12px;
margin-right: 12px;
}
.pac-controls {
display: inline-block;
padding: 5px 11px;
}
.pac-controls label {
font-family: Calibri;
font-size: 16px;
font-weight: 400;
}
#pac-input {
background-color: #fff;
font-family: Calibri;
font-size: 15px;
font-weight: 400;
margin-left: 12px;
padding: 8px 12px;
text-overflow: ellipsis;
width: 400px;
border: 1px solid #ccc;
}
#pac-input:focus {
border-color: #4d90fe;
}
#title {
color: #fff;
background-color: #4d90fe;
font-size: 25px;
font-weight: 400;
padding: 6px 12px;
}
#target {
width: 345px;
}
#form
{
display: none;
}
#heading {
text-align: center;
color: #003366;
font-size: 30px;
font-family: Calibri;
padding: 8px 12px;
}
label.field {
text-align: left;
font-weight: bolder;
font-size: 16px;
font-family: Calibri;
}
input[type=text], input[type=email], input[type=number], select {
font-family: Calibri;
font-size: 16px;
font-weight: bolder;
padding: 8px 12px;
width: 100%;
margin: 8px 0;
display: inline-block;
border: 1px solid black;
border-radius: 4px;
box-sizing: border-box;
}
input[type=submit] {
font-family: Calibri;
font-size: 16px;
font-weight: bolder;
background-color: white;
color: #00b300;
padding: 8px 12px;
margin: 8px 0;
border: 2px solid #00b300;
border-radius: 4px;
cursor: pointer;
box-sizing: border-box;
}
input[type=submit]:hover {
background-color: #00b300;
color: white;
}
input[type=reset] {
font-family: Calibri;
font-size: 16px;
font-weight: bolder;
background-color: white;
color: #cc0000;
padding: 8px 12px;
margin: 8px 0;
border: 2px solid #cc0000;
border-radius: 4px;
cursor: pointer;
box-sizing: border-box;
}
input[type=reset]:hover {
background-color: #cc0000;
color: white;
}
input[type=text]:focus {
background-color: #f2f2f2;
border-color: #4d90fe;
}
input[type=email]:focus {
background-color: #f2f2f2;
border-color: #4d90fe;
}
input[type=number]:focus {
background-color: #f2f2f2;
border-color: #4d90fe;
}
select:focus {
background-color: #f2f2f2;
border-color: #4d90fe;
}
</style>
<script src="https://maps.googleapis.com/maps/api/js?key=AIzaSyC8EAfEQrYVs4c_kMBUQw3tVY8uhEViDCU&libraries=places&callback=initAutocomplete"
async defer>
</script>
</head>
<body>
<div id="map"></div>
<input id="pac-input" class="controls" type="text" placeholder="Search your location..." />
<div id="form">
<form action="Library/WebServer/CGI-Executables/processing.py" method="post">
<p id="heading">PERSONAL INFORMATION</p>
<label class="field" for="fullname"><b>Full Name:</b></label>
<input type="text" name="fullname" placeholder="Eg. Suraj Makhija" size="30" required />
<br />
<br />
<label class="field" for="emailid"><b>E-mail:</b></label>
<input type="email" name="emailid" placeholder="Eg. contact#domain.com" size="30" required />
<br />
<br />
<label class="field" for="mobile"><b>Mobile:</b></label>
<input type="text" name="mobile" placeholder="Eg. 9999999999" maxlength="10" minlength="10" size="30" required />
<br />
<br />
<label class="field" for="gender"><b>Gender:</b></label>
<select name="gender" required>
<option value="male">Male</option>
<option value="female">Female</option>
<option value="other">Other</option>
</select>
<br />
<br />
<label class="field" for="areaofexpertise"><b>Area of Expertise:</b></label>
<input type="text" name="areaofexpertise" placeholder="Eg. Android/iOS Developer" size="30" required />
<br />
<br />
<label class="field" for="yearsofexperience"><b>Years of Experience:</b></label>
<input type="number" name="yearsofexperience" placeholder="Eg. 10" min="0" max="100" required />
<br />
<br />
<p style="text-align: center;">
<input type="submit" value="Submit" />
<input type="reset" value="Clear All" />
</p>
</form>
</div>
<script>
var map;
var iconBase;
var input;
var searchBox;
var markers=[];
var markerArray =[];
var infowindow;
function initAutocomplete() {
map = new google.maps.Map(document.getElementById('map'), {
center: {lat: 20.5937, lng: 78.9629},
zoom: 5,
mapTypeId: 'roadmap',
gestureHandling: 'cooperative'
});
//Declaring infowindow.
infowindow = new google.maps.InfoWindow({
content: document.getElementById('form')
});
// Create the search box and link it to the UI element.
input = document.getElementById('pac-input');
searchBox = new google.maps.places.SearchBox(input);
map.controls[google.maps.ControlPosition.TOP_LEFT].push(input);
// Bias the SearchBox results towards current map's viewport.
map.addListener('bounds_changed', function() {
searchBox.setBounds(map.getBounds());
});
//Creates a marker when the user clicks on a location on the map.
google.maps.event.addListener(map, 'click', function(event) {
addMarker(event.latLng, map);
});
// Listen for the event fired when the user selects a prediction and retrieve
// more details for that place.
searchBox.addListener('places_changed', function() {
var places = searchBox.getPlaces();
if (places.length == 0) {
return;
}
// Clear out the old markers.
markers.forEach(function(marker) {
marker.setMap(null);
});
// For each place, get the icon, name and location.
var bounds = new google.maps.LatLngBounds();
places.forEach(function(place) {
if (!place.geometry) {
console.log("Returned place contains no geometry");
return;
}
var icon = {
url: place.icon,
size: new google.maps.Size(71, 71),
origin: new google.maps.Point(0, 0),
anchor: new google.maps.Point(17, 34),
scaledSize: new google.maps.Size(25, 25)
};
// Create a marker for each place.
markers.push(new google.maps.Marker({
map: map,
icon: icon,
title: place.name,
position: place.geometry.location
}));
if (place.geometry.viewport) {
// Only geocodes have viewport.
bounds.union(place.geometry.viewport);
} else {
bounds.extend(place.geometry.location);
}
});
map.fitBounds(bounds);
});
}
//Marker function declaration.
function addMarker(location, map) {
// Add the marker at the clicked location.
var iconBase = 'https://maps.google.com/mapfiles/kml/paddle/';
marker = new google.maps.Marker({
draggable: true,
animation: google.maps.Animation.DROP,
position: location,
map: map,
icon: iconBase + 'ylw-blank.png'
});
//Infowindow pops up on clicking the marker.
google.maps.event.addListener(marker, 'click', function() {
document.getElementById('form').style.display='inline-block';
infowindow.open(map, marker);
});
}
</script>
</body>
</html>
My processing.py file (the python script that is used for processing the form data and inserting into the MySQL table) is in Library/WebServer/CGI-Executables/
I haven't made any changes to my httpd.conf file except for adding .py to AddHandler statement.
I am attaching all my files below.
Please help me with this as I have been stuck on this for a very long time now.
Please note that Im using Mac OS Sierra.
Thanks in advance.
#processing.py file
import MySQLdb
import cgi
import cgitb
cgitb.enable()
form = cgi.FieldStorage()
fullname = form.getvalue('fullname', '')
emailid = form.getvalue('emailid', '')
mobile = form.getvalue('mobile', '')
gender = form.getvalue('gender', '')
areaofexpertise = form.getvalue('areaofexpertise', '')
yearsofexperience = form.getvalue('yearsofexperience', '')
db = MySQLdb.connect(host="localhost", user="root", passwd="*****", db="client_database")
cursor = db.cursor()
query = """INSERT INTO client_info (FULL_NAME, EMAIL_ID, MOBILE, GENDER, AREA_OF_EXPERTISE, YEARS_OF_EXPERIENCE)
VALUES (%s, %s, %s, %s, %s, %s)"""
cursor.execute(query, (fullname, emailid, mobile, gender, areaofexpertise, yearsofexperience))
result = cursor.fetchall()
db.commit()
db.close()
Lets say there's a file that lives at the github repo:
https://github.com/someguy/brilliant/blob/master/somefile.txt
I'm trying to use requests to request this file, write the content of it to disk in the current working directory where it can be used later. Right now, I'm using the following code:
import requests
from os import getcwd
url = "https://github.com/someguy/brilliant/blob/master/somefile.txt"
directory = getcwd()
filename = directory + 'somefile.txt'
r = requests.get(url)
f = open(filename,'w')
f.write(r.content)
Undoubtedly ugly, and more importantly, not working. Instead of the expected text, I get:
<!DOCTYPE html>
<!--
Hello future GitHubber! I bet you're here to remove those nasty inline styles,
DRY up these templates and make 'em nice and re-usable, right?
Please, don't. https://github.com/styleguide/templates/2.0
-->
<html>
<head>
<meta http-equiv="Content-type" content="text/html; charset=utf-8">
<title>Page not found · GitHub</title>
<style type="text/css" media="screen">
body {
background: #f1f1f1;
font-family: "HelveticaNeue", Helvetica, Arial, sans-serif;
text-rendering: optimizeLegibility;
margin: 0; }
.container { margin: 50px auto 40px auto; width: 600px; text-align: center; }
a { color: #4183c4; text-decoration: none; }
a:visited { color: #4183c4 }
a:hover { text-decoration: none; }
h1 { letter-spacing: -1px; line-height: 60px; font-size: 60px; font-weight: 100; margin: 0px; text-shadow: 0 1px 0 #fff; }
p { color: rgba(0, 0, 0, 0.5); margin: 20px 0 40px; }
ul { list-style: none; margin: 25px 0; padding: 0; }
li { display: table-cell; font-weight: bold; width: 1%; }
#error-suggestions { font-size: 14px; }
#next-steps { margin: 25px 0 50px 0;}
#next-steps li { display: block; width: 100%; text-align: center; padding: 5px 0; font-weight: normal; color: rgba(0, 0, 0, 0.5); }
#next-steps a { font-weight: bold; }
.divider { border-top: 1px solid #d5d5d5; border-bottom: 1px solid #fafafa;}
#parallax_wrapper {
position: relative;
z-index: 0;
}
#parallax_field {
overflow: hidden;
position: absolute;
left: 0;
top: 0;
height: 370px;
width: 100%;
}
etc etc.
Content from Github, but not the content of the file. What am I doing wrong?
The content of the file in question is included in the returned data. You are getting the full GitHub view of that file, not just the contents.
If you want to download just the file, you need to use the Raw link at the top of the page, which will be (for your example):
https://raw.githubusercontent.com/someguy/brilliant/master/somefile.txt
Note the change in domain name, and the blob/ part of the path is gone.
To demonstrate this with the requests GitHub repository itself:
>>> import requests
>>> r = requests.get('https://github.com/kennethreitz/requests/blob/master/README.rst')
>>> 'Requests:' in r.text
True
>>> r.headers['Content-Type']
'text/html; charset=utf-8'
>>> r = requests.get('https://raw.githubusercontent.com/kennethreitz/requests/master/README.rst')
>>> 'Requests:' in r.text
True
>>> r.headers['Content-Type']
'text/plain; charset=utf-8'
>>> print r.text
Requests: HTTP for Humans
=========================
.. image:: https://travis-ci.org/kennethreitz/requests.png?branch=master
[... etc. ...]
You need to request the raw version of the file, from https://raw.githubusercontent.com.
See the difference:
https://raw.githubusercontent.com/django/django/master/setup.py vs. https://github.com/django/django/blob/master/setup.py
Also, you should probably add a / between your directory and the filename:
>>> getcwd()+'foo.txt'
'/Users/burhanfoo.txt'
>>> import os
>>> os.path.join(getcwd(),'foo.txt')
'/Users/burhan/foo.txt'
Just as an update, https://raw.github.com was migrated to https://raw.githubusercontent.com. So the general format is:
url = "https://raw.githubusercontent.com/user/repo/branch/[subfolders]/file"
E.g. https://raw.githubusercontent.com/earnestt1234/seedir/master/setup.py. Still use requests.get(url) as in Martijn's answer.
Adding a working example ready for copy+paste:
import requests
from requests.structures import CaseInsensitiveDict
url = "https://raw.githubusercontent.com/organization/repo/branch/folder/file"
# If repo is private - we need to add a token in header:
headers = CaseInsensitiveDict()
headers["Authorization"] = "token TOKEN"
resp = requests.get(url, headers=headers)
print(resp.status_code)
(*) If repo is not private - remove the headers part.
Bonus:
Check out this Curl < --> Python-requests online converter.