Python urllib2 fails to open web page - python

I have a problem where a page that I can open in Firefox, Chrome, or even Java fails to open in Python using urllib2:
import urllib2
sock = urllib2.urlopen('http://www.example.com')
li = sock.read()
sock.close()
print li
This code fails (for the particular company web page I am trying to load). The page is actually an interface to a complex back-end server, and the response we are getting is just a couple lines of (incorrect) text. At first we thought there was some bot filtering going on, but we did get the page to load using Java:
package com.ebay.marketing.rtm.components.impl.selector;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
public class RtmApiCheck {
private static Client mClient;
private void initClient() {
Client client = mClient;
if (client == null) {
synchronized (this) {
client = mClient;
if (client == null) {
mClient = client = Client.create();
}
}
}
}
public static void main(String[] args) {
RtmApiCheck check = new RtmApiCheck();
try {
check.initClient();
for(int i=0;i<100;i++) {
WebResource wr = mClient.resource("http://www.example.com");
ClientResponse result = wr.get(ClientResponse.class);
String strResult = result.getEntity(String.class);
System.out.println(strResult);
}
}
catch(Exception e) {
e.printStackTrace();
}
}
}
What is going on with Python that causes this code to fail? Is there an alternative way to load the page that might work?

Related

Python server and java client not working properly

I want to make a socket between Java client
`
import java.util.*;
import java.net.*;
import java.io.*;
public class Main
{
public static void client()
{
final int port = 2003;
final String host = "localhost";
try
{
Socket soc = new Socket(host, port);
if(soc.isBound()){
System.out.println("socket binded");
}else if(soc.isConnected()){
Scanner in = new Scanner(soc.getInputStream());
System.out.println("enter msg");
String x = in.nextLine();
PrintStream p = new PrintStream(soc.getOutputStream());
p.println("#client" + x);
} else if(soc.isClosed()){
System.out.println("Closed tunnel");
System.exit(0);
soc.close();
}
}
catch (Exception e)
{
}
}
public static void main(String[] args)
{
client();
}
}
`
Its having problem allowing me to use the scanner to send msg to Python server{I dunno how to make this}
PS : am new and dunno how to make this snippets formatted

Is there a way to turn PIL string bytes into a viewable image on my web application?

I have marshalled PIL image into bytes via toBytes() on my Flask Server.
Then these bytes was then embedded in Json as a string and transferred via http to my Angular web application.
Is there a way to turn this binary into a viewable image on my web application? Or is there a better way transfering image from my Flask server to my Angular web application?
A part of my code python flask code:
imgByteArrTemp = io.BytesIO()
img.save(imgByteArrTemp, format="PNG")
# imgByteArrTemp = base64.b64encode(imgByteArrTemp.getvalue())
imgByteArrTemp = imgByteArrTemp.getvalue()
My GET method:
#app.route("/get-resulting-image", methods=["GET"])
def get_resulting_image():
global q
if not q.empty():
item = q.get()
return send_file(io.BytesIO(item),
mimetype = "image/png",
as_attachment=True,
download_name="%s.png")
return "no image to provide"
My typescript app.component.ts:
createImageFromBlob(image:Blob){
let reader = new FileReader();
reader.addEventListener("load", () => {
this.imageBinary = reader.result;
console.log(this.imageBinary);
},false);
if (image){
reader.readAsDataURL(image);
}
}
getResponse(){
...
this.restService.getCorrespondingImage().subscribe(
data1 => {
this.createImageFromBlob(data1);
this.isImageLoading = false;
},
error => {
this.isImageLoading = false;
console.log(error)
}
);
...
}
app.component.html
<img [src]="imageBinary" *ngIf="!isImageLoading">
rest.service.ts
import { Injectable } from '#angular/core';
import {HttpClient} from '#angular/common/http';
import {Observable, throwError} from 'rxjs';
import {catchError, retry} from 'rxjs/operators';
#Injectable({
providedIn: 'root'
})
export class RestService {
constructor(private http: HttpClient) { }
....
getCorrespondingImage():Observable<Blob>{
return this.http.get(
"http://192.168.1.2:9500/get-resulting-image",
{responseType:"blob"});
}
}
Hope this could help someone out there.

How can I get the console output in Angular webpage?

I have a Django project which is using Angular as frontend. I have a button which on clicking is scanning the tables in the database. I have some print statements views.py file which is printing the scanned results constantly in the IDE console. I want that output in the webpage. I want that live printing of the console output in the frontend. Can any one know how i can achieve this?
You can achieve this by using server sent events. python can push these console logs to the frontend. Not a expert of python so giving a link below to how to send server side events from python to frontend
https://medium.com/code-zen/python-generator-and-html-server-sent-events-3cdf14140e56
In frontend you can listen to url exposed and as soon as server will push any message on this stream frontend can receive it and push it into component's array and can display over ui.
for frontend code, i am giving a minimal example below :-
import { Injectable, NgZone } from "#angular/core";
import { Observable } from "rxjs";
#Injectable({
providedIn: "root"
})
export class SseService {
constructor(private _zone: NgZone) {}
getServerSentEvent(url: string): Observable<any> {
return Observable.create(observer => {
const eventSource = this.getEventSource(url);
eventSource.onmessage = event => {
this._zone.run(() => {
observer.next(event);
});
};
eventSource.onerror = error => {
this._zone.run(() => {
observer.error(error);
});
};
});
}
private getEventSource(url: string): EventSource {
return new EventSource(url);
}
}
you can susbcribe to getServerSentEvent in above method and can continuously receive new messages, which is in your case your console logs.
You can try calling the following function with the information needed to be displayed.
addItem(val:any) {
let node = document.createElement("li")
let textnode = document.createTextNode(val)
node.appendChild(textnode)
document.getElementById("output").appendChild(node)
}
Make sure to have an element with the id="output".

Sending Image from Android with Retrofit and Getting the Image with Python Flask

I am trying to send an image to flask endpoint from android application using retrofit 2 but I seem to fail every time. Flask endpoint is working with both html and postman posts so the problem is on the retrofit part.
Here is the Flask endpoint:
#app.route("/uploadfile", methods=["POST"])
def uploadsingle():
file = request.files['file']
file.save(os.path.join("/home/moralalp/mysite/", file.filename))
return "Success"
Below is the interface for retrofit:
#Multipart
#POST("uploadfile")
Call<ResponseBody> uploadPhoto(#Part("description") RequestBody description, #Part MultipartBody.Part file);
And finally, the uploadFile method:
private void uploadFile(Uri fileUri) {
final EditText name = findViewById(R.id.editText);
RequestBody descriptionPart = RequestBody.create(MultipartBody.FORM, name.getText().toString());
File originalFile = new File(getRealPathFromURI(fileUri));
RequestBody filePart = RequestBody.create(MediaType.parse(getContentResolver().getType(fileUri)), originalFile);
MultipartBody.Part file = MultipartBody.Part.createFormData("file", originalFile.getName(), filePart);
Retrofit.Builder builder = new Retrofit.Builder()
.baseUrl("https://mysite.pythonanywhere.com/")
.addConverterFactory(GsonConverterFactory.create());
Retrofit retrofit = builder.build();
UserClient client = retrofit.create(UserClient.class);
Call<ResponseBody> call = client.uploadPhoto(descriptionPart, file);
call.enqueue(new Callback<ResponseBody>() {
#Override
public void onResponse(Call<ResponseBody> call, Response<ResponseBody> response) {
Toast.makeText(MainActivity.this, "YEAH", Toast.LENGTH_SHORT).show();
}
#Override
public void onFailure(Call<ResponseBody> call, Throwable t) {
Toast.makeText(MainActivity.this, "NOOO", Toast.LENGTH_SHORT).show();
}
});
}
I keep getting "NOOO" Toast message so I can not even get a response, what could be the problem here?
As for this error please use the next line to figure out what is the problem and please edit your question to contain it
#Override
public void onFailure(Call<ResponseBody> call, Throwable t) {
Toast.makeText(MainActivity.this, "NOOO", Toast.LENGTH_SHORT).show();
Log.d("Error_TAG", "onFailure: Error: " + t.getMessage());
}
After that Please Filter your log cat to the Error_TAG and Add the Error in the Question
Good Luck
Ok so adding the following in the AndroidManifest.xml file solved my problem:
<manifest ... >
<!-- This attribute is "false" by default on apps targeting Android Q. -->
<application android:requestLegacyExternalStorage="true" ... >
...
</application>
</manifest>

Uploading a file via POST multipart/form-data from windows phone 8.1 Silverlight Httpclient to python server corrupts files

Originally, I was using Windows Phone 8 System.Net.Http library to upload an image from SD card to a Python server with the following code:
private async void UploadFile()
{
try
{
// Make sure there is a picture selected
if (photoStream != null)
{
// initialize the client
// need to make sure the server accepts network IP-based
// requests.
// ensure correct IP and correct port address
var fileUploadUrl = #"http://IPAddress/";
var client = new HttpClient();
// Reset the photoStream position
// If you don't reset the position, the content lenght
// sent will be 0
photoStream.Position = 0;
// This is the postdata
MultipartFormDataContent content = new MultipartFormDataContent();
content.Add(new StreamContent(photoStream), "fn", fileName);
// upload the file sending the form info and ensure a result.
// it will throw an exception if the service doesn't return
// a valid successful status code
await client.PostAsync(fileUploadUrl, content)
.ContinueWith((postTask) =>
{
postTask.Result.EnsureSuccessStatusCode();
});
}
// Disable the Upload button
btnUpload.IsEnabled = false;
// reset the image control
DisplayImage.Source = null;
}
catch
{
}
}
In the server side, I used the following python code to decode the request:
ctype, pdict = cgi.parse_header(self.headers['content-type'])
data = cgi.parse_multipart(self.rfile, pdict)
It worked until I switched from Windows Phone 8 to Windows Phone 8.1 Silverlight. Since the System.Net.Http library is no longer supported in Windows Phone 8.1 Silverlight, I used Windows.Web.Http instead with modification of the code:
private async void UploadFile()
{
try
{
// Make sure there is a picture selected
if (file != null)
{
// initialize the client
// need to make sure the server accepts network IP-based
// requests.
// ensure correct IP and correct port address
var fileUploadUrl = new Uri(#"http://IPAddress/", UriKind.Absolute);
var client = new HttpClient();
HttpMultipartFormDataContent content = new HttpMultipartFormDataContent();
HttpStreamContent streamContent = new HttpStreamContent(photoStream);
content.Add(streamContent, "fn", fileName);
await client.PostAsync(fileUploadUrl, content);
}
// Disable the Upload button
btnUpload.IsEnabled = false;
// reset the image control
DisplayImage.Source = null;
}
catch
{
}
}
The server side is unchanged. However, the resulting file uploaded now has extra bytes inserted periodically that look something like '\r\n10000\r\n' or 0D 0A 31 30 30 30 30 0D 0A. The file also appears to begin and end with the same extra byte pattern. On the server pdict maps the boundary to something like xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, but this seems to be the same behavior from when the file was uploaded via System.Net.Http.
I tried removing these bytes by hand on a smaller file which seemed to confirm that the file is otherwise correct.
Why are these extra bytes present? Is this a problem with the Python server mishandling the POST request?
Here's my solution when I encountered this issue:
Client:
public async Task UploadImage(byte[] image, string url)
{
Stream stream = new System.IO.MemoryStream(image);
HttpStreamContent streamContent = new HttpStreamContent(stream.AsInputStream());
Uri resourceAddress = null;
Uri.TryCreate(url.Trim(), UriKind.Absolute, out resourceAddress);
Windows.Web.Http.HttpRequestMessage request = new Windows.Web.Http.HttpRequestMessage(Windows.Web.Http.HttpMethod.Post, resourceAddress);
request.Content = streamContent;
var httpClient = new Windows.Web.Http.HttpClient();
var cts = new CancellationTokenSource();
Windows.Web.Http.HttpResponseMessage response = await httpClient.SendRequestAsync(request).AsTask(cts.Token);
}
Controller:
public async Task<HttpResponseMessage> Post()
{
Stream requestStream = await this.Request.Content.ReadAsStreamAsync();
byte[] byteArray = null;
using (MemoryStream ms = new MemoryStream())
{
await requestStream.CopyToAsync(ms);
byteArray = ms.ToArray();
}
.
.
.
return Request.CreateResponse(HttpStatusCode.OK);
}

Categories