Long story short i'm making script for parsing sankakucomplex in search of pictures and downloading them. And everything until this function goes fine
def download_image_from_link(link, auth, number):
url = f"https:{link}"
bruh = requests.get(url)
with open(f"images/{auth}/image{number}.png", "wb") as im:
im.write(bruh.content)
{link} is something like //v.sankakucomplex.com/data/sample/a5/dc/sample-a5dcd9f21fc80b6bfe5acfa74c3bd936.jpg?e=1648321131&expires=1648321131&m=JqNaW8AUSE-G-jyihn0uiQ&token=hLsB-lYp7bQhgUbkLsZScpch1DBX6JQoFh4T-CglAKA
{auth} and {number} is not important
It just downloads all the puctires from array that i received from other inconsiderable in this situation functions.
And half of links to images starts with http://s. and other half is http://v.
So the thing is when it downloads http://v. files everything is fine, but in when it downloads http://s. it receives
b'<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body>\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'
Here is the safest links that i could find if you need to check:
for //s. link
https://chan.sankakucomplex.com/ru/post/show/30856755
for //v. link (requests for //beta. doesnt work too)
https://beta.sankakucomplex.com/ru/post/show/30856755
Related
I have a Flask app that generates video stream links. It connects to a server using login credentials and grabs a one time use link (that expires when a new link is generated using the same credentials). Using a list of credentials I am able to stream to as many devices as I like, so long as I have enough accounts.
The issue I am having is that one of the clients doesn't like the way the stream is returned.
#app.route("/play", methods=["GET"])
def play():
def streamData():
try:
useAccount(<credentials>)
with requests.get(link, stream=True) as r:
for chunk in r.iter_content(chunk_size=1024):
yield chunk
except:
pass
finally:
freeAccount(<credentials>)
...
# return redirect(link)
return Response(streamData())
If I return a redirect then there are no playback issues at all. The problem with a redirect is I don't have a way of marking the credentials as in use, then freeing them after.
The problem client is TVHeadend. I am able to get it to work by enabling the additional avlib inside of TVHeadend... But I shouldn't have to do that. I don't have to when I return a redirect.
What could be the cause of this?
Is it possible to make my app respond in the same way as the links server does?
My guess is that TVHeadend is very strict on if something complies to whatever standards... and I am guessing my app doesn't?
I'am trying to get case attachment to save in local folder. I have problem with using attachment url to download it, each time server returns empty results and status code 200.
This is a sample url I use (changed host and token) :
https://example.fogbugz.com/default.asp?pg=pgDownload&pgType=pgFile&ixBugEvent=385319&ixAttachment=56220&sFileName=Log.7z&sTicket=&sToken=1234567890627ama72kaors2grlgsk
I have tried using token instead of sToken but no difference. If I copy above url to chrome then it won't work either, but If I login to FogBugz (manuscript) and then try this url again then it works. So I suppose there are some security issues here.
btw. I use python FogBugz api for this and save url using urllib urllib.request.urlretrieve(url, "fb/" + file_name)
Solution I have found is to use cookies from the web browser where I have previously loged in to FB account I use. So it looks like a security issue.
For that I used pycookiecheat (for windows see my fork: https://github.com/luskan/pycookiecheat). For full code see here: https://gist.github.com/luskan/66ffb8f82afb96d29d3f56a730340adc
I recorded a HSL stream by writing the MPEG-TS streams contents into GridFS filesystem.
i'm now trying to serve this content back to the browser using aiohttps SessionResponse which fails for different reasons.
async def get_video(request):
stream_response = StreamResponse()
stream_response.headers['Content-Type'] = 'video/mp2t'
stream_response.headers['Cache-Control'] = 'no-cache'
stream_response.headers['Connection'] = 'keep-alive'
await stream_response.prepare(request)
fd = GridFS()
video_stream = await fd(video_id)
while True:
try:
chunk = await video_stream.readchunk()
if not chunk:
break
stream_response.write(chunk)
except CancelledError as e:
# fails here in safari or with diff content-type also in chrome
break
await stream_response.write_eof()
return stream_response
When trying to access the url using safari i get the player ui presented but nothing plays while the server throws a CancelledError exception trying to write on the already closed SessionResponse
Opening the URL in Chrome results in downloading the video file. This file works when playing it back in VLC. Even playing the URL inside VLC using "Network Source" works.
I also tried serving a static m3u playlist in front of this direct url like this but without luck (VLC also works using the playlist instread of direct stream):
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="medium",NAME="Medium",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=992000,RESOLUTION=852x480,CODECS="avc1.66.31,mp4a.40.2",VIDEO="medium"
http://localhost:8080/videos/{video_id}
I'm not sure how do debug this any further and would appreciate any help (or ask in comments if i'm unclear). What am i missing that the files don't get played back in browser when accessing them directly? Also embedding my resource url into a html video tag didn't help (obviously, since browser do the same when accessing a video directly)
Some more informations about the video content and the raw http resonses i'm sending:
Video Informations VLC
Direct Video Stream HTTP Response (start)
M3U Playlist HTTP Response
I have no experience with HLS personally but even vast overview of RFC draft displays that you breaks the protocol.
It's not about sending video chunks all together in single endless response but about sending multiple http responses utilizing the same socket connection by keep-alive usage.
Client sends request for new data portions providing protocol-specific EXT* flags and server should respond properly. At very beginning client asks for playlist, server should answer with proper data.
Communication protocol is complex enough, sorry.
I cannot just fix a couple lines in your snippet to make it work.
(unsure of how to phrase this question)
Essentially I'm working with Flask + Soundcloud and what I want to do is to request an http site (which i know will redirect me to a new site) and then i want to return that site (with the same headers and info i originally got). Maybe this explains it better:
#app.route('/play')
def SongURL2():
stream_url="https://api.soundcloud.com/tracks/91941888/stream?client_id=MYCLIENTID"
// newurl = HTTP_REQUEST(stream_url) <- This will redirect me to the actual song streaming link (which only lives for a little bit)
// return newurl;
This is because soundcloud's song's streaming url only live for a short period of time and the device I am using to call my RESTful api will not allow me to do a simple redirect to the newlink. So I need to somehow act like a proxy.
You can achieve this using the Request module:
import requests
#app.route('/play')
def SongURL2():
stream_url="https://api.soundcloud.com/tracks/91941888/stream?client_id=MYCLIENTID"
# Get the redirected url
r = request.get(stream_url)
return r.url
Found an interesting way to proxy through Flask, similar to what #Dauros was aiming at. http://flask.pocoo.org/snippets/118/ In the end bare in mind that this puts extra strain on the server.
I don't know if it is possible with Bottle.
My website (powered by Bottle) allow users to upload image files. But I limited the size of it to 100K. I use the following code in web server to do that.
uploadLimit = 100 # 100k
uploadLimitInByte = uploadLimit* 2**10
print("before call request.headers.get('Content-Length')")
contentLen = request.headers.get('Content-Length')
if contentLen:
contentLen = int(contentLen)
if contentLen > uploadLimitInByte:
return HTTPResponse('upload limit is 100K')
But when I clicked upload button in web browser to upload a file with its size like 2MB, it seems the server is receiving the whole 2MB http request.
I expect the above code just receive http headers instead of receiving whole http request. That could not prevent wasting time on receving unecessary bytes