I have a flash video upload that is going from the browser directly to S3. It works.
However, if the connection is interrupted, it will time out after about three seconds. On the other hand, if you upload a video to YouTube, even if you disconnect the internet, it will hang and then re-sume when the internet connection is working again. To test this, I uploaded a video, stopped my connection for five minutes (timed), and then started my internet connection again. After about thirty seconds the video continued uploading again. This is especially important, since the videos being uploaded are quite large (1GB+) and it may take several hours to upload.
How does YouTube do this? What do I need to change or examine in my own uploader in order for it to work as YouTube's does? Thank you.
Related
How can I quickly download audio from YouTube by URL or ID and send it to Telegram bot? I've been using youtube-dl to download audio, save it on hosting and after that send it to user. It takes 1-2 minutes to do that. But other bots (like this one #LyBot) do this with the speed of light. How do they do this?
As it says in their documentation "I send audio instantly if it has already been downloaded through me earlier. Otherwise, the download usually takes no more than 10 seconds."
They probably store a file the first time its downloaded by any user so that it can be served instantly for subsequent requests.
I'm trying to send my image from django server to app.There are two appoaches for above problem:
1)send url of the image
2)send encoded image
I'm not sure which one to use due to following:-
If i choose the first way. Json resoponse will be quicker but if a user returns a page twice, it will make two request on server. Moreever it would be impossible to cache the image.
In case of second approach, i can cache the image at client app, but does encoding increase overall response time from the server?
Which is the recommended approach for sending image from api.
Currently i'm sending image by following:-
return JsonResponse({'image':model.image.url})
The answer is approach 1. Encoding images will destroy your server response time unless they’re very tiny like thumbnails or avatars, even then I wouldn’t make a habit of it. I’ve seen apps rendered unusable by this practice. Most browsers cache images during sessions automatically. If server performance is a big concern and images are dragging it down, Generally I’ll store images in some kind of static file host though like s3 and use an edge cache anyway in real world environments.
You probably want to go with #1 and send over the URL. Usually you would not deliver the image files directly from your server but from a cloud storage like S3 or GCS. More advanced setups even include CDNs (Content Delivery Networks) like Fastly or Cloudfront to enable easy caching and serve traffic on a global scale.
If you should choose to send over the encoded image (in base64 for example), be aware that the increased response size of the body will increase the response times incredibly high and might even lead to complete timeouts at around 30s. Users will ultimately have longer response times, are more likely to churn and you are paying way more for your servers.
I'm using Flask to serve .m3u8 and .ts files to simulate a vod stream.
The video player does not stream the file and shows an error (see the screenshot below). I can't find a log of what error it is.
Is there a log message somewhere that I'm missing? What is causing this and how can I fix it?
The development server runs in single threaded mode by default, meaning it can only handle one request at a time. You are requesting streams of two files at once, the .m3u8 and the .ts. You can pass threaded=True or processes=value greater than 1 to app.run to allow handling of multiple requests at once, but that comes with it's own problems. The development server in general seems to have problems streaming html5 video and audio. The real solution is to use an actual server such as Nginx or Apache to serve the media files.
I am trying to design a web application that processes large quantities of large mixed-media files coming from asynchronous processes. Each process can take several minutes.
The files are either uploaded as a POST body or pulled by the web server according to a source URL provided. The files can be processed by a variety of external tools in a synchronous or asynchronous way.
I need to be able to load balance this application so I can process multiple large files simultaneously for as much as I can afford to scale.
I think Python is my best choice for this project, but beside this, I am open to any solution. The app can either deliver the file back or rely on a messaging channel to notify the clients about the process completion.
Some approaches I thought I might use:
1) Use a non-blocking web server such as Tornado that keeps the connection open until the file processing is done. The external processing command is launched and the web server waits until the file is ready and pipes the resulting IO stream directly back to the web app that returns it. Since the processes sending requests are asynchronous, they might afford to wait (unless memory or some other issues come up).
2) Use a regular web server like Cherrypy (which I am more confident with) and have the webapp use a messaging channel to report the processing progress. The web server returns a HTTP response as soon as it receives the file, validates it and sends it to a background process. At the same time it sends a message notifying the process start. The background process then takes care of delivering the file to an available location and sending another message to the channel notifying the location of the new file. This solution looks more flexible than 1), but requires writing a separate script to handle the messages outside the web application, as well as a separate storage space for the temp files that have to be cleaned up at a certain point.
3) Use some internal messaging capability of any of the webserves mentioned above, which I am not familiar with...
Edit: something like CherryPy's pub-sub engine (http://cherrypy.readthedocs.org/en/latest/extend.html?highlight=messaging#publish-subscribe-pattern) could be a good solution.
Any suggestions?
Thank you,
gm
I had a similar situation come up with a really large scale data processing engine that my team implemented. We wanted to build our api calls in Flask, some of which can take many hours to complete, but have a way to notify the user in real time what is going on.
Basically what I came up with is was what you described as option 2. On the same machine that I am serving the flask app through apache, I created a tornado app that serves up a websocket that reports progress to the end user. Once my main page is served, it establishes the websocket connection to the tornado server, and the flask app periodically sends updates to the tornado app, and down to the end user. Even if the browser is closed during the long running application, apache keeps the request alive and processing, and upon logging back in, I can still see the current progress.
I wrote about this solution in some more detail here:
http://jonfeatherstone.com/2013/08/01/mongo-and-websockets-for-application-logging/
Good luck!
How can I stream a live movie or video for my clients to connect to a webpage and watch? I would prefer that it's in a SWF form, so maybe I could load a file into the SWF. This isn't a webcam type of thing, like maybe if I want to stream a video I made a while ago and allow my users to watch it all at once, and not have it restart every time they refresh. Could this be done in Python, PHP, etc? Or does it involve a program? I am using Ubuntu for my dedicated server. I also need the audio to be live too.
Thanks
"stream a live movie or video" and "stream a video I made a while ago" are contradictory, what do you really mean by "live"? Are you referring to a "broadcasted video" (i.e. may be prerecorded, but everyone will be viewing from the same stream) or "live video" (i.e. viewing events as it happens, being live implies broadcast).
There are services like Justin.TV or Ustream.TV that allows you to broadcast vieo stream from either live cameras, live screen capture, and/or prerecorded videos. They have their own (flash-based or HTML5-based) web players that can handle live streams and you will need to use their broadcasting software to manage the live stream. There is also Youtube Live if you are a Youtube partner.