Question
How can I take a small sample of streamed frames, and manipulate them using Python? Are there any available libraries to use, or will I have to code the entire project alone?
Tech Specs
OS: Linux
Connection: CAT-5 Ethernet
Camera: dlink DCS-930L
Introduction
I recently asked a question, but it was closed because of clarity issues.
I am re-posting with many more details, and if it is still not clear, feel free to edit or add comments.
Background
I have a dlink DCS-930L camera which is directly connected to my Linux computer with a direct cat5 connection. I assigned it to a static IP adress, and everything works great.
When I open a web-browser, and connect to this static IP address (e.g. log into 192.168.0.20), the camera just works correctly in real time.
I did this was to verify that my camera was working, and that I was able to establish the Ethernet connection correctly.
Now, what I need to do some image processing on the video frames that I receive over the Ethernet from the camera.
I don't want to use the web-browser anymore as a means of display, and instead, I want to use Python to read the frames.
In other words, let's say that the camera produces 30 frames/second, and each frame has a certain size (e.g 1920x1080 pixels).
All I want to do is to start reading these frames in by Python. I don't mind if I am missing frames and if I am processing it slowly. Even if I am able to process one frame over a few seconds, I am still okay with that.
Since video is a collection of images (in this case 30 images per second), I want to be able to read these images using Python, and then be able to do whatever processing that I need to do on these images.
If I had these images saved on the computer, I would open these images with Python, and start to manipulate them. But, since in this case, the images are in fact being streamed, I just want to know how can I sample them (maybe one every few second), and do some manipulation using Python?
Please let me know if my question is still unclear, and I will try to clarify it as much as I can.
Thanks,
--Rudy
According to the manual, the camera serves video through a java applet, so that is will bedifficult to access through python without understanding that server protocol.
However, it does have an option to push images to at ftp server (page 34), so if you install vsftpd on your linux box, you can tell the camera to push images there at maybe as high as 4 fps. There are instructions on setting up vsftpd on ubuntu here, other versions of linux will be similar (I seem to remember fedora needing slightly less setup, but that was years ago).
You will need to enable uploads with the line write_enable=YES in /etc/vsftpd.conf. There are various ways to handle the uploads, the simplest one would be to have it log in with your user account, it will then dump images in your home directory (or a path you specify in the camera config).
You should then be able to open the images normally, ie with PIL.
If you don't want to set up a fileserver, you can try grabbing data directly with urllib2, see this page for how to handle the login. There is some chance by fooling around with fetching data you will be able to extract a video stream, but I think the ftp option will be a lot easier.
I am not familiar with exactly how the dlink DCS-30 works, but I have an earlier-generation model, the dlink DCS-20, and had the same objective, so maybe you can leverage my DCS-20 solution, or parts of it, to solve same for the DCS-30.
The key was just parsing the HTML provided by the built-in web browser access.
External modules requests, PIL, and BeautifulSoup simplify the solution.
Assuming your camera IP is 192.168.0.20, and that you've set up via the webadmin a user login to the camera of user1/pw1, here's the crux of the solution:
from StringIO import StringIO
import requests
from PIL import Image
from bs4 import BeautifulSoup
DCS_IP = "192.168.0.20"
userauth = ('user1', 'pw1')
snapurl = "http://" + DCS_IP + "/top.htm"
r = requests.get(snapurl, auth=userauth)
soup = BeautifulSoup(r.content)
# There are several <img> tags in page, so use border=0 attribute of
# objective <img> to distinguish it
imgtag = soup.find_all("img", attrs={'border':0})
imgsrc = BeautifulSoup(str(imgtag[0])).img['src']
imgurl = "http://" + DCS_IP + "/" + imgsrc
img = requests.get(imgurl, auth=userauth)
i = Image.open(StringIO(img.content))
i.save("snapshot.png")
Once you've retrieved the image (i), you can manipulate further with PIL, or, afterwards, use ffmpeg to, for example, stitch resulting image set into a time-lapse video.
HTH
Related
Forgive me if I've left anything out or goofed up formatting conventions; this is my first time posting on this sort of forum.
So I've got a Nikon D5600 that I'm using as part of an (extremely basic) image analysis setup. I'd like to be able to use images from it without having to manually transfer the files over each time I run a test, but I've had some trouble getting access to the files.
To be clear, I don't want to capture screenshots of a video; I understand that this is possible, but the resolution is about 1/3 smaller in video, which is a bit of an issue for my application.
So, when I was 6 hours more naive, I plugged in the camera via USB to my (Windows 10) desktop, tried calling the image using the exact (well, I did change the slashes out) file path windows gave me in the properties screen:
img = cv2.imread("This PC/D5600/Removable storage/DCIM/314D5600/CFW_0031.jpg")
That didn't work.
I checked that the command I was using wasn't the issue by copying the picture to another drive:
img = cv2.imread("D:/CFW_0031.jpg")
That worked.
So I think, and think is a bold claim here, that it's something to do with the "This PC" bit of the path. I've read some old (circa 2009) posts about MTP and such things, but I'm honestly not sure if that's even what this camera uses, or how to get started with that if it is in fact the correct protocol.
I've also tried using pygrabber (I believe it's a wrapper of direct show, though my terminology may be wrong) to control the camera via python, but that also didn't work, although I did manage to control my webcam, which was interesting.
Finally, I attempted to set the assign a letter drive to the camera, but found that the camera wasn't in the manager's list of discs. It's entirely possible I just did this method wrong, but I don't quite see how.
Edit regarding comment from Cristoph
-I just need to be able to use the image files in python, probably with opencv. I suppose that counts as reading them?
-I've attached a screenshot of what the "This PC" location looks like in the file explorer. The camera shows up under devices and drives, but doesn't have a drive letter.
I have been trying to build a project in which a Flask Application can automatically concatenate a selected amount of video's to a 'core video'.
User's can upload a video, which is sent to amazon s3 for storage.
All video's are preprocessed by Moviepy to be an mp4 file, running on 24 fps without audio, with a resolution of 720p.
After this preprocessing, the video is uploaded to amazon s3.
Of all new uploads in s3, a queue is created which an administrator can approve or delete.
All approved video's end up in a list that is concatenated with a current 'core video'.
This is done by using Pythons Moviepy library.
from moviepy.editor import VideoFileClip, concatenate_videoclips, AudioFileClip
videos_to_concat(VideoFileClip(core_video.s3_link))
for video in approved_videos:
videos_to_concat.append(VideoFileClip(video.s3_link))
result = concatenate_videoclips(videos_to_concat, method=compose)
Later, some audio is added under the full duration of the video.
result_with_audio = result.set_audio(some_audio.mp3)
The problem however, is that without throwing any errors, some videos are frozen after the first couple of frames after concatenation has been successfull. A frame remains stationary for the duration of the original clip. The audio keeps playing though. When a next clip is loaded, that one either plays normally or has the same behaviour of freezing after a couple of frames. There seems to be no obvious patern.
Initially I thought the mistake might be that ffmpeg does not download video's from the normal s3 link properly, but that would not explain why the the biggest video in the beginning and some other video's get rendered correctly, and some others aren't.
Could it that this is caused by a potential difference in codecs? (libx264 vs. mpeg4)?
Or is this way of accessing the files by URL and then directly feeding that to moviepy a potential cause of troubles? (VideoFileClip(https://amazon.s3.link.to.file.here.mp4)
Should I try to download all files and then locally concatenating them, or am I right to assume that the current approach should work.
When inspecting the files, nothing obvious like filename, filetype, resolution seems to be the issue, the preprocessing seems to do what it should.
Would love to hear any idea's on how the corruption of the resulting concatenated video could be explained and hopefully resolved.
Okay, I did manage to figure it out in the end. The problem was solved by downloading all video's with the boto3 client that amazon s3 provides for Python. Once downloading all video's to local storage of the webserver, concatenation worked without any issues.
I'm guessing that this might have something to do with s3 not serving the entire video file instantly through the link. In the end it seems quite logical to just use the provided s3 client to download store video's before performing any edits with moviepy.
The final goal would be to capture the regular webcam feed, manipulate it in some way (blur face, replace background, ...) and then output the result in some way so that the manipulated feed can be chosen as input for whatever application expects a webcam (Discord, Teams, ...).
I am working on a Windows machine and would prefer to do this in Python. This combination has me lost, at the moment.
capturing and manipulating is easy with https://pypi.org/project/opencv-python/
the exposing the feed step seems overly complicated
Apparently, on Linux there are Python libraries just offering that functionality, but they do not work on Windows. Everything that sounded like it could hint towards a good solution went directly into C++ country. There are programs which basically do what I want, e.g. webcamoid (https://webcamoid.github.io/) and I could hack together a solution which captures and processes the feed via Python, then uses webcamoid to record the output and feed it into a virtual webcam. But I'd much prefer to do the whole thing in one.
I have been searching around a bit and found these questions on stackoverflow on the topic:
Using OpenCV Output as Webcam (uses C++ but also gives a Python solution - however, pyfakewebcam does not work on Windows)
How do I stream to a new video source? (not really answered, just links to other question)
How to simulate a webcam device (more C++ hints, links to msdn's Writing a Custom Media Source)
Artificial webcam on windows (basically what I want, but in C++ again)
Writing a virtual webcam? (more explanation on how this might work in C++)
I am getting the strong impression that I need C++ for this or have to work on Linux. However, lacking both a Linux machine and any setup as well as experience in programming in C++, this seems like a large amount of work for the "toy project" this was supposed to be. But maybe I am just missing an obvious library or functionality somewhere?
Hence, the question is: Is there a way to expose a "webcam" stream via Python on Windows?
And, one last idea: What if I used a docker container with a Linux Python environment to implement the functionality I want. Could that container then stream a "virtual webcam" to the host?
You can do this by using pyvirtualcam
First, you need to install it using pip
pip install pyvirtualcam
Then go to This Link and download the zip file from the latest release
Unzip and navigate to \bin\[your computer's bittedness]
Open Command Prompt in that directory and type
regsvr32 /n /i:1 "obs-virtualsource.dll"
This will register a fake camera to your computer
and if you want to unregister the camera then run this command:
regsvr32 /u "obs-virtualsource.dll"
Now you can send frames to the camera using pyvirtualcam
This is a sample:
import pyvirtualcam
import numpy as np
with pyvirtualcam.Camera(width=1280, height=720, fps=30) as cam:
while True:
frame = np.zeros((cam.height, cam.width, 4), np.uint8) # RGBA
frame[:,:,:3] = cam.frames_sent % 255 # grayscale animation
frame[:,:,3] = 255
cam.send(frame)
cam.sleep_until_next_frame()
So here's the situation: I've got two Python programs, one to control a uEye camera module, making use of the SimpleCV library, and another to do a bit of analysis on the image. The reason for them being separate is that SimpleCV is 2.7, while a few modules I need to use in the analysis stage are for 3.X only.
The camera program will continuously capture and save an image to a location (rewriting the old image), which I've timed to be around every 30 ms. The analysis program takes in an image every 100 ms or so. Now, the issue I'm concerned with is that if the analysis program tries to read in the image while the camera program happens to be writing it, it will spring an error.
I'm fairly certain placing an exception statement to catch the OSError and have it simply try again would suffice, but I feel that is a bit forceful. I've also thought about having the camera program write a number (say, 100) of images, to lesson the odds that the two will happen to be working on the same file at once, but that seems unreliable. In a perfect world, I could ditch SimpleCV and go with a 3.X module, allowing the writing and reading to happen in sequence only, but I've yet to find a suitable replacement that works with the camera.
Any thoughts on the most efficient, robust way of avoiding this issue?
Here is the (simplified) camera program:
from SimpleCV import *
cam = Camera(0)
while True:
img = cam.getImage()
img.save("nav.jpg")
And the important part of the analysis program:
from PIL import Image
img = Image.open("nav.jpg")
The easiest way is to open the file with exclusive access so no-one can have it open for the duration of you working with it. See What is the best way to open a file for exclusive access in Python? for implementation details.
Be sure to file.close() or with <file_open> as f to close the file as soon as you can to minimize interference with the agents that "continuously update" it. Yes, and be sure to handle the file locked case in those agents.
I'm trying to download a spreadsheet from Google Drive inside a program I'm writing (so the data can be easily updated across all users), but I've run into a few problems:
First, and perhaps foolishly, I'm only wanting to use the basic python distribution, so I'm not requiring people to download multiple modules to run it. The urllib.request module seems to work well enough for basic downloading, specifically the urlopen() function, when I've tested it on normal webpages (more on why I say "normal" below).
Second, most questions and answers on here deal with retrieving a .csv from the spreadsheet. While this might work even better than trying to parse the feeds (and I have actually gotten it to work), using only the basic address means only the first sheet is downloaded, and I need to add a non-obvious gid to get the others. I want to have the program independent of the spreadsheet, so I only have to add new data online and the clients are automatically updated; trying to find a gid programmatically gives me trouble because:
Third, I can't actually get the feeds (interface described here) to be downloaded correctly. That does seem to be the best way to get what I want—download the overview of the entire spreadsheet, and from there obtain the addresses to each sheet—but if I try to send that through urlopen(feed).read() it just returns b''. While I'm not exactly sure what the problem is, I'd guess that the webpage is empty very briefly when it's first loaded, and that's what urlopen() thinks it should be returning. I've included what little code I'm using below, and was hoping someone had a way of working around this. Thanks!
import urllib.request as url
key = '1Eamsi8_3T_a0OfL926OdtJwLoWFrGjl1S2GiUAn75lU'
gid = '1193707515'
# Single sheet in CSV format
# feed = 'https://docs.google.com/spreadsheets/d/' + key + '/export?format=csv&gid=' + gid
# Document feed
feed = 'https://spreadsheets.google.com/feeds/worksheets/' + key + '/private/full'
csv = url.urlopen(feed).read()
(I don't actually mind publishing the key/gid, because I am planning on releasing this if I ever finish it.)
Requres OAuth2 or a password.
If you log out of google and try again with your browser, it fails (It failed when I did logged out). It looks like it requires a google account.
I did have it working with and application password a while ago. But I now use OAuth2. Both are quite a bit of messing about compared to CSV.
This sounds like a perfect use case for a wrapper library i once wrote. Let me know if you find it useful.