I am new to Windows Media Foundation, I am currently following some tutorials on the basics to get started and decoding audio. However, I am running int a few issues. Using: Windows 10 64bit (1809), and am using Python (ctypes and COM's to interface).
1) The IMFSourceReader will not allow me to select or deselect any stream out. I have tried wav and mp3 formats (and multiple different files), but they all error out. According to the docs, to speed up performance you want to deselect other streams and select the stream you want, in this case, audio.
However:
source_reader.SetStreamSelection(MF_SOURCE_READER_ANY_STREAM, False)
Produces an error:
OSError: [WinError -1072875853] The stream number provided was invalid.
Which should be correct since MF_SOURCE_READER_ANY_STREAM value (DWORD of 4294967294) should be universal? Or am I incorrect in that?
I've tried seeing if I could just select the audio stream:
source_reader.SetStreamSelection(MF_SOURCE_READER_FIRST_AUDIO_STREAM, True)
Which produces a different error:
OSError: exception: access violation reading 0x0000000000000001
My current code up until that point:
MFStartup(MF_VERSION) # initialize
source_reader = IMFSourceReader()
filename = "C:\\test.mp3"
MFCreateSourceReaderFromURL(filename, None, ctypes.byref(source_reader)) # out: source reader.
if source_reader: # Not null
source_reader.SetStreamSelection(MF_SOURCE_READER_ANY_STREAM, False) # invalid stream #?
source_reader.SetStreamSelection(MF_SOURCE_READER_FIRST_AUDIO_STREAM, True) # access violation??
IMFSourceReader seems to be functioning just fine for other functions, Such as GetCurrentMediaType, SetCurrentMediaType, etc. Could it still return IMFSourceReader if there are any issues?
2) I am not sure if not being able to select the streams is causing further issues (I suspect it is). If I just skip selecting or deselecting streams, everything actually works up until trying to convert a sample into a single buffer with ConvertToContiguousBuffer, which, according to the docs, outputs into a IMFMediaBuffer. The problem is, after running that, it does return as S_OK, but the buffer is null. I used GetBufferCount to make sure there are some buffers in the sample atleast, and it always returns 1-3 depending on the file used, so it shouldn't be empty.
Here is the relevant code:
while True:
flags = DWORD()
sample = IMFSample()
source_reader.ReadSample(streamIndex, 0, None, ctypes.byref(flags), None, ctypes.byref(sample)) # flags, sample [out]
if flags.value & MF_SOURCE_READERF_ENDOFSTREAM:
print("READ ALL OF STREAM")
break
if sample:
buffer_count = DWORD()
sample.GetBufferCount(ctypes.byref(buffer_count))
print("BUFFER COUNT IN SAMPLE", buffer_count.value)
else:
print("NO SAMPLE")
continue
buffer = IMFMediaBuffer()
hr = sample.ConvertToContiguousBuffer(ctypes.byref(buffer))
print("Conversion succeeded", hr == 0) # true
if buffer:
print("CREATED BUFFER")
else:
print("BUFFER IS NULL")
break
I am unsure where to go from here, I couldn't find much explanations on the internet regarding these specific issues. Is WMF still the goto for Windows 10? Should I be using something else? I really am stumped and any help is greatly appreciated.
Try using MF_SOURCE_READER_ALL_STREAMS instead of MF_SOURCE_READER_ANY_STREAM.
source_reader.SetStreamSelection(MF_SOURCE_READER_ALL_STREAMS, False)
When reading the samples you also need to specify a valid stream index which in your case I suspect 0 isn't. Try:
source_reader.ReadSample((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, 0, None, ctypes.byref(flags), None, ctypes.byref(sample))
Also does your Python wrapper return a result when you make your Media Foundation calls? Nearly all Media Foundation methods return a HRESULT and it's important to check it equals S_OK before proceeding. If you don't it's very hard to work out where a wrong call occurred.
Is WMF still the goto for Windows 10?
Many people have asked that question. The answer depends on what you need to do (in your case audio codec support is very limited so perhaps it isn't the best option). But for things like rendering audio/video, reading/writing to media files, audio/video device capture etc. it is still the case that Media Foundation is the most up to date Microsoft supported option.
Usually, with MediaFoundation, you need to call CoInitializeEx before MFStartup :
Media Foundation and COM
Best Practices for Applications
In Media Foundation, asynchronous processing and callbacks are handled by work queues. Work queues always have multithreaded apartment (MTA) threads, so an application will have a simpler implementation if it runs on an MTA thread as well. Therefore, it is recommended to call CoInitializeEx with the COINIT_MULTITHREADED flag.
MFCreateSourceReaderFromURL function
Remarks
Call CoInitialize(Ex) and MFStartup before calling this function.
In your code, I don't see the call to CoInitializeEx.
EDIT
Because you are using audio file, normally you should have only one audio stream, and index should be 0.
Try :
source_reader.SetStreamSelection(0, True)
and tell us the result.
Related
I am using the built in lzma python to decode compressed chunk of data. Depending on the chunk of data, I get the following exception :
Compressed data ended before the end-of-stream marker was reached
The data is NOT corrupted. It can be decompressed correctly with other tools, so it must be a bug in the library. There are other people experiencing the same issue:
http://bugs.python.org/issue21872
https://github.com/peterjc/backports.lzma/issues/6
Downloading large file in python error: Compressed file ended before the end-of-stream marker was reached
Unfortunately, none seems to have found a solution yet. At least, one that works on Python 3.5.
How can I solve this problem? Is there any work around?
I spent a lot of time trying to understand and solve this problem, so i thought it would a good idea to share it. The problem seems to be caused by the a chunk of data without the EOF byte properly set. In order to decompress a buffer, I used to use the lzma.decompress provided by the lzma python lib. However, this method expects each data buffer to contains a EOF bytes, otherwise it throws a LZMAError exception.
To work around this limitation, we can implement an alternative decompress function which uses LZMADecompress object to extract the data from a buffer. For example:
def decompress_lzma(data):
results = []
while True:
decomp = LZMADecompressor(FORMAT_AUTO, None, None)
try:
res = decomp.decompress(data)
except LZMAError:
if results:
break # Leftover data is not a valid LZMA/XZ stream; ignore it.
else:
raise # Error on the first iteration; bail out.
results.append(res)
data = decomp.unused_data
if not data:
break
if not decomp.eof:
raise LZMAError("Compressed data ended before the end-of-stream marker was reached")
return b"".join(results)
This function is similar to the one provided by the standard lzma lib with one key difference. The loop is broken if the entire buffer has been processed, before checking if we reached the EOF mark.
I hope this can be useful to other people.
I have a data acquisition system that produces ASCII data. The data is acquired over USB with serial communication protocol (virtual serial, as the manufacturer of the box claims). I have a Python program/script that uses PySerial with a PySide GUI that plots the acquired data and saves it to HDF5 files. I'm having a very weird problem and I don't know how to tackle it. I wish you guys could help me and provide advice on how you would debug this problem.
How the problem shows up: The problem is that if I use a software like Eltima Data Logger, the data acquired looks fine. However, if I use my software (with PySerial), some chunks of the data seems to be missing. What's weird is that the missing chunk of the data is incompatible with the method of reading. I read line by line, and what is missing from the data is like 100 bytes or 64 bytes chunks that sometimes include newlines!!! I know what's missing because the device buffers the data on an SD Card before sending it to the computer. This made me believe for a long time that the hardware has a problem, until I used this software, Eltima, that showed that it's acquiring the data fine.
The following is the configuration of Eltima:
My configuration:
This whole thing is running in a QThread.
The following is the methods I use in my code (with some minor polishing to make it reusable here):
self.obj = serial.Serial()
self.obj.port = instrumentName
self.obj.baudrate = 115200
self.obj.bytesize = serial.EIGHTBITS
self.obj.parity = serial.PARITY_ODD
self.obj.stopbits = serial.STOPBITS_ONE
self.obj.timeout = 1
self.obj.xonxoff = False
self.obj.rtscts = False
self.obj.dsrdtr = False
self.obj.writeTimeout = 2
self.obj.open()
The algorithm I use for reading, is that I have a loop that looks for a specific header line, and once found, it keeps pushing lines into buffer until a specific end line is found; and this data is finally processed. Following is my code:
try:
# keep reading until a header line is found that indicates the beginning of a batch of data
while not self.stopped:
self.line = self.readLine()
self.writeDumpFileLine(self.line)
if self.line == DataBatch.d_startString:
print("Acquiring batch, line by line...")
self.dataStrQueue.append(self.line)
break
# after the header line, keep reading until a specific string is found
while not self.stopped:
self.line = self.readLine()
self.writeDumpFileLine(self.line)
self.dataStrQueue.append(self.line)
if self.line == DataBatch.d_endString:
break
except Exception as e1:
print("Exception while trying to read. Error: " + str(e1))
The self.writeDumpFileLine() takes the line from the device and dumps it in a file directly before processing for debugging purposes. These dump files have confirmed the problem of missing chunks.
The implementation of self.readLine() is quite simple:
def readLine(self):
lineData = decodeString(self.obj.readline())
lineData = lineData.replace(acquisitionEndlineChar, "")
return lineData
I would like to point out that I also have an implementation that pulls thousands of lines and parses them based on inWaiting(), and this method has the same problem too!
Now I'm starting to wonder: Is it PySerial? What else could be causing this problem?
Thank you so much for any efforts. If you require any additional information, please ask!
UPDATE:
Actually I have just confirmed that the problem can be reproduced by getting the system to lag a little bit. I use PyCharm to program this software, and while the program is running, if I press Ctrl+S to save, the GUI of PyCharm freezes a little bit (and hence its terminal). Repeating this many times causes the problem in a reproducible manner!!!!
My medical office has a network drive (run off a DLink-325 NAS, if that makes any difference) which holds all the documents for our patients. Each patient has a folder with their name, and each folder holds those documents. The problem is that the Windows file explorer is not well suited to this sort of situation - users can accidentally rename folders (throwing off the sort), move patient folders, etc. So, I'm working on developing a python app (built on django for the web interface) and I'm to the point where I need to make sure the django database matches with the file system so that whenever something does change with the file/folder names, the database points to the correct files. I read Tim Golden's article and thought that would be an excellent solution. And it was - as long as I was testing it on local folders. However, as soon as I pointed it to the network drive, I ran into this strange error:
Traceback (most recent call last): File "C:\projects\RecordKeeper\test.py", line 50, in <module>
None pywintypes.error: (58, 'ReadDirectoryChangesW', 'The specified server cannot perform the requested operation.') [Finished in 16.4s]
Oddly enough, this only happens when the file name is changed to something new. If I change the file name, then change it back, everything works perfectly.
For instance, if the original file name is "Referral.pdf" and I change it to "Raferral.pdf" (just changed the "e" to an "a"), I get the error. However, if I change it back to "Referral.pdf", I get the proper output. Here is my code (I've changed the folder name to protect patient health information):
import os
import win32file
import win32con
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something",
5 : "Renamed to something"
}
# Thanks to Claudio Grondi for the correct set of numbers
FILE_LIST_DIRECTORY = 0x0001
#path_to_watch = "C:\projects\working"
path_to_watch = "Z:\_Charts\Abruzzo, Sandra J - 1947-11-18"
hDir = win32file.CreateFile (
path_to_watch,
FILE_LIST_DIRECTORY,
win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE | win32con.FILE_SHARE_DELETE,
None,
win32con.OPEN_EXISTING,
win32con.FILE_FLAG_BACKUP_SEMANTICS,
None
)
while 1:
#
# ReadDirectoryChangesW takes a previously-created
# handle to a directory, a buffer size for results,
# a flag to indicate whether to watch subtrees and
# a filter of what changes to notify.
#
# NB Tim Juchcinski reports that he needed to up
# the buffer size to be sure of picking up all
# events when a large number of files were
# deleted at once.
#
try:
results = win32file.ReadDirectoryChangesW (
hDir,
1024,
True,
win32con.FILE_NOTIFY_CHANGE_FILE_NAME |
win32con.FILE_NOTIFY_CHANGE_DIR_NAME |
win32con.FILE_NOTIFY_CHANGE_ATTRIBUTES |
win32con.FILE_NOTIFY_CHANGE_SIZE |
win32con.FILE_NOTIFY_CHANGE_LAST_WRITE |
win32con.FILE_NOTIFY_CHANGE_SECURITY,
None,
None
)
for action, file in results:
full_filename = os.path.join (path_to_watch, file)
print full_filename, ACTIONS.get (action, "Unknown")
except Exception as e:
print "Unexpected error:", e
UPDATE
I added try/catch block to the code above to troubleshoot this problem a little more. It seems just the renaming stuff that's causing a problem. It still can tell if the file was updated. Here's the output I got during one test:
Z:\_Charts\Abruzzo, Sandra J - 1947-11-18\2013-06-12 Request for records.pdf Renamed from something
Z:\_Charts\Abruzzo, Sandra J - 1947-11-18\2013-06-12 Requeast for records.pdf Renamed to something
Z:\_Charts\Abruzzo, Sandra J - 1947-11-18\2013-06-12 Requeast for records.pdf Updated
Unexpected error: (58, 'ReadDirectoryChangesW', 'The specified server cannot perform the requested operation.')
Z:\_Charts\Abruzzo, Sandra J - 1947-11-18\2013-06-12 Request for records.pdf Updated
As you can see, it handled the first rename with no problem. But when I tried to rename it back, the rename operation threw an error, but the update notification worked perfectly fine. Anyone have any ideas given these new wrinkles?
UPDATE
I think I've figured out a solution. As many users have said, actually getting accurate data from the network drive all the time is not going to happen. However, the program did at least report every time a file was updated, even if it threw an error when getting the rename data. So, I wrote several special methods that went through the database, found records that no longer pointed to valid files, and updated those records. Hopefully that will do it. If anyone has a better solution, please feel free to share! :D
According to this blog, NAS devices typically do not support the required functionality to support the ReadDirectoryChangesW function, so it sounds like the problem is with your drive, not your python code. This accompanying page mentions a couple of other functions which offer similar functionality and their various pros and cons, but doesn't go into details such as whether they work over network drives.
I am trying to read only one file from a tar.gz file. All operations over tarfile object works fine, but when I read from concrete member, always StreamError is raised, check this code:
import tarfile
fd = tarfile.open('file.tar.gz', 'r|gz')
for member in fd.getmembers():
if not member.isfile():
continue
cfile = fd.extractfile(member)
print cfile.read()
cfile.close()
fd.close()
cfile.read() always causes "tarfile.StreamError: seeking backwards is not allowed"
I need to read contents to mem, not dumping to file (extractall works fine)
Thank you!
The problem is this line:
fd = tarfile.open('file.tar.gz', 'r|gz')
You don't want 'r|gz', you want 'r:gz'.
If I run your code on a trivial tarball, I can even print out the member and see test/foo, and then I get the same error on read that you get.
If I fix it to use 'r:gz', it works.
From the docs:
mode has to be a string of the form 'filemode[:compression]'
...
For special purposes, there is a second format for mode: 'filemode|[compression]'. tarfile.open() will return a TarFile object that processes its data as a stream of blocks. No random seeking will be done on the file… Use this variant in combination with e.g. sys.stdin, a socket file object or a tape device. However, such a TarFile object is limited in that it does not allow to be accessed randomly, see Examples.
'r|gz' is meant for when you have a non-seekable stream, and it only provides a subset of the operations. Unfortunately, it doesn't seem to document exactly which operations are allowed—and the link to Examples doesn't help, because none of the examples use this feature. So, you have to either read the source, or figure it out through trial and error.
But, since you have a normal, seekable file, you don't have to worry about that; just use 'r:gz'.
In addition to the file mode, I attempted to seek on a network stream.
I had the same error when trying to requests.get the file, so I extracted all to a tmp directory:
# stream == requests.get
inputs = [tarfile.open(fileobj=LZMAFile(stream), mode='r|')]
t = "/tmp"
for tarfileobj in inputs:
tarfileobj.extractall(path=t, members=None)
for fn in os.listdir(t):
with open(os.path.join(t, fn)) as payload:
print(payload.read())
Basically i am trying to find out what version of ArcGIS the user currently has installed, i looked through the registry and couldn't find anything related to a version string. However i know it is stored, within the .exe.
I've done a fair bit of googling, and can't find anything really worth it. I tried using the GetFileVersionInfo, and i seem to get a random mishmash of stuff.
Any ideas?
EDIT
Sigh....
Turns out pywin32 is not always installed on all machines. Does anyone know if its possible to do the same thing via ctypes?
Also this is only for windows.
If you prefer not to do this using pywin32, you would be able to do this with ctypes, for sure.
The trick will be decoding that silly file version structure that comes back.
There's one old mailing list post that is doing what you're asking. Unfortunately, I don't have a windows box handy to test this myself, right now. But if it doesn't work, it should at least give you a good start.
Here's the code, in case those 2006 archives vanish sometime:
import array
from ctypes import *
def get_file_info(filename, info):
"""
Extract information from a file.
"""
# Get size needed for buffer (0 if no info)
size = windll.version.GetFileVersionInfoSizeA(filename, None)
# If no info in file -> empty string
if not size:
return ''
# Create buffer
res = create_string_buffer(size)
# Load file informations into buffer res
windll.version.GetFileVersionInfoA(filename, None, size, res)
r = c_uint()
l = c_uint()
# Look for codepages
windll.version.VerQueryValueA(res, '\\VarFileInfo\\Translation',
byref(r), byref(l))
# If no codepage -> empty string
if not l.value:
return ''
# Take the first codepage (what else ?)
codepages = array.array('H', string_at(r.value, l.value))
codepage = tuple(codepages[:2].tolist())
# Extract information
windll.version.VerQueryValueA(res, ('\\StringFileInfo\\%04x%04x\\'
+ info) % codepage, byref(r), byref(l))
return string_at(r.value, l.value)
print get_file_info(r'C:\WINDOWS\system32\calc.exe', 'FileVersion')
--
Ok - back near a windows box. Have actually tried this code now. "Works for me".
>>> print get_file_info(r'C:\WINDOWS\system32\calc.exe', 'FileVersion')
6.1.7600.16385 (win7_rtm.090713-1255)
there's a gnu linux utility called 'strings' that prints the printable characters in any file(binary or non-binary), try using that and look for a version number like pattern
on windows, you can get strings here http://unxutils.sourceforge.net/