I've been struggling for quite some time to find a static dataflow graph generator for Python.
This is my ideal:
Given a small python script example.py, (written in Python3), return some representation of the data flow graph.
I was able to achieve this result using IBM's pyflowgraph, https://github.com/IBM/pyflowgraph which outputs data in graph.ml format, unfortunately this package only performs dynamic analysis.
I'm wondering if anyone knows of a DFG tool that could do this type of static dataflow analysis for Python?
I just found this open source project focused on dataflow analysis for Python. Check it out!
https://github.com/SMAT-Lab/Scalpel/
It's made in Python too; haven't used it, but looks very interesting!
This is the pre-print of their paper:
https://arxiv.org/pdf/2202.11840.pdf
Related
I am currently working on a project where I need to create mel-spectrograms to classify WAV audio-files with a neuronal network. In order to have a valid input to train my network, I first have to convert these audio-files into a mel-spectrogram. To do so I am currently using librosa.feature.melspectrogram and this works perfectly fine.
Part of the project is also to classify unknown WAV audio-files in a webrowser. I am using ONNX.js for this task and this works also perfectly fine.
The problem is now that I need to create the exact same mel-spectrogram like the server would do in Python with librosa. Otherwise the input would be different and therefore the output/prediction as well.
So my question is: is there any library in JavaScript that allows one to create a mel-spectrogram that is identical for the same WAV-audio-file both on client-side (JS) and server-side (Python)?
Is there a port of librosa for JS? Any other ideas are also welcomed, e.g. changing the server-library just to use a library that works in both languages (JS and Python) and gave the exact same result for a given WAV.
I already considered TensorFlow.js but there is implemented only a subset of TensorFlow in JavaScript (tfio.experimental.audio.melscale exists only in Python server-side).
I have a ROS/CPP simulator that saves large amounts of data to a rosbag (around 90 MB). I want to read this data frequently from Python and since reading rosbags is slow and cumbersome, I currently have another python script that reads the rosbag and saves the relevant contents to a HDF5 file.
It would be nice though to be able to just save the data from the simulator directly (in C++) and then read it from my scripts (in Python). So I was wondering which data format I should use.
It should be:
Fast to load from Python
Be compact (so ideally a binary of some sort)
Be easy to use
You might be wondering why I don't just save to HDF5 from my C++ simulator, but it just doesn't seem to be easy. There is basically nothing on forums such as Stackoverflow and the HDF5 Group website is opaque, seems to have some complicated licensing and very poor examples. I just want something quick and dirty that I can get running this afternoon.
You may want to have a look at HDFql as it is a high-level language (similar to SQL) to manage HDF5 files. Amongst others, HDFql supports C++ and Python. There are some examples that illustrates how to use HDFql in these languages here.
I see two solutions that can be useful for your problem :
LV: Length Value that you can store directly in binary into a file.
JSON: This does not add many data more than you need, and there are many libraries in Python or C++ that can simplify you the work
Protocol Buffers is an option with language bindings in C++ and Python, though it might be more time investment than quick/dirty running this afternoon.
I want to create a program, which automates excel reporting including various graphs in colours. The program needs to be able to read an excel dataset. Based on this dataset, the program then has to create report pages and graphs and then export to an excel file as well as pdf file.
I have done some research and it seems this is possible using python with pandas - xlsxWriter or xlswings as well as Ruby gems - axlsx or win32ole.
Which is the user-friendlier and easy to learn alternative? What are the advantages and disadvantages? Are there other options I should consider (I would like to avoid VBA - as this is how the reports are currently produced)?
Any responses and comments are appreciated. Thank you!
If you already have VBA that works for your project, then translating it to Ruby + WIN32OLE is probably your quickest path to working code. Anything you can do in VBA is doable in Ruby (if you find something you can't do, post here to ask for help).
I prefer working with Excel via OLE since I know the file produced by Excel will work anywhere I open it. I haven't used axlsx but I'm sure it's a fine project; I just wouldn't trust that it would produce working Excel files every time.
I'm building a web application which will include functionality that takes MS Word (and possibly input from a web-based rich text editor) documents, substitutes values into the formfield placeholders in those documents, and generates a PCL document as output.
I'm developing in python and django on windows, but this whole solution will need to be deployed to a web host (yet to be chosen), which in practice means that the solution will need to run on linux.
I'm open to linux-only solutions if that's the only way. I'm open to solutions that involve talking to a server written in another language. I am able to write C++ or java if necessary to get this done. The final output does have to be in PCL format.
My question is: what is a good tool chain for generating PCL from word documents using python?
I am considering using some kind of interface to openoffice to open the word documents, do the substitutions, and send the output to some kind of printer driver. Does anyone have experience with this? What libraries would you recommend?
Options for interfacing that I have identified include the following; any other suggestions would be greatly welcomed:
Ulif.openoffice: http://pypi.python.org/pypi/ulif.openoffice/0.4
Py3o.renderserver: https://bitbucket.org/faide/py3o.renderserver
OpenOffice-python: http://openoffice-python.origo.ethz.ch/
A second approach would be to use something like paradocx ( https://bitbucket.org/yougov/paradocx/wiki/Home ) to open the word files, do the substitutions using that in python, then somehow interface with something that can output PCL. Again, any experience or comments on this approach would be appreciated.
I will very much appreciate any comments on tools and toolchains, and ideas or recipes that you may have.
This question covers similar ground to, but is not the same as: How to Create PCL file from MS word
Ghostscript can read PS (Postscript) or PDF and create PCL. You can use python libraries or just subprocess....
OK, so my final solution involved creating a java webservice to perform my transcoding.
Docx4j provides a class org.docx4j.convert.out.pdf.viaXSLFO.Conversion which hooks into apache FOP to convert Docx to PDF; that can be easily hacked to convert to PCL (because FOP outputs PCL)
Spark is a lightweight java web framework which allowed me to wrap my transcoder in a web service
Because I also manipulate the document, I need to have some metadata, so the perfect thing is a multipart form. I decode that using Apache Fileupload
In almost all cases, I had to upgrade to the development versions of libraries to get this to work.
On the python side I use:
requests to communicate with the web service
poster to prepare the multi-part request
Matlab has this great tool called publish. This tool converts a regular matlab script with minimal formatting stuff into structured, nice looking reports (HTML, LateX, RTF). It is capable of handling graphics, mathematical formulae etc.
Is there a similar tool for Python?
There's pyreport. It captures the "captures its output, compiling it to a pretty report in a pdf or an html file". With this, you can write scripts that print out your source, getting a nice report in the end.