I'm working on a regression-testing tool that will validate a very large number of Excel spreadsheets. At the moment I control them via COM from a Python script using the latest version of the pywin32 product. Unfortunately COM seems to have a number of annoying drawbacks:
For example, the slightest upset seems to be able to break the connection to the COM-Server, once severed there seems to be no safe way to re-connect to the Excel application. There's absolutely no safety built into the COM Application object.
The Excel COM interface will not allow me to safely remote-control two seperate instances of the Excel application operating on the same workbook file, even if they are read-only.
Also when something does go wrong I seldom get any useful error-messages... at best I can except a numerical error-code or a barely useful message such as "An Exception has occurred". It's almost impossible to know why something went wrong.
Finally, COM lacks the ability to control some of the most fundamental aspects of Excel? For example there's no way to do a guaranteed close of just the Excel process that a COM client is connected to. You cannot even use COM to find Excel's PID.
So what if I were to completely abandon COM? Is there an alternative way to control Excel?
All I want to do is run macros, open and close workbooks and read and write cell-ranges? Perhaps some .NET experts know a trick or two which have not yet bubbled into the Python community? What about you office-hackers? Could there be a better way to get at Excel's innards than COM?
There is no way that completely bypasses COM. You can use VSTO (Visual Studio Tools for Office), which has nice .NET wrappers on the COM objects, but it is still COM underneath.
The Excel COM interface will not allow me to safely remote-control two seperate instances of the Excel application operating on the same workbook file, even if they are read-only.
This is not a limitation of COM, this is a limitation of Excel. Excel will not even let you open two files with the same name at the same time if they exist in different directories. It is a fundamental limitation of the Excel program.
To answer your other questions
If you check your python documentation, there should be a way to connect to an existing server if the connection is lost.
The lack of useful error messages again may be to do with Python.
You cannot even use COM to find Excel's PID.
COM is an internal object model and exposed what it wishes. PID are available to outside processes as much as they are to internal, there is no real reason to expose as a COM interface.
It is also possible to run Excel as a server application and use it as a calculation engine. This allows non IT users to specify business rules within Excel and call them through webservices. I have not worked with this myself, but I know a coworker of mine used this once. Walkthrough: Developing a Custom Application Using Excel Web Services could be a good starting point. A first glance at that page looks like it requires Sharepoint. This might not be suiteable for every environment.
Have you looked at the xlrd and xlwt packages? I'm not in need of them any more, but I had good success with xlrd on my last project. Last I knew, they couldn't process macros, but could do basic reading and writing of spreadsheets. Also, they're platform independent (the program I wrote was targetted to run on Linux)!
You could use Jython with the JExcelApi (http://jexcelapi.sourceforge.net/) to control your Excel application. I've been considering implementing this solution with one of my PyQt projects, but haven't gotten around to trying it yet. I have effectively used the JExcelApi in Java applications before, but have not used Jython (though I know you can import Java classes).
NOTE: the JExcelApi may be COM under the hood (I'm not sure).
Related
We are using software called Enovia.
I would like to query Enovia from inside Excel / VBA.
After some search, Enovia seems to have a programming interface called Abaqus.
http://www.3ds.com/products-services/simulia/services/training-courses/course-descriptions/introduction-to-the-abaqus-scripting-interface/
I would like to access this interface in VBA directly but I didn't find anything to that effect.
I found two projects that might allow Excel/VBA to talk to python components or at least to python scripts.
They are as follows
Python For Excel
http://opentradingsystem.com/PythonForExcel/main.html
And the discontinued excelpython
github.com/ericremoreynolds/excelpython
which became integrated in xlwings
xlwings.org
I am not familiar with python nor these two projects. I'm not sure if they will allow me to do what I want.
I'm ok with learning some python to do this but my main program and user interface is in excel.
I figure I could make some simple python scripts then call them through the python interpreter from excel with shellexecute but that is not a very elegant solution I think.
Any advice welcomed
thanks !
Enovia uses the MQL language for scripting. There are some guides available, such as http://emdros.org/MQL-Programmers-Guide.pdf
I would recommend some training such as http://www.techniatranscat.com/services/education/mql-and-tcl-intro as this doesnt seem to be a very common language.
Not to be confused with the financial trading language MQL4/MQL5 from MetaQuotes Software Corp.
ENOVIA does not have a Python programming interface.
Abaqus is a FEA simulation software which can be scripted with its own Python programming interface.
ENOVIA though, is a PLM software solution. They are different software for completely different applications. As programming interfaces you may look at maybe this
In short, you may need to contact your Dassault Systemes local support team to point you in the right direction.
Using tcl and mql you can solve the problem
TCL |
MQL
Latest Enovia version 3DEXPERIENCE support Rest based webservices. you can create a rest API in enovia and can call rest service using vb script in excel.
3DEXPERIENCE Enovia Rest webservice Framework
Above mentioned Rest framework may not be available with 2014X version. You have to try SOAP based service or JAX-RS.
Option 1
Enovia is java based application and doesn't support VBA ADK to directly work with backend data. However webservice can be used communicate between applications of different platforms or managed by different programming ADK.
For this you need to create your own service (REST/SOAP). For this you will need server access and first you have to write some java code (Rest/SOAP service) and update the class/jar into 2014 server (tomcat).
Then you can make use of that service in to your vba function.
Again you have to decide the kind of data whether xml or json it is separate topic.
Option 2
As I told earlier Enovia is Java based application, It provides JAVA ADK to extend its functionality.
Sample code shows how you can get the context and work with Enovia ADK.
matrix.db.Context ctx = new matrix.db.Context("http://myserver-nocas:8080/enovia");
ctx.setUser("Test Everything");
ctx.setPassword("");
ctx.connect();
System.out.println("Is context set "+ctx.isConnected());
As you mentioned you are currently using 3DEXPERIENCE then 3DPassport might block above URL method connection, So you need to pass NO-CAS URL.
You can get the context by running above java code from your local eclipse. Using this ADK you can also call MQL queries and work seamlessly.
import com.matrixone.apps.domain.util.MqlUtil;
String strPartFamily = MqlUtil.mqlCommand(ctx,"temp query bus Part * * select attribute[Length]");
But again this is java based solution and not VBA.
Finally 3DEXPERIENCE provides MQL console in server, to play around with all the data using SQL like queries in more developer friendly manner. Enovia also provides web based MQL console but you need to have some license for that (Administrator should be having access).
Use TCL to go through Excel sheet, it is very easy. Use "/t" to separate data.
I am developing a rather large python application (wxpython) that allows a workflow of data analysis. Performing all steps of the workflow can be quite long and the user is not likely to everything at once. More likely he would prefer to do different parts of the processing at different points in time. It would therefore be very handy to be able to store the application's current status with some sort of "save project" functionality. Opening the application and loading a project file would set up the application as it was previously and allow one to continue where he/she left off last time.
However I have a large amount of objects to save, most of which are imbued with attributes coming from wxpython. This causes pickle to fail with the following error:
TypeError: can't pickle PySwigObject objects
Does anyone has experience with this? What would be a best practice to obtain the required functionality? Are there libraries devoted to this?
Thanks you.
wxPython is a wrapper around a C++ library known as wxWidgets. So you cannot use normal Python serialization to save its state. However, you can use the persist library to save most widget's state: http://wxpython.org/Phoenix/docs/html/lib.agw.persist.html
I'm not sure when this library was added to wxPython, but I'm guessing it was with 2.9 or perhaps the latest version of 2.8. Otherwise you can probably find it in the latest version of 2.8's source.
As others have said, it's usually better to just save the process's state and then load that information back to the GUI when it's started.
This question already has answers here:
How can I sandbox Python in pure Python?
(7 answers)
Closed 9 years ago.
I'd like to make a website where people could upload their Python scripts. Of course I'd like to execute those scripts. Those scripts should do some interesting work. The problem is that people could upload scripts that could harm my server and I'd like to prevent that. What is the option to run arbitrary scripts without harming my system - actually without seeing my system at all? Thank you
"Can't be done."
Running arbitrary (untrusted) scripts and staying safe is a contradiction. You should go as far as using custom kernels, jails, vms, the like.
You can look at how http://codepad.org/about does it, it's a lot of work.
I dont know in earlier versions, in Python 3 you can create functions with access to a custom scope through types.FunctionType.
def f():
return __builtins__
f() # this will work because it has access to __builtins__
scope = {}
sandboxed = FunctionType(f.__code__,scope)
sandboxed() # will throw NameError, builtins is not defined
the returned function has only access to whatever you supplied in the scope dictionary. I wonder if still there are hacks around this.
there are quite a lot of web-server running untrusted python codes nowadays:
http://codepad.org/ (probably the most notorious pastebin for python codes)
http://codingbat.com/ (previously Javabat, name change to reflect Python addition)
http://appengine.google.com/ (host python code on Google's infrastructure)
http://www.spoj.pl/ the infamous Sphere Online Judge coding challenge
you may want to look at how they approached their problems.
or you may want to look at a different approach:
http://pyjs.org/ - pyjamas - python-to-javascript compiler (running client-side, switch the security problem to their side)
"Can't be done," is too harsh. JavaScript engines live in your web browser and they accept and run untrusted scripts safely. There's always the possibility of exploits, but in correct engine operation they are innocuous. There are even "slow script" checks that prevent infinite loops from denial-of-service attacking your browser, making those little alert dialogs.
Google App Engine runs a sandboxed version of the Python VM which effectively removes all the naughty native bits that let you get at the underlying system. To do this yourself in a safe manner would take some Python VM expertise.
For sanity, you could start off by removing all builtins and whitelisting the ones you want to allow users once you certify they don't touch the underlying system.
It feels like something somebody must have already done, but I don't know of any existing project that does it. :-/
I think the way to do this is to run those scripts in normal Python shell, but on a virtual machine. I might be biased, because my "job" is currently to play around with VMs (universities are great!).
A new VM instance can be created and started in seconds. If you keep a few around and replace only those that get broken, you have good service, absolute security and almost no effort.
But there is one thing: Virtually all web hosts today are virtual machines and they don't support another virtual machine inside. You need a real, physical server to do this.
Brett Cannon has a tentative design for doing this, last I knew, but it has not been developed. So unless you're looking to put a LOT of effort into making this happen, there currently isn't a solution publicly available.
Brett's blog is at: http://sayspy.blogspot.com/ if you want to try to read up on it, I couldn't find a direct link to his discussions about the new security design. I can't recall if I read his blog talking about it, or if it was in person where he mentioned it, sorry.
There used to be some restricted execution abilities, but they were dropped because they just didn't work.
It's not impossible to do, but it's not something that Python is able to do right now. It's something people would like, but it's not really a high priority from what I've seen.
trypython.org (BSD licensed source here) does a safe browser oriented version of such a sanbox in IronPython (via Silverlight/Moonlight). You may be able to mash together a headless version of this for use on a server -- but you could definitely let users distribute scripts between each other, or you could distribute these scripts to be executed within the plugin environment.
If you use Linux maybe seccomp is the solution, even the mode 2 is nicer. With those you can create a new process that will fail any syscall and can only read already existing file descriptors.
Maybe using also namespaces and cgroup would help, this can be done with ctypes.
You could try Ideone API - it allows Python 2 and Python 3
Somebody really needs to fix this "subjective questions evaluator"
I usually compile my functions in a DLL and call them from excel. That works fine (well, let's just say it works)
Unfortunatelly, python cannot be compiled. I know of py2exe but I don't know that it can make a DLL.
So, ..., is there any other way ? I appreciate all ideas and suggestions on the matter.
One way is to write a COM server in Python, and call that from Excel. There are tutorials describing Win32 COM servers, and screencasts on calling such servers from Excel.
This is probably not a possible solution for you, but there is the Resolver One spreadsheet application (which is like Excel combined with Python). It is not connected with Excel in any way, but claims to be compatible to some extent.
Im surprised nobody mentioned pyxll. From the website:
PyXLL is an Excel addin that enables functions written in Python to be
called in Excel. Python functions can either be exposed as Excel user
defined functions that can be called from worksheets, as custom menu
items, or as macros.
There is an Excel Addin that allows you to do this called Discovery Script at xefion.com.
It's free but not open source. It's also based on the IronPython implementation.
I don't know any py2dll similar to py2exe.
However, you could create a dll in C and use the Very High Level Layer to call your script. I don't know it is an acceptable solution for you. Just an idea.
I had to do this some years back. My solution was to run small Python server that exported the functions using SOAP, then call the functions using Visual Basic's SOAP library. The advantage is that you don't have to ship a Python environment with your spreadsheets. The disadvantage is that the clients will need a network connection.
With my luck this question will be closed too quickly. I see a tremendous possibility for a python application that basically is like a workbook. Imagine if you will that instead of writing code you select from a menu of choices. For example, the File menu would have an open command that lets the user navigate to a file or directory of file or a webpage, even a list of web pages and specify those as the things that will be the base for the next actions.
Then you have a find menu. The menu would allow easy access to the various parsing tools, regular expression and string tools so you can specify the thing you want to find within the files.
Another menu item could allow you to create queries to interact with database objects.
I could go on and on. As the language becomes more higher level then these types of features become easier to implement. There is a tremendous advantage to developing something like this. How much time is spent reinventing the wheel for mundane tasks? Programmers have functions that they have built to do many mundane tasks but what about democratizing the power offered by a tool like Python.
I have people in my office all of the time asking how to solve problems that seem intractable to them, but when I show them how with a few lines of code their problem is solvable except for the edge cases they become amazed. I deflect their gratitude with the observation that it is not really that hard except for being able to construct the right google search to identify the right package or library to solve the problem. There is nothing amazing about my ability to use lxml and sets to pull all bolded sections from a collection of say 12,000 documents and compare across time and across unique identifiers in the collection how those bolded sections have evolved/changed or converged. The amazing piece is that someone wrote the libraries to do these things.
What is the advantage to the community for something like this. Imagine if you would an interface that looks like a workbook but interacts with an app-store. So if you want to pull something from html file you go to the app store and buy a plug-in that handles the work. If the workbook is built robustly enough it could be licensed to a machine, the 'apps' would be tied to a particular workbook.
Just imagine the creativity that could be unleashed by users if they could get over the feeling that access to this power is difficult. You guys may not see this but I see Python being so close to being able to port to something like a workbook framework. Weren't the early spreadsheet programs nothing more than a frame around some Fortran libraries that had been ported to C?
Comments or is there such an application and I have not found it.
There are Python application that are based on generating code -- the most amazing one probably Resolver One, which focuses on spreadsheets (and hinges on IronPython). With that exception, however, interacting based on the UI paradigm you have in mind (pick one of this, one of that, etc) tends to be pretty limited in the gamut of choices it offers to let the user generate the exact application they need -- there's just so much more you can say by writing even a little script, than what you can say by point-and-grunt.
That being said, Python would surely be a great choice both to implement such an app and as the language to generate... if and when you have a UI sketch that looks like it can actually allow non-programmers to specify a large-enough spectrum of apps in a broad-enough domain!-). Spreadsheets have proven themselves in this sense, but I don't know of other niches or approaches that have actually done so -- do you?
Your idea kinda reminded me of something I stumbled across months ago: http://www.ailab.si/orange/
Is your concept very similar to Microsoft Access? Generally programmers tend not to write such programs because they produce such horrible code that the authors themselves would never want to use their program.