I have a Python code that needs to be able to execute a C++ code. I'm new to the idea of creating libraries but from what I have learned so far I need to know whether I need to use static or dynamic linking.
I have read up on the pros and cons of both but there is a lot of jargon thrown around that I do not understand yet and since I need to do this ASAP I was wondering if some light can be shed on this from somebody who can explain it simply to me.
So here's the situation. My C++ code generates some text files that have data. My Python code then uses those text files to plot the data. As a starter, I need to be able to run the C++ code directly from Python. Is DLL more suitable than SL? Or am I barking up the completely wrong tree?
Extra: is it possible to edit variables in my C++ code, compile it and execute it, all directly from Python?
It depends on your desired deployment. If you use dynamic linking will need to carefully manage the libraries (.so, .dll) on your path and ensure that the correct version is loaded. This can be helped if you include the version number in the filename, but then that has its own problems (security... displaying version numbers of your code is a bad idea).
Another benefit is that you can swap your library functionality without a re-compile as long as the interface does not change.
Statically linking is conceptually simpler and practically simpler. You only have to deploy one artefact (an .exe for example). I recommend you start with that until you need to move to the more complicated shared library setup.
Edit: I don't understand your "extra credit" question. What do you mean by "edit values"? If you mean can you modify variables that were declared in your C++ code, then yes you can as long as you use part of the public interface to do it.
BTW this advice is for the general decision. If you are linking from Python to C/C++ I think you need to use a shared library. Not sure as I haven't done it myself.
EDIT: To expand on "public interface". When you create a C++ library of whatever kind, you specify what functions are available to outside classes (look up how to to that). This is what I mean by public interface. Parts of your library are inaccessible but others (that you specify) are able to be called from client code (i.e. your python script). This allows you to modify the values that are stored in memory.
If you DO mean that you want to edit the actual C++ code from within your python I would suggest that you should re-design your application. You should be able to customise the run-time behaviour of your C++ library by providing the appropriate configuration.
If you give a solid example of what you mean by that we'll be able to give you better advice.
Yes it is possible!!
Try exploring subprocess module in python.
Following can be an example implementation of your scenario:
yourfile.cpp
#compilation
args = ['g++','-o','your_executable_name_with_path','yourfile.cpp_with_path']
your_compile = subprocess.Popen(args,stdin=subprocess.PIPE,stderr=subprocess.PIPE,stdout=subprocess.PIPE)
output,compilation_error = your_compile.communicate()
your_compile.wait()
#successful compilation then there will be execuatble
if not compilation_error:
#execuation
args = ['your_executable_name_with_path'] #command to run a an execuatble
your_run = subprocess.Popen(args,stdin=subprocess.PIPE,stderr=subprocess.PIPE,stdout=subprocess.PIPE)
your_code_output,runtime_error = your_run.communicate()
your_run.wait()
Further, you can tackle more cases and come up with an efficient design
I'm not quite sure how the idea of linking comes into what you are asking, but it sounds to me like you want to use something like SWIG, which allows you to create wrappers around C++ functions (and many other languages) which you can then call directly from your Python code.
Extra: is it possible to edit values in my C++ code, compile it and execute it directly from Python?
If I'm understanding this correctly, you want to use Python to change your C++ code, then compile and execute it? If this is the case, you may want to look into embedding the Python interpreter in your C++ program. This would mean doing things the other way around and having C++ run your Python script, instead of trying to do everything from Python.
Related
When using app = win32com.client.Dispatch('Some.Application'), is there any feasible way get code-completion in PyCharm? It is rather tedious having to retype (or copy-paste) everything from an API documentation, so would creating skeletons be. Is there no other way to let PyCharm know about the Interface provided via COM, especially if I can provide a .tlb file? Or is there at least some way automatically generate such a skeleton (or a wrapping module?) from the TypeLib?
Since there is no way for PyCharm to know the runtime type of app, you shouldn't expect to get code completion on app directly; at least not until they decide to add built-in support for generating code from type libraries.
However, you can exploit the fact that win32com implicitly generates code based on the type library as described in the first part of this answer, together with PyCharm's support for type hinting, to get code completion on COM methods.
Make sure that the Python types have been generated; their location is determined by the GUID of the COM object. For example, the types for Microsoft Word 2016 on my machine are available in
C:\Users\username\appdata\local\temp\gen_py\3.6\00020905-0000-0000-c000-000000000046x0x8x7\.
Add this folder to the path of your PyCharm Python interpreter; see e.g. this answer.
Import the modules for which you want code completion.
In the screenshots below, we use this approach with Word's Find:
Now, besides feeling dirty, this approach relies on the relevant types having been generated and the code completion is limited to the methods published by the object, so I imagine its usefulness in practice might be somewhat limited; in particular, anybody working on the code will have to generate the code, or the annotations will cause NameErrors. Personally, I would probably prefer using Jupyter for the exploratory part of the implementation process, and with minimal tweaks outlined in the answer mentioned above, Jupyter can be extended to have full code completion with win32com.
I am new to object oriented programming and I am struggling to find a good tutorial on how to make a library in C++ that I can import into Python.
At the moment I am just trying to make a simple example that adds two numbers. I am confused about the process. Essentially I want to be able to do something like this in Python:
import MyCPPcode
MyCPPcode.Add(5,3) #function prints 5+3=8
I am not requesting a full example with code, just the steps that I need to take.
Do I need to make a .dll or a static library? I am using MS Visual Studio 2013.
Also, does the process tailor the C++ library code for Python in any way or will this library be available for other languages as well?
While I cannot guide you through the whole process, because I do not know python too well, Here is what I know:
It is absolutely possible. While not being something for someone who is new to object-oriented programing, it's called the python-C/C++ API. If you search for that in the python documentation there are several chapters about it.
While the example function you're showing might look like that from python, the process is a lot more redundant in c++ (behind the scenes). There are tools that combat that issue, for example Cython, but if you want to learn I'd suggest going pure python API.
As for availability with other languages... Well, the internal functions (i.e. adding two numbers) are of course general c++, so you can reuse them in other projects, but yes, the created library will be made to work with python, and not something else.
I'm working with an API which is distributed as a dll/so file that I need to dynamically link with my python program. To accomplish this, I want to use Cython.
I have been able to, in the past, link with the dll statically. This works well, except that the API comes in four different flavors, with theoretically infinitely more to come and the users should be able to compile them with whatever name they want (Kinda like a plugin system). Because of that, I can't just make an so/pxd file that statically links with just one library, or even links with a selection of them.
What I need is to be able to pass the so/dll name into the Cython code and have it "import" it. I know this can be done with ctypes via ctypes.cdll.LoadLibrary, but is this kind of thing possible in Cython? Am I going to have to use ctypes to do this?
I assume you are talking about writing C modules here. If so yes you can. I don't know what the equivalent on Windows is, but in Linux you can use dlopen and friends. There is a man page for it, and several web sites documenting it. Try this link "http://linux.die.net/man/3/dlopen" it provides you with a nice example near the bottom of the page. This is doing the same thing as what ctypes does, in fact I think this might even be what ctypes uses.
let us assume that there is a big, commercial project (a.k.a Project), which uses Python under the hood to manage plugins for configuring new control surfaces which can be attached and used by Project.
There was a small information leak, some part of the Project's Python API leaked to the public information and people were able to write Python scripts which were called by the underlying Python implementation as a part of Project's plugin loading mechanism.
Further on, using inspect module and raw __dict__ readings, people were able to find out a major part of Project's underlying Python implementation.
Is there a way to keep the Python secret codes secret?
Quick look at Python's documentation revealed a way to suppres a import of inspect module this way:
import sys
sys.modules['inspect'] = None
Does it solve the problem completely?
No, this does not solve the problem. Someone could just rename the inspect module to something else and import it.
What you're trying to do is not possible. The python interpreter must be able to take your bytecode and execute it. Someone will always be able to decompile the bytecode. They will always be able to produce an AST and view the flow of the code with variable and class names.
Note that this process can also be done with compiled language code; the difference there is that you will get assembly. Some tools can infer C structure from the assembly, but I don't have enough experience with that to comment on the details.
What specific piece of information are you trying to hide? Could you keep the algorithm server side and make your software into a client that touches your web service? Keeping the code on a machine you control is the only way to really keep control over the code. You can't hand someone a locked box, the keys to the box, and prevent them from opening the box when they have to open it in order to run it. This is the same reason DRM does not work.
All that being said, it's still possible to make it hard to reverse engineer, but it will never be impossible when the client has the executable.
There is no way to keep your application code an absolute secret.
Frankly, if a group of dedicated and determined hackers (in the good sense, not in the pejorative sense) can crack the PlayStation's code signing security model, then your app doesn't stand a chance. Once you put your app into the hands of someone outside your company, it can be reverse-engineered.
Now, if you want to put some effort into making it harder, you can compile your own embedded python executable, strip out unnecessary modules, obfuscate the compiled python bytecode and wrap it up in some malware rootkit that refuses to start your app if a debugger is running.
But you should really think about your business model. If you see the people who are passionate about your product as a threat, if you see those who are willing to put time and effort into customizing your product to personalize their experience as a danger, perhaps you need to re-think your approach to security. Assuming you're not in the DRM business, or have a similar model that involves squeezing money from reluctant consumers, consider developing an approach that involves sharing information with your users, and allowing them to collaboratively improve your product.
Is there a way to keep the Python secret codes secret?
No there is not.
Python is particularly easy to reverse engineer, but other languages, even compiled ones, are easy enough to reverse.
You cannot fully prevent reverse engineering of software - if it comes down to it, one can always analyze the assembler instructions your program consists of.
You can, however, significantly complicate the process, for example by messing with Python internals. However, before jumping to how to do it, I'd suggest you evaluate whether to do it. It's usually harder to "steal" your code (one needs to fully understand them to be able to extend them, after all) than code it oneself. A pure, unobfuscated Python plugin interface, however, can be vital in creating a whole ecosystem around your program, far outweighing the possible downsides to having someone peek in your maybe not perfectly designed coding internals.
I work in analyzing binary files, using Python. I have been using debuggers to do a dynamic analysis (i.e running the application and using breakpoints to get runtime execution). however, results can be improved if i can use some binary instrumentation fremework like PIN. The PIN is developed in C++ and provided as closed source (only dlls). We write something called PinTools do describe where and what we want to intercepts. I want to port PIN functionality into Python so that i continue using Python. I am aware of "ctypes" and boost-python.
My problem is: in order to use PIN, we write a pintool and run our bibnary executable with Pin and pintool (it is like running application with JIT). Now, I have no idea if I can use ctypes etc. to import PIN functions and use this python code for dynamically analyzing the binary. Can you please provide some suggestions or guidelines on how to proceed with this task.
So, in nut-n-shell, I want to create a Python interface (wrapper) to PIN framework.
Check out the ProcessTap project. Appears to implement exactly what you are looking for: http://code.google.com/p/processtap/
I was thinking about this recently, while I haven't looked into it, I would approach the problem like this: write a pintool that, upon initialization, starts an embedded python interpreter and imports a python module. I'd look at using SWIG to generate bindings for all the PIN api calls you want to use. Then the pintool would call a hardcoded function in the imported python module that would issue calls to the api to register more functions and do whatever you want to do.
I'm not sure how the callbacks would work, I don't know enough about SWIG. Also, this may fail if the program you're trying to instrument itself uses Python. But that's how I'd try to solve this problem to start out.