I'm using a library which offers a python wrapper to a c++ executable.
I installed it (https://github.com/bulletphysics/bullet3) using venv (https://docs.python.org/3/library/venv.html) - and all is working great.
I'm considering trying to build https://github.com/bulletphysics/bullet3
From the root of the venv folder I found gym/lib/python3.7/site-packages/pybullet.cpython-37m-x86_64-linux-gnu.so. I'm guessing this is the executable that is invoked eventually from python.
What steps are involved in calling from Python to the correct external binary executable? How does import pybullet as p resolve to gym/lib/python3.7/site-packages/pybullet.cpython-37m-x86_64-linux-gnu.so?
This seems to be close to the end of the c++ world; but I can't find the right key word searches to see exactly how that allows python usages.
Thanks
In short words: C-python just looks for correctly named dynamic libraries in PYTHONPATH, loads such library and uses predefined interface to understand what exactly from this library shall be visible as contents of the module inside Python.
In long words details of how to prepare such shared object and what are required contents of it are described in https://docs.python.org/3/extending/index.html
So venv just puts dynamic library in directory which is part of PYTHONPATH of virtual environment.
python just looks for correctly named dynamic libraries in PYTHONPATH, loads such library and uses predefined interface
for further info :
https://docs.python.org/3/extending/index.html
All my previous work (some big functions) are programmed by Matlab (some .m files). Now I get some robots of which the embedded functions are programmed by python. I want to apply my Matlab functions into the robots, but it will be very hard to recode the functions by python. I find two methods:
call matlab function in python script: it need the Matlab environment, but the robot is independent of my PC when it works. I can not install Matlab in the robot.
translate matlab to python. I installed numpy and scipy, and tried to configure the tool SMOP (under Win7), but there is always an import error: no module named readline.
Another possibility is to write an interface with C to call the .M function, and another interface of C called by python, and then link these two interfaces. But I have no idea with this work.
So I can not continue my work. Can you give me some advice? thank you.
Ive been trying to install matlab engine for python so I can call matlab functions from within my python scripts.
The guide Im using is this one: http://uk.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html
It says the setup.py file is located in matlabroot\extern\engines\python, however, my extern folder does not contain an engines folder (only examples, include, lib)
Am I looking in the wrong place? Or is there another way to get this installed? Im using matlab 2012b and python 2.7 if that makes a difference
Looking at the release notes for Matlab 2014, the Python engine didn't exist until R2014b, so I don't think you'll be able to use it in 2012.
http://uk.mathworks.com/help/matlab/release-notes.html
I'm having a problem importing the numpy module in Eclipse. The import numpy line runs with no problems. But then if I try use a function from the numpy module (e.g. numpy.zeros, numpy.uint8, etc.) I get the error Undefined variable from import.
The functions work fine in the python interpreter or actually running a python script. The problem is Eclipse recognizing the module functions. I've had no trouble adding other such modules. All of the modules I'm using come from the unofficial windows binaries. Once I've installed them, I've simply added the module's directory to the Eclipse interpreter libraries to get them to work. This has been unsuccessful for numpy. I've even tried adding each of the subfolders of the numpy package to the interpreter libraries. I've checked the import in the interpreter using a verbose run of Python to see what all happens during the import, but I don't see much that I haven't already tried added to the Eclipse interpreter libraries. I attempted using earlier versions of numpy and the version from the official numpy site as well, but with the same result. I tried adding the numpy directory to the project PYTHONPATH specifically as well, but with no success there either.
I'm using Windows 7, Python 2.7, and 64-bit versions of all the packages (and python).
Any suggestions? Thank you much!
My bet is that since numpy.__init__ conditionally loads itself on import with the if __NUMPY_SETUP__ block, the static analyzer has no idea which branch will run, and thus just fails to identify what would end up in the module's namespace whatsoever. You should try adding NumPy to the forced builtins in PyDev. I've had similar problems with other libraries (for instance, anything off flask.ext), but basically have just learned to ignore them.
Well, I have a Python package. I need to compile it as dll before distribute it in a way easily importable. How? You may suggest that *.pyc. But I read somewhere any *.pyc can be easily decompiled!
Update:
Follow these:
1) I wrote a python package
2) want to distribute it
3) do NOT want distribute the source
4) *.pyc is decompilable >> source can be extracted!
5) dll is standard
Write everything you want to hide in Cython, and compile it to pyd. That's as close as you can get to making compiled python code.
Also, dll is not a standard, not in Python world. They're not portable, either.
Nowadays a simple solutino exists: use Nuitka compiler as described in Nuitka User Manual
Use Case 2 - Extension Module compilation
If you want to compile a single extension module, all you have to do is this:
python -m nuitka --module some_module.py
The resulting file some_module.so can then be used instead of some_module.py.
You need to compile for each platform you want to support and write some initialization code to import so/pyd file ~~appropriate for given platform/python version etc.~~
[EDIT 2021-12]: Actually in python 3 the proper so/dll is determined automatically based on the file name (if it includes python version and platform - can't find PEP for this feature at the moment but Nuitka creates proper names for compiled modules). So for python 2.7 the library name would be something.pyd or something.so whereas for python 3 this would change to something.cp36-win32.pyd or something.cpython-36m-x86_64-linux-gnu.so (for 32bit python 3.6 on x86).
The result is not DLL as requested but Python-native compiled binary format (it is not bytecode like in pyc files; the so/pyd format cannot be easily decompiled - Nuitka compiles to machine code through C++ translation)
EDIT [2020-01]: The compiled module is prone to evaluation methods using python standard mechanisms - e.g. it can be imported as any other module and get its methods listed etc. To secure implementation from being exposed that way there is more work to be done than just compiling to a binary module.
You can use py2exe.org to convert python scripts into windows executables. Granted this will only work on windows, but it's better then nothing.
You can embed python inside C. The real trick is converting between C values and Python values. Once you've done that, though, making a DLL is pretty straightforward.
However, why do you need to make a dll? Do you need to use this from a non-python program?
Python embedding is supported in CFFI version 1.5, you can create a .dll file which can be used by a Windows C application.
I would also using Cython to generate pyd files, like Dikei wrote.
But if you really want to secure your code, you should better write the important stuff in C++. The best would be to combine both C++ and Python. The idea: you would leave the python code open for adjustments, so that you don't have to compile everything over and over again. That means, you would write the "core" in C++ (which is the most secure solution these days) and use those dll files in your python code. It really depends what kind of tool or program you are building and how you want to execute it. I create mostly an execution file (exe,app) once I finish a tool or a program, but this is more for the end user. This could be done with py2exe and py2app (both 64 bit compatible). If you implement the interpreter, the end user's machine doesn't have to have python installed on the system.
A pyd file is the same like a dll and fully supported inside python. So you can normally import your module. You can find more information about it here.
Using and generating pyd files is the fastest and easiest way to create safe and portable python code.
You could also write real dll files in C++ and import them with ctypes to use them (here a good post and here the python description of how it works)
To expand on the answer by Nick ODell
You must be on Windows for DLLs to work, they are not portable.
However the code below is cross platform and all platforms support run-times so this can be re-compiled for each platform you need it to work on.
Python does not (yet) provide an easy tool to create a dll, however you can do it in C/C++
First you will need a compiler (Windows does not have one by default) notably Cygwin, MinGW or Visual Studio.
A basic knowledge of C is also necessary (since we will be coding mainly in C).
You will also need to include the necessary headers, I will skip this so it does not become horribly long, and will assume everything is set up correctly.
For this demonstration I will print a traditional hello world:
Python code we will be converting to a DLL:
def foo(): print("hello world")
C code:
#include "Python.h" // Includes everything to use the Python-C API
int foo(void); // Declare foo
int foo(void) { // Name of our function in our DLL
Py_Initialize(); // Initialise Python
PyRun_SimpleString("print('hello world')"); // Run the Python commands
return 0; // Finish execution
}
Here is the tutorial for embedding Python. There are a few extra things that should be added here, but for brevity I have left those out.
Compile it and you should have a DLL. :)
That is not all. You will need to distribute whatever dependencies are needed, that will mean the python36.dll run-time and some other components to run the Python script.
My C coding is not perfect, so if anyone can spot any improvements please comment and I will do my best to fix the it.
It might also be possible in C# from this answer How do I call a specific Method from a Python Script in C#?, since C# can create DLLs, and you can call Python functions from C#.
You can use pyinstaller for converting the .py files into executable with all required packages into .dll format.
Step 1. pip install pyinstaller,
step 2. new python file let's name it code.py .
step 3. Write some lines of code i.e print("Hello World")
step 4. Open Command Prompt in the same location and write pyinstaller code.py hit enter. Last Step see in the same location two folders name build, dist will be created. inside dist folder there is folder code and inside that folder there is an exe file code.exe along with required .dll files.
If your only goal is to hide your source code, it is much simpler to just compile your code to an executable(use PyInstaller, for example), and use an module with readable source for communication.
NOTE: You might need more converter functions as shown in this example.
Example:
Module:
import subprocess
import codecs
def _encode_str(str):
encoded=str.encode("utf-32","surrogatepass")
return codecs.encode(encoded,"base64").replace(b"\n",b"")
def _decode_str(b64):
return codecs.decode(b64,"base64").decode("utf-32","surrogatepass")
def strlen(s:str):#return length of str;int
proc=subprocess.Popen(["path_to_your_exe.exe","strlen",_encode_str(str).decode("ascii")],stdout=subprocess.PIPE)
return int(proc.stdout.read())
def random_char_from_string(str):
proc=subprocess.Popen(["path_to_your_exe.exe","randchr",_encode_str(str).decode("ascii")],stdout=subprocess.PIPE)
return _decode_str(proc.stdout.read())
Executable:
import sys
import codecs
import random
def _encode_str(str):
encoded=str.encode("utf-32","surrogatepass")
return codecs.encode(encoded,"base64").replace(b"\n",b"")
def _decode_str(b64):
return codecs.decode(b64,"base64").decode("utf-32","surrogatepass")
command=sys.argv[1]
if command=="strlen":
s=_decode_str(sys.argv[2].encode("ascii"))
print(len(str))
if command=="randchr":
s_decode_str(sys.argv[2].encode("ascii"))
print(_encode_str(random.choice(s)).decode("ascii"))
You might also want to think about compiling different executables for different platforms, if your package isn't a windows-only package anyways.
This is my idea, it might work. I don't know, if that work or not.
1.Create your *.py files.
2.Rename them into *.pyx
3.Convert them into *.c files using Cython
4.Compile *.c into *.dll files.
But I don't recommend you because it won't work on any other platforms, except Windows.
Grab Visual Studio Express and IronPython and do it that way? You'll be in Python 2.7.6 world though.