I am a beginner to python programming language. I have seen the definition "python is portable" at https://docs.python.org/.
Query 1:
Python is portable - Does it mean python scripts or python executable??
Query 2:
Like JVM, for Java, is anything needed to run "python executable" on target machine.
Most probably you will have read somewhere that the Python language is an interpreted programming or a script language. The truth is: Python is both an interpreted and a compiled language. But calling Python a compiled language would be misleading.
People would assume that the compiler translates the Python code into machine language but this is not the way python code is executed. Python code is translated into intermediate code, which has to be executed by a virtual machine, known as the PVM, the Python virtual machine. This is a similar approach to the one taken by Java. There is even a way of translating Python programs into Java byte code for the Java Virtual Machine (JVM). This can be achieved with Jython. The default implementation of python is Cython but there is more like Jython, PyPy,IronPython and more!
Python is portable - Does it mean python scripts or python executable?
The Python script is portable because it runs on the Python Virtual Machine (PVM)
Python is basically an interpreted language. However Python implementations may compile source code to bytecode in order to reuse/optimize/whatever so in some sense/implementation Python is a compiled language
Python is an interpreted language, as opposed to a compiled one, though the distinction can be blurry because of the presence of the bytecode compiler. This means that source files can be run directly without explicitly creating an executable which is then run.
https://docs.python.org/3/glossary.html#term-interpreted
Like JVM, the PVM is an abstract machine that runs on the current computer so we needs a PVM implementation for each platform which means the Python executor is not portable, but the compiled code (*.pyc) is portable, like the Python script
However currently it's even possible to have a portable PVM. Just a single executable can run on Linux, Mac, Windows, NetBSD, FreeBSD, OpenBSD because of the great αcτµαlly pδrταblε εxεcµταblε (APE) project so one can distribute their work easily by providing a small Python executable (which is just more than 4MB at the moment) along with a *.py file
See Python is Actually Portable
See also
If Python is interpreted, what are .pyc files?
Java "Virtual Machine" vs. Python "Interpreter" parlance?
Is Python interpreted, or compiled, or both?
https://news.ycombinator.com/item?id=32245430
I have generated a Python module using SWIG and Python 2.7 64 bit.
This Python module can successfully be imported (i.e. used) with this Python version, but when trying to import the module using a different Python version, let's say 3.5 64 bit, it fails.
Inversely, when I generate the same Python module using SWIG and Python 3.5 64 bit, it can successfully be imported (i.e. used) with this version, but not with Python 2.7 64 bit.
It would seem that SWIG generates a Python module just for that Python version that it was used upon the generation. Is this conclusion correct? If yes, what would be the approach to take to "tell" SWIG to generate a Python module that is Python version agnostic?
SWIG ultimately produces a Python extension which is inherently tied to a single version (and configuration) of Python.
SWIG itself is responsible only for the generation of code, not building it. It produces both C code to implement a Python extension, and a Python wrapper around that. Both of these outputs that SWIG generates should be version-agnostic.
When you actually compile the code however, you have to point it at an include path for your specific version of Python.
If you use setuptools to build everything, this mutiple-step process is somewhat hidden from you, as the include path for the Python version running the script is used automatically.
Look at 36.2.3 Hand compiling a dynamic module
I have made an application in Python. It contains several plugins, organized into different subdirectories. I need to compile entirely to C code to improve security of source code. I have dealt with Cython, but cannot find how to compile the entire directory, with all plugin dependencies. I need a way to compile each of the dependencies to C, and that the application runs from C compiled.
http://docs.cython.org/src/quickstart/build.html
How to compile and link multiple python modules (or packages) using cython?
Python does not compile to native code. Scripts can be "frozen" with a few different tools, which makes them into standalone executables, but it's not actually compiling to C, it's packaging the script (or just its Python byte code representation) with a copy of the interpreter and all of its dependencies; the binary still has all the Python code (or the trivial byte code transform thereof) in it.
Cython lets you compile a syntactic variant of Python into Python C extensions, but they still run on the Python interpreter, and they still expose enough information to reverse the transformation.
Get the proper legal protections in place and freeze your Python executable if you like (freezing is enough to make the source code "non-obvious" even if anyone who went to trivial effort could get it back), but Python does not compile to plain C directly (if it did, I'd expect the CPython reference interpreter to do that more often just to gain performance with the built-in modules, yet they only write C accelerators by hand).
I have an application written in python 2.7 . In my app I want to give users the ability to compile a python source file (i.e py to pyc).
The builtin compile function does this but the output file is only compatible with python 2.7.
What I want is to compile the file to a range of different python versions (specifically python 2.5 , 2.6 & 2.7 )
I know that one way to approach this problem is re-write my application in each of those versions and just use the inbuilt compile function, but I do not want to do this.
I am open to all suggestions including writing a C extension, embedding python etc...
EDIT
The application which I am writing is a tool which allows to inject/modify arbitrary code inside a pyinstaller exe.
Now you may be knowing that pyinstaller actually puts compiled python source files within the exe.
My app just extracts these embedded pyc files, decompiles them, allows the user to modify them, and then rebuild the exe.
Since the exe can be from a different python version, so I need the functionality to compile the decompiled output to that version.
Here is a screenshot of that app.
I solved my own problem in a different approach.
Now, my main app is written in python 2.7. I wrote 2 dll each of which had a python interpreter embedded. Within one I embedded python 2.6 and within other python 2.5.
Each of these two dlls uses the Python C API to compile a script given as an argument to the respective python version.
Now, from my main app (written in python 2.7) , I used ctypes to call the exported functions from these two dlls. The script to compile was passed as an argument.
Using this approach I was able to compile a given script to any of the 3 python versions.
Well, I have a Python package. I need to compile it as dll before distribute it in a way easily importable. How? You may suggest that *.pyc. But I read somewhere any *.pyc can be easily decompiled!
Update:
Follow these:
1) I wrote a python package
2) want to distribute it
3) do NOT want distribute the source
4) *.pyc is decompilable >> source can be extracted!
5) dll is standard
Write everything you want to hide in Cython, and compile it to pyd. That's as close as you can get to making compiled python code.
Also, dll is not a standard, not in Python world. They're not portable, either.
Nowadays a simple solutino exists: use Nuitka compiler as described in Nuitka User Manual
Use Case 2 - Extension Module compilation
If you want to compile a single extension module, all you have to do is this:
python -m nuitka --module some_module.py
The resulting file some_module.so can then be used instead of some_module.py.
You need to compile for each platform you want to support and write some initialization code to import so/pyd file ~~appropriate for given platform/python version etc.~~
[EDIT 2021-12]: Actually in python 3 the proper so/dll is determined automatically based on the file name (if it includes python version and platform - can't find PEP for this feature at the moment but Nuitka creates proper names for compiled modules). So for python 2.7 the library name would be something.pyd or something.so whereas for python 3 this would change to something.cp36-win32.pyd or something.cpython-36m-x86_64-linux-gnu.so (for 32bit python 3.6 on x86).
The result is not DLL as requested but Python-native compiled binary format (it is not bytecode like in pyc files; the so/pyd format cannot be easily decompiled - Nuitka compiles to machine code through C++ translation)
EDIT [2020-01]: The compiled module is prone to evaluation methods using python standard mechanisms - e.g. it can be imported as any other module and get its methods listed etc. To secure implementation from being exposed that way there is more work to be done than just compiling to a binary module.
You can use py2exe.org to convert python scripts into windows executables. Granted this will only work on windows, but it's better then nothing.
You can embed python inside C. The real trick is converting between C values and Python values. Once you've done that, though, making a DLL is pretty straightforward.
However, why do you need to make a dll? Do you need to use this from a non-python program?
Python embedding is supported in CFFI version 1.5, you can create a .dll file which can be used by a Windows C application.
I would also using Cython to generate pyd files, like Dikei wrote.
But if you really want to secure your code, you should better write the important stuff in C++. The best would be to combine both C++ and Python. The idea: you would leave the python code open for adjustments, so that you don't have to compile everything over and over again. That means, you would write the "core" in C++ (which is the most secure solution these days) and use those dll files in your python code. It really depends what kind of tool or program you are building and how you want to execute it. I create mostly an execution file (exe,app) once I finish a tool or a program, but this is more for the end user. This could be done with py2exe and py2app (both 64 bit compatible). If you implement the interpreter, the end user's machine doesn't have to have python installed on the system.
A pyd file is the same like a dll and fully supported inside python. So you can normally import your module. You can find more information about it here.
Using and generating pyd files is the fastest and easiest way to create safe and portable python code.
You could also write real dll files in C++ and import them with ctypes to use them (here a good post and here the python description of how it works)
To expand on the answer by Nick ODell
You must be on Windows for DLLs to work, they are not portable.
However the code below is cross platform and all platforms support run-times so this can be re-compiled for each platform you need it to work on.
Python does not (yet) provide an easy tool to create a dll, however you can do it in C/C++
First you will need a compiler (Windows does not have one by default) notably Cygwin, MinGW or Visual Studio.
A basic knowledge of C is also necessary (since we will be coding mainly in C).
You will also need to include the necessary headers, I will skip this so it does not become horribly long, and will assume everything is set up correctly.
For this demonstration I will print a traditional hello world:
Python code we will be converting to a DLL:
def foo(): print("hello world")
C code:
#include "Python.h" // Includes everything to use the Python-C API
int foo(void); // Declare foo
int foo(void) { // Name of our function in our DLL
Py_Initialize(); // Initialise Python
PyRun_SimpleString("print('hello world')"); // Run the Python commands
return 0; // Finish execution
}
Here is the tutorial for embedding Python. There are a few extra things that should be added here, but for brevity I have left those out.
Compile it and you should have a DLL. :)
That is not all. You will need to distribute whatever dependencies are needed, that will mean the python36.dll run-time and some other components to run the Python script.
My C coding is not perfect, so if anyone can spot any improvements please comment and I will do my best to fix the it.
It might also be possible in C# from this answer How do I call a specific Method from a Python Script in C#?, since C# can create DLLs, and you can call Python functions from C#.
You can use pyinstaller for converting the .py files into executable with all required packages into .dll format.
Step 1. pip install pyinstaller,
step 2. new python file let's name it code.py .
step 3. Write some lines of code i.e print("Hello World")
step 4. Open Command Prompt in the same location and write pyinstaller code.py hit enter. Last Step see in the same location two folders name build, dist will be created. inside dist folder there is folder code and inside that folder there is an exe file code.exe along with required .dll files.
If your only goal is to hide your source code, it is much simpler to just compile your code to an executable(use PyInstaller, for example), and use an module with readable source for communication.
NOTE: You might need more converter functions as shown in this example.
Example:
Module:
import subprocess
import codecs
def _encode_str(str):
encoded=str.encode("utf-32","surrogatepass")
return codecs.encode(encoded,"base64").replace(b"\n",b"")
def _decode_str(b64):
return codecs.decode(b64,"base64").decode("utf-32","surrogatepass")
def strlen(s:str):#return length of str;int
proc=subprocess.Popen(["path_to_your_exe.exe","strlen",_encode_str(str).decode("ascii")],stdout=subprocess.PIPE)
return int(proc.stdout.read())
def random_char_from_string(str):
proc=subprocess.Popen(["path_to_your_exe.exe","randchr",_encode_str(str).decode("ascii")],stdout=subprocess.PIPE)
return _decode_str(proc.stdout.read())
Executable:
import sys
import codecs
import random
def _encode_str(str):
encoded=str.encode("utf-32","surrogatepass")
return codecs.encode(encoded,"base64").replace(b"\n",b"")
def _decode_str(b64):
return codecs.decode(b64,"base64").decode("utf-32","surrogatepass")
command=sys.argv[1]
if command=="strlen":
s=_decode_str(sys.argv[2].encode("ascii"))
print(len(str))
if command=="randchr":
s_decode_str(sys.argv[2].encode("ascii"))
print(_encode_str(random.choice(s)).decode("ascii"))
You might also want to think about compiling different executables for different platforms, if your package isn't a windows-only package anyways.
This is my idea, it might work. I don't know, if that work or not.
1.Create your *.py files.
2.Rename them into *.pyx
3.Convert them into *.c files using Cython
4.Compile *.c into *.dll files.
But I don't recommend you because it won't work on any other platforms, except Windows.
Grab Visual Studio Express and IronPython and do it that way? You'll be in Python 2.7.6 world though.