We Keep Coding

ModuleNotFoundError: No module named 'encodings' while running python in iOS

I am trying to integrate python in iOS. I tried same thing as mentioned here - https://github.com/beeware/Python-Apple-support/tree/3.9
Here is my python script in Xcode projects
func RunPythonScript() -> PythonObject {
if let path = Bundle.main.path(forResource:"/Users/projects/extra/python_apple_support/PAS_10_11_v3/PAS_10_11_v3/Resources/",
ofType: nil) {
setenv("PYTHONPATH", path, 1)
setenv("PYTHONHOME", path, 1)
}
let sys = Python.import("sys")
sys.path.append("/Users/projects/extra/python_apple_support/PAS_10_11_v3/PAS_10_11_v3/PAS_10_11_v3/")
let file = Python.import("pythonscript")
let response = file.hello_world()
print(response)
return response
}
It builds successfully but when I call python program it end up saying
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = (not set)
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/Users/karimkhan/Library/Developer/CoreSimulator/Devices/C444D135-C393-4631-AFE2-FF5F86935EF6/data/Containers/Bundle/Application/642E9540-CBAF-448C-8E8D-856B8E5D03EC/PAS_10_11_v3.app/PAS_10_11_v3'
sys.base_prefix = '/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14'
sys.base_exec_prefix = '/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14'
sys.platlibdir = 'lib'
sys.executable = '/Users/karimkhan/Library/Developer/CoreSimulator/Devices/C444D135-C393-4631-AFE2-FF5F86935EF6/data/Containers/Bundle/Application/642E9540-CBAF-448C-8E8D-856B8E5D03EC/PAS_10_11_v3.app/PAS_10_11_v3'
sys.prefix = '/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14'
sys.exec_prefix = '/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14'
sys.path = [
'/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14/lib/python39.zip',
'/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14/lib/python3.9',
'/Users/runner/work/Python-Apple-support/Python-Apple-support/install/iOS/iphonesimulator.x86_64/python-3.9.14/lib/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000000108fd4600 (most recent call first):
<no Python frame>

I just got this working after days of trying! Using Python 3.11.
For anyone wondering, we're using a version of Python patched for iOS (from Beeware's Python Apple Support), and PythonKit (Swift pkg) to embed Python in an iOS app using Xcode. The goal (for me at least) is to do my UI in SwiftUI, but use my own Python logic on some of the app's data.
OP, I don't know what all of your problems might be (there could be several), but I am immediately noticing at least one thing wrong. You're looking on your mac for Python when your goal is to put Python itself inside of your app. You should be looking in the app for the directory containing the Python standard library ('python-stdlib'), as well as the 'lib-dynload' subdir, provided to you by Beeware's Python Apple Support repo. You want to set PYTHONPATH and PYTHONHOME to this combo of paths.
This post is intended to supplement to "How to embed a Python interpreter in an iOS app - presented by Łukasz Langa." on YT.
In the video the OP linked to, Lukasz's 'python-stdlib' has a different name and a very different directory structure than the latest version of Py for iOS provides. What you'll need to figure out is the path to the 'python-stdlib' dir, AND [probably] its subdir, 'lib-dynload'. Counterintuitively, you do need to specify both, even though the latter is a subdir of the former.
Your line:
if let path = Bundle.main.path(forResource:"/Users/projects/extra/python_apple_support/PAS_10_11_v3/PAS_10_11_v3/Resources/",
ofType: nil) {
Should read something like:
if let libPath = Bundle.main.path(forResource: "python-stdlib", ofType: nil),
let libPath2 = Bundle.main.path(forResource: "python-stdlib/lib-dynload", ofType: nil) {
let mergedPaths = "\(libPath):\(libPath2)"
...where 'python-stdlib' is the PATH of the python libraries directory--not just the directory name. (If that's confusing to a newbie, this is the same: let mergedPaths = libPath + ":" + libPath2.
To get an idea of the path, go to Product > Show Build Folder in Finder, then find Products/Debug-iphonesimulator, right click .app (the greyed / X'd out icon) > Show package contents.
Assuming you have properly copied this stuff into the project, you should be able to find 'python-stdlib' inside the .app. For me, it was right in the root of the .app (since that's where I effectively put it, by adding it to the GROUP (not dir!) called 'Resources'. (Because Resources is a group, whatever is in there will be in the root level of the .app, NOT in a dir called Resources. It seems there can be groups that are backed by folders, and groups that are not. This is an important distinction.)
Now, set PYTHONPATH and PYTHONHOME to mergedPaths like you were doing:
setenv("PYTHONHOME", mergedPaths, 1)
setenv("PYTHONPATH", mergedPaths, 1)
If you don't include the path to 'lib-dynload', then this will only work in the simulator. Why would it work at all? Lovely question... 🤷🏽
IMPORTANT: The script.py file Lukasz created for his custom Python code must go into the 'python-stdlib' directory for this to work. I imagine you could stick it anywhere, as long as you append its new path to mergedPaths, as can be seen in a project generated by Beeware's Briefcase. To append that path to the merged paths, you're just concatenating strings with a ":" between each path. I have not tested this.
COPYING THE LIBRARY PROPERLY
Here's something that screwed me up for a while:
When you copy over Python.xcframework and the std-lib, make sure you're creating folders and not groups (in the prompt after you drag and drop), else you will end up with hundreds of errors due to the flattening of directories resulting from creating groups and not dirs (think of every main.py from the lib ending up in the same folder--no bueno). Make sure your prompt looks exactly like this:
screenshot of said prompt ☝🏽
SIGNING THE PYTHON LIBRARY
Follow 'The Manual Way' > Step 6 # Python Apple Support > USAGE.md.
All I had to change was this line segment:
"$CODESIGNING_FOLDER_PATH/Contents/Resources/python-stdlib/lib-dynload"
Due to my directory structure ('python-stdlib' in root of .app), I changed it to:
"$CODESIGNING_FOLDER_PATH/python-stdlib/lib-dynload"
...omitting the extra directories in the path to 'lib-dynload'.
ALMOST FORGOT--module.modulemap!
Assuming you have created 'module.modulemap', and copied it to all three locations (in the Xcode project and in both Headers dirs in the framework), you'll want it to read like this:
module Python {
umbrella header "Python.h"
export *
link "python-stdlib"
}
...where Lukasz's read link Python, ours must read link python-stdlib, as above. I wouldn't be surprised this is actually the path to 'python-stdlib'. Again, mine is right in the root of the .app, so under that assumption, the name is also the path, here. I have not tested this theory.
NOT NECESSARILY RELEVANT TO THE SOLUTION BUT MAYBE ENTERTAINING
This part is pretty amazing. I've had this working in the simulator only for the last several days. For most of that time, on the device, it would import Random, but not Math (which was being called from Random). Then I figured how to point the app to both 'python-stdlib' and 'python-stdlib/lib-dynload'. Got a new error -- code signature invalid for '.../math.cpython-311-iphoneos.so' (that Python math module!) in 'lib-dynload'! Wow! It was finally SEEING the math module.
PARTIAL CONSOLE OUTPUT:
...(code signature in <45B34416-425D-3E01-BC39-CB7A8C170A0A> '/private/var/containers/Bundle/Application/1465B572-3399-4B76-B017-4EE168637AF5/SIXTH_try.app/python-stdlib/lib-dynload/math.cpython-311-iphoneos.so' not valid for use in process: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.)
Here's the amazing part... YESTERDAY, I told Github to tell me about any changes to Beeware's Python Apple Support. Last night at 7pm, I get a notification that dude updated the USAGE.md to add instructions for code signing! Ha! What are the chances that this would happen on exactly the day that I need it to happen. Anyway, I followed those instructions, ran on my phone, and WHUHBAM! MY IPHONE IS USING THE RANDOM MODULE TO GENERATE INTS UPON A BUTTON TAP! THE WHOLE GUI WAS MADE IN SWIFTUI. HOLY MOTHER#(#)!#$ IT'S WORKING! (I realize enthusiasm and emotion are not always welcome on SO, but I'm going to celebrate, and no one will stop me!)

Python script called from VBA called from Python is not working

I maintain an Excel with macro's that download some data from the internet. The downloading is done within Python (I will call this Python A), stored intermediately, and picked up by the Excel again. This Python flow is triggered by a macro within that Excel. Because I have to do this at specific times I wanted to automatize this by using another Python scheduler. The scheduler opens a
Nothing fancy, did that before, at least so I thought. The problem I am currently facing is that Python A is not running correctly when triggered from Python B. The Excel macro is running fine. I know that because some files are being exported, which is also done within a macro.
What I have tried so far:
Running the macro's manually is all fine
Setting all paths absolute, but that was already the case, so nothing to be improved there.
Calling the Python B flow from a bat file. This does work (?!)
Calling the bat from the scheduled flow does not work
Code in VBA:
cmdLine = "python ""path_with_spaces_to_file"" "
lngResult = ShellAndWait(cmdLine, 0, vbNormalFocus, AbandonWait)
Code in Python B to call Macro:
import win32com.client
def func():
filename_excel = r"filename_to_excel_with_spaces.xlsm"
xl = win32com.client.DispatchEx('Excel.Application')
xl.Visible = False
xl.Workbooks.Open(Filename=filename_excel, ReadOnly=1)
sheet = xl.ActiveWorkbook.Sheets("Sheetname")
xl.Application.Run("Macroname")
xl.DisplayAlerts = False
xl.Application.Quit()
How I call this function from the scheduler:
subprocess.run(["python3_location.bat", "-c", 'from python_B_file import func; func()'],
stdout=subprocess.PIPE,
cwd=r"path_to_python_B_file",
universal_newlines=True,
timeout=60)
I see an extra cmd window popping up, but there is no new file downloaded. I cannot see an error message

Trying out different things, I found out that in the normal namespace the command python refers to the system defaults Python 2.7 installation, while the Python B is 3.7. Python A code was not Python 3 compatible (something with urllib, easily solved to something working in both Python versions). Calling the Excel macro from Python B changed the namespace somehow, and the ShellAndWait command referred to Python 3.7.

spawn python ENOENT error with node child process

I am trying to call a python script as a child process within a node script. The output of the script is to be used within a callback. The code looks like this:
//myFunction.js
const myFunction = callback => {
let py = process.spawn('python', ['../folder/pyscript.py'], {
cwd: '../folder/'
});
let str = '';
py.stdout.on('data', data => {
str += data.toString();
}
py.stdout.on('end', () => {
callback(str);
}
}
exports.myFunction = myFunction;
This code works as expected when I directly run node myFunction.js (with an instance of myFunction within the script) and it works fine when I require the module in any other files within the same directory as myFunction.js.
But it fails with the following error when the module is required in a different higher level directory:
error: spawn python ENOENT
I'm guessing this has something to do with paths (value of cwd maybe?) but I can't seem to fix this. I've looked up similar questions but the answers aren't helping.
Any help will be appreciated. :)

Apparently, the issue is with the cwd. Everything in the script is relative to the path of the directory from where the script is invoked. So basically, running node myFunction.js from the project root directory (say ~/projects/myProject would set the cwd to ~/projects/myProject/../folder which would evaluate to ~/projects/folder. This is obviously incorrect, since in all probability, no directory named folder exists on the system, and thus this would lead to an ENOENT error.
The solution would be to construct the absolute path of your script in the code, perhaps by using the __dirname__ property in combination with the functionalities provided by the path module.

I struggled with this issue for days, before realizing that my script file was not getting picked up and spawned by nodeJS, because of some filepath issue.
Although I don't guarantee this will work for everyone, depending on their setup, this is what I did in my nodejs file:
let py = process.spawn('python', [__dirname + '../folder/pyscript.py']);
As you can see, I didn't have to use the {cwd: '../folder/'} option.
If your script is in the current directory as your javascript file, just do
let py = process.spawn('python', [__dirname + './pyscript.py']);
I should also point out that:
process.spawn('python', ['./pyscript.py']);
never worked for me and I spent days wondering why. Could find an answer until I tried this technique. Hope someone having this issue find this answer useful.

Using ${process.cwd()} worked for me... you can write it like this
let py = process.spawn('python', [`${process.cwd()}/folder/pyscript.py`]});

vscode extension: how to create a language specific command

I wrote a simple extension for vscode, which registers a command, namely comandName.
In my package.json file I have:
"activationEvents": [
"onLanguage:python",
"onCommand:commandName"
]
I meant to execute this command only for Python. To my knowledge, however, the activationEvents in my package.json means that the command would be activated when a Python file is opened or the command is executed. But the command can be executed within any language. I searched the documentation but found no way to execute the command for certain languages.
Is there any way to achieve this target?

I'm afraid this is impossible for now. However you can work around this to avoid any side effects on other files.
If you bind this command to some key combination or to context menu you can use when clause to limit the command to specific types of files.
It will still allow to execute the command from command palette though. To work around this you can just ignore it when being in file other than Python:
vsc.commands.registerTextEditorCommand('testCommand', editor => {
if (editor.document.languageId !== 'python') {
return
}
// command logic
}))
Notice I used registerTextEditorCommand instead of regular command. The difference is that this one requires user to be in text editor context to work which is what you probabl

How can I launch a python script using Adobe AIR's NativeProcess?

I'm having more than a little trouble running a python script from an AIR application using the NativeProcess interface. In theory, this should be quite simple. Adobe even uses this as their example in the ActionScript 3.0 documentation for NativeProcess, as follows:
var nativeProcessStartupInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
var file:File = File.applicationDirectory.resolvePath("test.py");
nativeProcessStartupInfo.executable = file;
They even include the contents of what test.py might include:
#!/usr/bin/python
# ------------------------------------------------------------------------------
# Sample Python script
# ------------------------------------------------------------------------------
import sys
for word in sys.argv: #echo the command line arguments
print word
print "HI FROM PYTHON"
print "Enter user name"
line = sys.stdin.readline()
sys.stdout.write("hello," + line)
The problem is that, as far as I can see, this simply doesn't work. I get the following error when I attempt it:
Error #3219: The NativeProcess could not be started. '%1 is not a valid Win32 application.
Presumably the latest version of AIR (19.0) doesn't allow the execution of anything without an "exe" file extension. The following code does seem to do what I want:
var nativeProcessStartupInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
var file:File = new File("C:/Python/Python35/python.exe");
nativeProcessStartupInfo.executable = file;
nativeProcessStartupInfo.workingDirectory = File.applicationDirectory.resolvePath(".");
var processArgs:Vector.<String> = new Vector.<String>();
processArgs[0] = "test.py";
nativeProcessStartupInfo.arguments = processArgs;
The problem here is twofold. First, you need to know the absolute path to the executable, which I can't assume. Second, the code is no longer platform independent. The file extension would be something else on Linux or Mac.
I thought I might solve the first problem by requiring a %PYTHON_PATH% environment variable and then making the executable dependent on that. However, I can't figure out a way to use an environment variable within the ActionScript File object. It "helpfully" escapes all the "%" characters before ever sending something to the command line.
At this point this fairly simple problem has turned into a showstopper. Could someone help me understand a way to either:
Execute something with the "py" extension with NativeProcess
Successfully resolve a path that depends on an environment variable in the File object?