I am stuck using JES for this particular project, and yes, I would rather be poked in the eye. In the mean time, I am stuck with this interesting issue where I cannot load my python package in JES. I was told that this strategy would work. When I give dummy definitions to all the JES functions, and then run the Project in Pycharm, everything works "fine" (given the expected behavior of the dummy functions). I can't figure out what exaclty is going on with JES.
Does anyone have any experience with loading custom python packages in JES?
Project Structure
Project
main.py
MyPackage
__init__.py
allmygoodies
Main.py
""" getMediaPath() and setMediaPath() are built in JES functions """
import sys
setMediaPath() # Setting it to project root
sys.path.append(getMediaPath()) # Appending Project Root
sys.path.append(getMediaPath() + "MyPackage") # Also tried this
from MyPackage import * # Importing everything from package
myObject = MyObject()
Error
The error was:call of non-function ('module' object)
Inappropriate argument type.
An attempt was made to call a function with a parameter of an invalid type. This means that you did something such as trying to pass a string to a method that is expecting an integer.
In order to accomplish this, I had to switch to JES 5+. I was previously using JES 4 which does not support this ability.
Related
I have written a small Python module, with a single submodule. So far, life is good, and everything works. Here is a skeleton of the layout of said module
my_module/
__init__.py
setup.py
submod1/
__init__.py
my_matlab_fn.m
my_python.py
my_matlab_fn.m is a useful matlab function, which for the purposes of reproducing this example, may look like this:
Function Out = my_matlab_fn(x)
dummy = 2.0*x;
Out.val = dummy;
So far so good. Now, let's look inside my_python.py
import matlab
import matlab.engine
def double_my_t(t):
t_eng = matlab.engine.start_matlab('nodesktop -nosplash -nodisplay')
output = t_eng.my_matlab_fn(t)['val']
return output
if __name__ == '__main__':
print(double_my_t(4.0))
(In reality this is a simplified version of my actual code that I cannot show, in reality I am passing a dict to the matlab function and doing lots more operations with it).
OK. So, I am able to successfully do the following:
python3 my_python.py
when inside the submod1 directory. All is well and good and I get the correct result from this function call.
For context, here is what init.py looks like in the submod1 dir
from .my_python import double_my_t
and here is what init.py looks like in the my_module dir
from submod1.my_python import double_my_t
from submod1.my_python import double_my_t as double_my_t
I then go ahead and build my python module. Again, no issues here (have tested and seen my script successfully get inside the double_my_t function).
The issue is if I were to run something like this
from my_module.submod1 import double_my_t
print(double_my_t(4))
I get
matlab.engine.MatlabExecutionError: Undefined function 'my_matlab_fn' for inputs arguments ........
I think I understand what is going wrong, but I am not sure on this. I think that the reason why running my_python.py works is because matlab can 'see' the my_matlab_fn' function in 'my_matlab_fn.m' as this file is in the same dir as my_python.py when it is ran, and that when you try to call double_my_t from somewhere else outside this dir, then the matlab engine doesn't 'know' where my_matlab_fn.m is anymore.
I was wondering if anyone knew how I should resolve this issue. I want to be able to keep using these user-defined matlab functions in my Python module, however the documentation on this is not entirely clear.
Cheers
So I am pretty new with Python. I've been working on running with a code similar to one I hope to build myself (so a reference code). Aside from some bugs I need to work out with invalid syntax, all seems to work except for one issue with one particular .py file I have.
My structure is this:
MoodForecasting -> eval -> nsga_two.py
I do have _init_.py in eval folder though, so I'm not sure why this block of code isn't working.
I am trying to load one particular fucntion from it, so the structure should look like this
from nsga_two import PatientProblem
Unfortunately, I keep getting the error ModuleNotFoundError: No module named 'nsga_two'.
I checked nsga_two.py itself and found that it couldn't load inspyred. I went in and was able to fix this. So, nsga_two.py runs fine on its own. However, I cannot import it into the main script I will be working with.
Some extra details: I am working with the IDE Spyder with Python Custom Version 3.7.9.
I'm not sure if it is an issue with Spyder or just how I am loading in my working directory. (Most of my coding experience is in MatLab and R so having an IDE similar to RStudio and MatLab is the reason I chose to work in Spyder)
Edit:
I got a syntax error when using from eval import nsga_two.PatientProblem. Python didn't like the period. So, I instead tried it with no period. I got the error cannot import name 'nsga_twoPatientProblem' from 'eval' (C:\Users\name\Desktop\MoodForecasting-master\MoodForecasting-master\eval\__init__.py). I don't know why. But doing from eval import nsga_two works. nsga_two.py only consists of PatientProblem. This solve should be ok for this purpose. I'm just not sure why this could be happening.
Suppose your structure is like:
MoodForecasting-master/
main.py
eval/
__init__.py
nsga_two.py
When you run the main script to import something, the directory of that script is added to the module search path sys.path, .../MoodForecasting-master/ in this case. from nsga_two import PatientProblem raised ModuleNotFoundError because nsga_two.py is not in that directory.
As Iguananaut said, from eval import nsga_two.PatientProblem in the first comment has never been a valid statement. The valid ways of doing so are:
import by from eval import nsga_two and use as nsga_two.PatientProblem().
import by from eval.nsga_two import PatientProblem and use as PatientProblem() directly.
Module search starts from .../MoodForecasting-master/, first option go to .../MoodForecasting-master/eval/ to find nsga_two.py, second option go to .../MoodForecasting-master/eval/nsga.py to find attribute named PatientProblem.
The correct syntax would be:
from package.module import function
so:
from eval.nsga_two import PatientProblem
Over the years I have written a few hundred functions in matlab for space engineering purposes which I use on a daily basis. They are all nicely put in one folder ordered in subfolders, and in matlab I just have an addpath() command for the root folder in the startup.m file, and then I can use any of the functions right away after starting up matlab.
I am now trying to do the same in python.
As far as I understand in python I shouldn't have 1 file for every function like in matlab, but rather bundle all the functions together in 1 py file. Is this correct? I am trying to avoid this, since I have a strong preference for short scripts rather than 1 huge one, due to it being way more intuitive for me that way.
And then, once I have all my python scripts, can I place them anywhere in order to use them? Because I read that python works differently than matlab in this aspect, and scripts need to be in the working directory in order to import them. However, I want to load the scripts and be able to use them regardless of my active working directory. So I am guessing I have to do something with paths. I have found that I can append to pythonpath using sys.path.insert or append, however this feels like a workaround to me, or is it the way to go?
So considering I have put all my rewritten matlab fuctions in a single python file (lets call it agfunctions.py) saved in a directory (lets call it PythonFunctions). The core of my startup.py would then be something like (I have added the startup file to PYTHONSTARTUP path):
# startup.py
import os, sys
import numpy as np
import spiceypy as spice
sys.path.append('C:\Users\AG5\Documents\PythonFunctions')
import agfunctions as ag
Does any of this make sense? Is this the way to go, or is there a better way in python?
Well, the python package is probably the best way to solve your problem. You can read more here. Python packages have no need to be built and not always are created for sharing, so do not worry.
Assume you had this file structure:
Documents/
startup.py
PythonFunctions/
FirstFunc.py
SecondFunc.py
Then you can add file __init__.py in your PythonFunctions directory with next content:
__all__ = ['FirstFunc', 'SecondFunc']
Init file must be updated if you change filenames, so maybe it isn't best solution for you. Now, the directory looks like:
Documents/
startup.py
PythonFunctions/
__init__.py
FirstFunc.py
SecondFunc.py
And it's all - PythonFunctions now is a package. You can import all files by one import statement and use them:
from PythonFunctions import *
result_one = FirstFunc.function_name(some_data)
result_two = SecondFunc.function_name(some_other_data)
And if your startup.py is somewhere else, you can update path before importing as next:
sys.path.append('C:\Users\AG5\Documents')
More detailed explanations can be found here
There is no need to write all your matlab functions within one file. You can also keep (kind of) your desired folder structure of your library. However, you should write all functions in the deepest subfolder into one *.py file.
Suppose your MATLAB library is in the folder space_engineering and is set up like this:
space_engineering\
subfolder1\
functiongroup1\
printfoo.m
printbar.m
subfolder2\
...
subfolder3\
...
...
After doing addpath(genpath('\your_path\space_engineering')) all functions in subfolders subfolder* and functiongroup* are available in the global namespace like this
>> printfoo % function printfoo does only do fprintf('foo')
foo
I understand this is an behaviour your want to preserve in your migrated python library. And there is a way to do so. Your new python library would be structured like this:
space_engineering\
__init__.py
subfolder1\
__init__.py
functiongroup1.py
functiongroup2.py
subfolder2\
__init__.py
...
subfolder3\
__init__.py
...
...
As you can see the deepest subfolder layers functiongroup* of the MATLAB structure is replaced now by functiongroup*.py files, so called modules. The only compromise you have to allow for is that functions printfoo() and printbar() are now defined in this .py modules instead of having individual .py files.
# content of functiongroup1.py
def printfoo():
print(foo)
def printbar():
print(bar)
To allow for doing the same function calling as in MATLAB you have to make the function names printfoo and printbar available in the global namespace by adjusting the __init__.py files of each subfolder
# content of space_enginieering\subfolder1\__init__.py
from .functiongroup1 import *
from .functiongroup2 import *
as well as the __init__.py of the main folder
# content of space_engineering\__init__.py
from .subfolder1 import *
from .subfolder2 import *
from .subfolder3 import *
The from .functiongroup1 import * statement loads all names from module functiongroup1 into the namespace of subfolder1. Successively from .subfolder1 import * will forward them to the global namespace.
Like this you can do in an python console (or in any script) e.g.:
>>> sys.path.append('\your_path\space_engineering')
>>> from space_engineering import *
>>> printfoo()
foo
This way you can use your new python library in the same way as you former used the MATLAB library.
HOWEVER: The usage of from xyz import * statement is not recommend in python (see here why, it is similar to why not using eval in MATLAB) and some Pythonistas may complain recommending this. But for your special case, where you insist on creating a python library with MATLAB like comfort, it is a proper solution.
I have a main file that imports a class from another file as such:
from pybrain.rl.environments.HoldemTask import HoldemTask.
When I change HoldemTask.py, the changes are not reflected in the main file. The only workaround I have found is to run Pybrain's
python setup.py install
Can I reload the module or something? Reload() doesn't seem to work.
First off: python setup.py install generally makes a copy of the code it is installing, so if you're finding that you need to run that before changes take effect, chances are that for development you should be adjusting your PYTHONPATH or sys.path so that your relevant imports come directly from the source tree rather than from the Python site-packages library. You can quickly check which file your code is importing by putting this on the top of the main file when you run it:
from pybrain.rl.environments import HoldemTask # module object, not class
print(HoldemTask.__file__)
Secondly, in general it is far better to restart a Python process when making code changes to ensure that they come into effect. If you really need to get changes to show up without a restart, read on.
Reloading a module in Python only affects future imports. For a reload to work in-process, you have to replace the imported class object after the reload. For example, in the context of the "main file" performing the import you listed (inside a class method or function is fine):
# we need a module object to reload(), not the class inside it
from import pybrain.rl.environments import HoldemTask as HoldemTask_module
reload(HoldemTask_module)
# we then need to replace the old class object with the reloaded one
# in the main file's module-wide (aka "global") namespace
global HoldemTask
HoldemTask = HoldemTask_module.HoldemTask
One final caveat here is that any existing HoldemTask objects will continue to use the old code, because they embed in themselves a reference to the pre-reload class object. The only way for an in-process reload to be complete is if the code is specifically written to throw away every instance of anything it made based on the original module.
I'm teaching myself Python (I have experience in other languages).
I found a way to import a "module". In PHP, this would just be named an include file. But I guess Python names it a module. I'm looking for a simple, best-practices approach. I can get fancy later. But right now, I'm trying to keep it simple while not developing bad habits. Here is what I did:
I created a blank file named __init__.py, which I stored in Documents (the folder on the Mac)
I created a file named myModuleFile.py, which I stored in Documents
In myModuleFile.py, I created a function:
def myFunction()
print("hello world")
I created another file: myMainFile.py, which I stored in Documents
In this file, I typed the following:
import myModuleFile.py
myModuleFile.myFunction()
This successfully printed out "hello world" to the console when I ran it on the terminal.
Is this a best-practices way to do this for my simple current workflow?
I'm not sure the dot notation means I'm onto something good or something bad. It throws an error if I try to use myFunction() instead of myModuleFile.myFunction(). I kind of think it would be good. If there were a second imported module, it would know to call myFunction() from myModuleFile rather than the other one. So the dot notation makes everybody know exactly which file you are trying to call the function from.
I think there is some advanced stuff using sys or some sort of exotic configuration stuff. But I'm hoping my simple little way of doing things is ok for now.
Thanks for any clarification on this.
For your import you don't need the ".py" extension
You can use:
import myModuleFile
myModuleFile.myFunction()
Or
from myModuleFile import myFunction
myFunction()
Last syntax is common if you import several functions or globals of your module.
Besides to use the "main" function, I'd put this on your module:
from myModuleFile import myFunction
if __name__ == '__main__':
myFunction()
Otherwise the main code could be executed in imports or other cases.
I'd use just one module for myModuleFile.py and myMainFile.py, using the previous pattern let you know if your module is called from command line or as import.
Lastly, I'd change the name of your files to avoid the CamelCase, that is, I'd replace myModuleFile.py by my_module.py. Python loves the lowercase ;-)
You only need to have init.py if you are creating a package (a package in a simple sense is a subdirectory which has one or more modules in it, but I think it may be more complex than you need right now).
If you have just one folder which has MyModule.py and MyMainFile.py - you don't need the init.py.
In MyMainFile.py you can write :
import myModuleFile
and then use
myModuleFile.MyFunction()
The reason for including the module name is that you may reuse the same function name in more than one module and you need a way of saying which module your program is using.
Module Aliases
If you want to you can do this :
import myModuleFile as MyM
and then use
MyM.MyFunction()
Here you have created MyM as an alias for myModuleFile, and created less typing.
Here Lies Dragons
You will sometimes see one other forms of IMport, which can be dangerous, especially for the beginner.
from myModuleFile import MyFunction
if you do this you can use :
MyFunction()
but this has a problem if you have used the same function name in MyMainFile, or in any other library you have used, as you now can't get to any other definition of the name MyFunction. This is often termed Contaminating the namespace - and should really be avoided unless you are absolutely certain it is safe.
there is a final form which I will show for completeness :
from myModuleFile import *
While you will now be able to access every function defined in myModuleFile without using myModuleFile in front of it, you have also now prevented your MyMainFile from using any function in any library which matches any name defined in myModuleFile.
Using this form is generally not considered to be a good idea.
I hope this helps.