In my company we decided to structure own python modules using this convention:
dsc.<package_name>
It works without any problem when two modules is used in other project that doesn't follow this convention. However, when in a develop environment I try to develop a new module "dsc.new_module" that references to other, for example, "dsc.other_module", the import raises a not module found exception. Is there any way to solve this?
If I package the module and install, everything is correct but not when I'm developing the module that is not able to find it. The only way I overcome this problem is doing that:
try:
from dsc.other_module import send_message
except ImportError:
def dummy(a, b):
pass
send_message = dummy
Beacause the function is not essential.
What you can do is install your packages in development mode. pip install -e . (from the parent folder) After this the imports should work as you envision them, so the same as other packages that use them.
Development mode is not required, but it adds the benefit of implementing changes made to your code immediately.
I am new to Julia. I developed a few lines of code to get the results I needed from packages I was not able to find in Python or R. Now, I am trying to get this file to be easily accessible, and wrap the code in Python or R. Has anyone done this before? I have tried a few methods and have not found anything that has helped.
The most simple way to do this would be just a few lines of code that calls the .jl file, runs it (which the code is then added to a .txt file from julia), and then alerts you when the code is done.
Any help would be greatly appreciated. R is the preferable method and at this point Python would be appreciated as well.
Please find below instructions for Python, R and just an external process (which of course is an executable command that can be run from any other process). I recommend putting your code in a package and loading it in one of those languages rather than executing this as an external process.
Python
Use Python Anaconda (not in-built system Python) and install Julia
Run Julia and install PyCall
using Pkg
ENV["PYTHON"]="/path/to/your/python/executable"
pkg"add PyCall"
pkg"build PyCall"
Put your code into a Julia package
using Pkg
Pkg.generate("MyPackage")
In the folder src you will find MyPackage.jl, edit it to look like this:
module MyPackage
function main(x,y)
#do very complex staff or place it in your_other_file.jl
2x.+y
end
include("your_other_file.jl")
export main, and_whatever_other_functio_you_defined
end
Install pyjulia
python -m pip install julia
(On Linux systems you might want to use python3 instead of python command)
For this step note that while an external Python can be used with Julia. However, for a convenience it might be worth
to consider using a Python that got installed together with Julia as PyCall.
In that case for installation use a command such this one:
%HOMEPATH%\.julia\conda\3\python -m pip install julia
or on Linux
~/.julia/conda/3/python -m pip install julia
Note that if you have JULIA_DEPOT_PATH variable defined you can replace %HOMEPATH%\.julia or ~/.julia/ with its value.
Run the appropiate Python and tell it to configure the Python-Julia integration:
import julia
julia.install()
Now you are ready to call your Julia code:
>>> from julia import Pkg
>>> Pkg.activate(".\\MyPackage") #use the correct path
Activating environment at `MyPackage\Project.toml`
>>> from julia import MyPackage
>>> MyPackage.main([1,2],5)
[7,9]
Gnu R
Configure your system PATH variable to point to your Julia location. Hence when you type julia in the console it should start Julia
Run the script below to install R-Julia integration
install.packages("JuliaCall")
library(JuliaCall)
julia <- julia_setup()
Follow the above instructions for Python (step 3 only) and create the package named MyPackage
Run the code
library(JuliaCall)
julia_eval("using Pkg;Pkg.activate(\"C:/temp/rrr/MyPackage\")")
julia_library("MyPackage")
julia_eval("MyPackage.main(3,5)")
Bash (or just any language)
Build the package following instructions for Python (step 3 only)
Configure the system PATH variable
Being in the package directory run the command (note string(:.) is a Julian trick that I use to avoid apostrophe escaping in bash commands):
julia -e "using Pkg;Pkg.activate(string(:.));Pkg.instantiate();using MyPackage;MyPackage.main(3,4)"
This will install all dependencies for your package. In order to skip the installation remove Pkg.instantiate() from the above command.
The answer from #przemyslaw-szufel is correct but maybe a bit overcomplicated. You don't necessarily need to wrap your code in a module or define a custom environment (yes it is good practice, but a step at the time...)
First create a file juliaScripts.jl with content:
function getAnElement(array,n)
return array[n]
end
To run functions defined in a .jl file in R
Then in R you just do:
> install.packages("JuliaCall")
> library(JuliaCall)
> julia_setup() # on every new R session !
> julia_source("juliaScript.jl")
> out <- julia_call("getAnElement",c(10,20,30),2)
> out
[1] 20
Note that the R vector has been automatically converted to a Julia Array.
To run functions defined in a .jl file in Python
In Python, it is even easier:
$ python3 -m pip install --user julia
>>> import julia
>>> julia.install() # only once, not every session
>>> jl=julia.Julia(compiled_modules=False)
>>> from julia import Main
>>> Main.include("juliaScript.jl")
>>> Main.getAnElement([1,2,3],2)
20
Also in Python arrays (native python lists as well Numpy arrays and other commonly used data structures) are automatically converted between Python and Julia.
Not to make advertising, but more details on interfacing R <-> Julia or Python <-> Julia are on my Apress(2019) book "Julia Quick Syntax reference" in Chapter 7 "Interfacing Julia with other languages" (I shouldn't say it, but you can easily find the pdf online in well-known sites...)
Using the JuliaConnectoR package:
library(JuliaConnectoR)
fun <- juliaCall("include", "/path/to/file.jl") # you may need to provide the full path
For more info on JuliaConnectoR, see the link above as well as this paper, which additionally compares it to alternative packages suck as JuliaCall and XRJulia.
I am trying to get clear concept on how to get the Erwin generated DDL objects with python ? I am aware Erwin API needs to be used. What i am looking if what Python Module and what API needs to used and how to use them ? I would be thankful for some example !
Here is a start:
import win32com.client
ERwin = win32com.client.Dispatch("erwin9.SCAPI")
I haven't been able to browse the scapi dll so what I know is from trial and error. Erwin publishes VB code that works, but it is not straightforward to convert.
Install pywin32 (run the below from pip folder e.g. c:\Program Files\Python37\Scripts)
python -m pip install pywin32
python pywin32_postinstall.py -install
Sample script to extract DDL using Erwin's Forward Engineer functionality (change paths accordingly):
import win32com.client
api = win32com.client.Dispatch("erwin9.SCAPI")
unit = api.PersistenceUnits.Add("c:/models/data_model.erwin", "RDO=Yes")
unit.FEModel_DDL("c:/scripts/ddl_script.sql")
For the above to work, Erwin application should be running (probably).
I have been coding in python for about 2 months, but I'm only familiar with basic object-oriented programming, so I do not really understand things like how searching for modules is implemented. (Basically I'm a noob.)
I pip installed a package called Opentrons Opentrons 2.5.2 and all its dependencies into the samefolder as a python script I'm currently writing. However when I tried to import the module below[1], I get an error saying that "Opentrons is not a module". Then, I tried shifting it into the python library because I found out the search path using the pprint module and it seems to work. I was wondering if I can specify the search path from the .py file itself instead of manually printing the search path and putting the file into the library that the script searches for. (Willing to put in images of the directories I put the opentrons package in if it helps.)
[1]
import sys
import pprint
pprint.pprint(search.path)
from opentrons import robot, containers, instruments
Edit: I realise that the fact that I am running all my scripts in a Spyder console located in a python 3.6 environment might be important.
You can try using the __import__ function, or importlib. This should allow you to specify the path.
How do I learn where the source file for a given Python module is installed? Is the method different on Windows than on Linux?
I'm trying to look for the source of the datetime module in particular, but I'm interested in a more general answer as well.
For a pure python module you can find the source by looking at themodule.__file__.
The datetime module, however, is written in C, and therefore datetime.__file__ points to a .so file (there is no datetime.__file__ on Windows), and therefore, you can't see the source.
If you download a python source tarball and extract it, the modules' code can be found in the Modules subdirectory.
For example, if you want to find the datetime code for python 2.6, you can look at
Python-2.6/Modules/datetimemodule.c
You can also find the latest version of this file on github on the web at
https://github.com/python/cpython/blob/main/Modules/_datetimemodule.c
Running python -v from the command line should tell you what is being imported and from where. This works for me on Windows and Mac OS X.
C:\>python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# C:\Python24\lib\site.pyc has bad mtime
import site # from C:\Python24\lib\site.py
# wrote C:\Python24\lib\site.pyc
# C:\Python24\lib\os.pyc has bad mtime
import os # from C:\Python24\lib\os.py
# wrote C:\Python24\lib\os.pyc
import nt # builtin
# C:\Python24\lib\ntpath.pyc has bad mtime
...
I'm not sure what those bad mtime's are on my install!
I realize this answer is 4 years late, but the existing answers are misleading people.
The right way to do this is never __file__, or trying to walk through sys.path and search for yourself, etc. (unless you need to be backward compatible beyond 2.1).
It's the inspect module—in particular, getfile or getsourcefile.
Unless you want to learn and implement the rules (which are documented, but painful, for CPython 2.x, and not documented at all for other implementations, or 3.x) for mapping .pyc to .py files; dealing with .zip archives, eggs, and module packages; trying different ways to get the path to .so/.pyd files that don't support __file__; figuring out what Jython/IronPython/PyPy do; etc. In which case, go for it.
Meanwhile, every Python version's source from 2.0+ is available online at http://hg.python.org/cpython/file/X.Y/ (e.g., 2.7 or 3.3). So, once you discover that inspect.getfile(datetime) is a .so or .pyd file like /usr/local/lib/python2.7/lib-dynload/datetime.so, you can look it up inside the Modules directory. Strictly speaking, there's no way to be sure of which file defines which module, but nearly all of them are either foo.c or foomodule.c, so it shouldn't be hard to guess that datetimemodule.c is what you want.
If you're using pip to install your modules, just pip show $module the location is returned.
The sys.path list contains the list of directories which will be searched for modules at runtime:
python -v
>>> import sys
>>> sys.path
['', '/usr/local/lib/python25.zip', '/usr/local/lib/python2.5', ... ]
from the standard library try imp.find_module
>>> import imp
>>> imp.find_module('fontTools')
(None, 'C:\\Python27\\lib\\site-packages\\FontTools\\fontTools', ('', '', 5))
>>> imp.find_module('datetime')
(None, 'datetime', ('', '', 6))
datetime is a builtin module, so there is no (Python) source file.
For modules coming from .py (or .pyc) files, you can use mymodule.__file__, e.g.
> import random
> random.__file__
'C:\\Python25\\lib\\random.pyc'
Here's a one-liner to get the filename for a module, suitable for shell aliasing:
echo 'import sys; t=__import__(sys.argv[1],fromlist=[\".\"]); print(t.__file__)' | python -
Set up as an alias:
alias getpmpath="echo 'import sys; t=__import__(sys.argv[1],fromlist=[\".\"]); print(t.__file__)' | python - "
To use:
$ getpmpath twisted
/usr/lib64/python2.6/site-packages/twisted/__init__.pyc
$ getpmpath twisted.web
/usr/lib64/python2.6/site-packages/twisted/web/__init__.pyc
In the python interpreter you could import the particular module and then type help(module). This gives details such as Name, File, Module Docs, Description et al.
Ex:
import os
help(os)
Help on module os:
NAME
os - OS routines for Mac, NT, or Posix depending on what system we're on.
FILE
/usr/lib/python2.6/os.py
MODULE DOCS
http://docs.python.org/library/os
DESCRIPTION
This exports:
- all functions from posix, nt, os2, or ce, e.g. unlink, stat, etc.
- os.path is one of the modules posixpath, or ntpath
- os.name is 'posix', 'nt', 'os2', 'ce' or 'riscos'
et al
On windows you can find the location of the python module as shown below:i.e find rest_framework module
New in Python 3.2, you can now use e.g. code_info() from the dis module:
http://docs.python.org/dev/whatsnew/3.2.html#dis
Check out this nifty "cdp" command to cd to the directory containing the source for the indicated Python module:
cdp () {
cd "$(python -c "import os.path as _, ${1}; \
print _.dirname(_.realpath(${1}.__file__[:-1]))"
)"
}
Just updating the answer in case anyone needs it now, I'm at Python 3.9 and using Pip to manage packages. Just use pip show, e.g.:
pip show numpy
It will give you all the details with the location of where pip is storing all your other packages.
On Ubuntu 12.04, for example numpy package for python2, can be found at:
/usr/lib/python2.7/dist-packages/numpy
Of course, this is not generic answer
Another way to check if you have multiple python versions installed, from the terminal.
$ python3 -m pip show pyperclip
Location: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
$ python -m pip show pyperclip
Location: /Users/umeshvuyyuru/Library/Python/2.7/lib/python/site-packages
Not all python modules are written in python. Datetime happens to be one of them that is not, and (on linux) is datetime.so.
You would have to download the source code to the python standard library to get at it.
For those who prefer a GUI solution: if you're using a gui such as Spyder (part of the Anaconda installation) you can just right-click the module name (such as "csv" in "import csv") and select "go to definition" - this will open the file, but also on the top you can see the exact file location ("C:....csv.py")
If you are not using interpreter then you can run the code below:
import site
print (site.getsitepackages())
Output:
['C:\\Users\\<your username>\\AppData\\Local\\Programs\\Python\\Python37', 'C:\\Users\\<your username>\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages']
The second element in Array will be your package location. In this case:
C:\Users\<your username>\AppData\Local\Programs\Python\Python37\lib\site-packages
In an IDE like Spyder, import the module and then run the module individually.
enter image description here
as written above
in python just use help(module)
ie
import fractions
help(fractions)
if your module, in the example fractions, is installed then it will tell you location and info about it, if its not installed it says module not available
if its not available it doesn't come by default with python in which case you can check where you found it for download info