I have created a python file (.py) that includes two different functions.
For example,
def function1():
print("Hello World")
def funtion2(a, b):
y=a+b
if y>=5:
print("Correct")
else:
print("Wrong")
Now, I want to use those two functions in a Databricks Notebook. What I did, was to follow the steps written in this databricks post. Even though I succeeded on creating an egg file that was later imported as a library in databricks I didn't manage to import my custom functions from the egg file. Please check the screenshots below:
setup.py
egg file on Databricks
When I try the python command:
import function2
I get an error that this module was not found. I appreciate any help and comments.
Note: init.py file is not used
I found the following solution after some searching in the Web:
Create library notebook.
For example - Lib with any functions/classes there (no runnable code).
To import into Main all the classes and functions from Lib to Main use this command:
%run "./Lib"
(this will work like: from Lib import *)
After that you can call any functions / use classes that used in the Lib from Main notebook.
This is the online post with the information.
Related
I have created two new python notebooks for databricks in /workspace/Shared/Notebooks.
I would share some functions between the both notebooks. I have created a python file containing a few generic functions.
It exists a way to import my functions into my both notebooks ?
Thanks for your help.
It is really easy, when it comes to sharing python code between .ipynb files.
Say, you have following files:
source.py
dest_1.ipynb
dest_2.ipynb
With following contents:
source.py
a = "hello from source"
dest_1.ipynb and dest_2.ipynb
import source
print(source.a)
And you can simply run cells in your notebooks.
The main part is that your source file should be located in the same folder with notebooks.
There are two different methods to achieve this:
Use the %run <another_notebook> to include content of another notebook into a current one (doc)
If you use Databricks Repos, it has support for so-called "Files in Repos" - in this case you can use Python or R files (not notebooks!) as Python or R modules, so for Python you can just do import some_file.
Unit tests in this demo repository shows both approaches.
I am struggling a bit to get a package to work, so I would like to just get the simplest possible test case to work.
Here is my current attempt.
Inside of a folder called Python_experiment I have two files: a jupyter notebook with the code
from .pleasejustwork import eat_muffin
and a file called pleasejustwork.py with the code
def eat_muffin():
print('i ate a muffin')
When I run the line from the jupyter notebook I get the error "attempted relative import with no known parent package". What am I missing?
The syntax you’re using is for a python package. To achieve this, place a file, __init__.py in the directory. It can be empty. If you’d like to use that package from anywhere, write a setup.py script to build and install your package. There are a lot of good tutorials on how do that, like in the python docs. Then you could do
from python_experiment.pleasejustwork import eatmuffin or from another file in the package from .pleasejustwork import eatmuffin
If you’re just trying to learn about modules, you can simply take out that leading period
from pleasejustwork import eatmuffin
My project has two main folders: sourceCode and lib: Highlighted file tree here
I'm working in \sourceCode\mainFile.ipynb and would like to import a library residing in lib called modifiedLibrary, which has an __init__.py file.
Currently, I'm using a symbolic link for relative-importing the library. The symbolic link is located in \sourceCode and called sym_link with the following content:
../lib/modifiedLibrary/modifiedLibrary
In the project, the library and the symbolic link have the same name.
but when I import in python using
import modifiedLibrary
I receive ModuleNotFoundError: No module named 'modifiedLibrary'
I understand that the same code functions on another device that I do not have access to right now, and I do not seem to find what the issue is.
I successfully included the needed library by:
changing the working directory temporarily to where the library's __init__.py is located,
importing the library
then reverting back to my original directory
but I would like to know what the issue is with the current symbolic link.
Windows 10 / Python 3.7.3 / Jupyter
Relevant Question: Interactive Python - solutions for relative imports
The other solution I found, rather than changing the working directory temporarliy to include a local module, was to add the location of the module on my device to sys.path before importing it:
import sys
sys.path.append('C:/Users/user/myProject/../modifiedLibrary/')
import modifiedLibrary
It doesn't make use of the symbolic link but it seems to do the trick for now. Would be an issue when the code is shared and ran on another device.
I have added a new_folder to C:\Users\Name\Anaconda3\Lib\site-packages which contains an empty _init_.py and various custom modules (uncompiled python files). Scouring this site indicates I should now be able to generically call these custom modules in other python files. However, I keep on getting the ModuleNotFoundError when running these other programs. I am using Windows10. Any ideas where I might be going wrong?
I am using from module1 import function1 where function1 is contained in C:\Users\Name\Anaconda3\Lib\site-packages\new_folder\module1
I don't manage to find how import custom class in Python with the Jupyter notebook in anaconda.
In my work folder I have a file 'user.ipynb' that contains a class name User. In an other file in the same folder, I try to import this class with : from user import User.
I get this error: ImportError: No module named user.
I tried to create a file ' _ _init__.py' and _ _init__.ipynb in this folder but it doesn't work.
Do you know how I could do this ?
Thank you in advance
Python modules are either files named *.py or directories with an __init__.py in them. user.ipynb is neither, it's an IPython (Jupyter) notebook, so it is not a module and you can't import it.
The easiest way to get this to work is to convert the notebook to a .py file (this can be done using the export function) and then you'll be able to import it.
An alternative is to provide code that lets you import notebooks directly; such code is available here. Looks like a lot of bother, though. I'm not sure why they didn't just bake this in; it seems like it would be useful.