I have created two new python notebooks for databricks in /workspace/Shared/Notebooks.
I would share some functions between the both notebooks. I have created a python file containing a few generic functions.
It exists a way to import my functions into my both notebooks ?
Thanks for your help.
It is really easy, when it comes to sharing python code between .ipynb files.
Say, you have following files:
source.py
dest_1.ipynb
dest_2.ipynb
With following contents:
source.py
a = "hello from source"
dest_1.ipynb and dest_2.ipynb
import source
print(source.a)
And you can simply run cells in your notebooks.
The main part is that your source file should be located in the same folder with notebooks.
There are two different methods to achieve this:
Use the %run <another_notebook> to include content of another notebook into a current one (doc)
If you use Databricks Repos, it has support for so-called "Files in Repos" - in this case you can use Python or R files (not notebooks!) as Python or R modules, so for Python you can just do import some_file.
Unit tests in this demo repository shows both approaches.
Related
I have uploaded my databricks notebooks to a repo and replace %run sentences with import using the new databrick public available features (Repo integration and python import): https://databricks.com/blog/2021/10/07/databricks-repos-is-now-generally-available.html
But its seems its not working
I already activate the repo integration option in the Admin panel but i Get this error
ModuleNotFoundError: No module named 'petitions'
For simplicity I moved all python files to the same directory. I get the error in the procesado notebook
[
Please try to use full path to file from repo root:
from folders.file import class
If your are using Azure DataBricks and Python Notebooks, you can't import them as modules.
From the documentation:
If you want to import the notebook as a Python module, you must edit
the notebook in a code editor and remove the line # Databricks
Notebook source. Removing that line converts the notebook to a regular
Python file.
Documentation:
https://learn.microsoft.com/en-us/azure/databricks/repos#work-with-python-and-r-modules
Importing functions from an external ipynb file works as intended if it is in the same directory. However, if I have a project with multiple directories, I would like to be able to import functions from other directories. I understand that it may be easier to use another IDE for this purpose, however, I am documenting my work and would like to continue leveraging Jupyter Notebook's seamless Markdown integration.
I am not seeing anything in the ipynb.fs documentation here and answers on ipynb import another ipynb file and import a function from another .ipynb file all specify that documents must be in the same directory.
This is normally what I would do to import a function from an external ipynb file in the same directory,
from ipynb.fs.full.my_functions import split_into_sentences
I've moved my_functions into functions/ and I've tried this,
from ipynb.fs.full."functions/my_functions" import split_into_sentences
and
from ipynb.fs.full.functions/my_functions import split_into_sentences
Neither works.
Is there any workaround to this?
I am struggling a bit to get a package to work, so I would like to just get the simplest possible test case to work.
Here is my current attempt.
Inside of a folder called Python_experiment I have two files: a jupyter notebook with the code
from .pleasejustwork import eat_muffin
and a file called pleasejustwork.py with the code
def eat_muffin():
print('i ate a muffin')
When I run the line from the jupyter notebook I get the error "attempted relative import with no known parent package". What am I missing?
The syntax you’re using is for a python package. To achieve this, place a file, __init__.py in the directory. It can be empty. If you’d like to use that package from anywhere, write a setup.py script to build and install your package. There are a lot of good tutorials on how do that, like in the python docs. Then you could do
from python_experiment.pleasejustwork import eatmuffin or from another file in the package from .pleasejustwork import eatmuffin
If you’re just trying to learn about modules, you can simply take out that leading period
from pleasejustwork import eatmuffin
I have created a python file (.py) that includes two different functions.
For example,
def function1():
print("Hello World")
def funtion2(a, b):
y=a+b
if y>=5:
print("Correct")
else:
print("Wrong")
Now, I want to use those two functions in a Databricks Notebook. What I did, was to follow the steps written in this databricks post. Even though I succeeded on creating an egg file that was later imported as a library in databricks I didn't manage to import my custom functions from the egg file. Please check the screenshots below:
setup.py
egg file on Databricks
When I try the python command:
import function2
I get an error that this module was not found. I appreciate any help and comments.
Note: init.py file is not used
I found the following solution after some searching in the Web:
Create library notebook.
For example - Lib with any functions/classes there (no runnable code).
To import into Main all the classes and functions from Lib to Main use this command:
%run "./Lib"
(this will work like: from Lib import *)
After that you can call any functions / use classes that used in the Lib from Main notebook.
This is the online post with the information.
I don't manage to find how import custom class in Python with the Jupyter notebook in anaconda.
In my work folder I have a file 'user.ipynb' that contains a class name User. In an other file in the same folder, I try to import this class with : from user import User.
I get this error: ImportError: No module named user.
I tried to create a file ' _ _init__.py' and _ _init__.ipynb in this folder but it doesn't work.
Do you know how I could do this ?
Thank you in advance
Python modules are either files named *.py or directories with an __init__.py in them. user.ipynb is neither, it's an IPython (Jupyter) notebook, so it is not a module and you can't import it.
The easiest way to get this to work is to convert the notebook to a .py file (this can be done using the export function) and then you'll be able to import it.
An alternative is to provide code that lets you import notebooks directly; such code is available here. Looks like a lot of bother, though. I'm not sure why they didn't just bake this in; it seems like it would be useful.