We Keep Coding

Can you automate an ETL to dashboard pipeline using python/SQL/R and PowerBI?

Here's what I want to automate:
get data from an API,
do some cleaning and wrangling in python,
save as Excel,
load to PowerBI and
produce very simple dashboard.
I've been searching for ages on Microsoft forums and here to find out if there's a way to run PowerBI via a script, like a Powershell script, for instance. I'm comfortable with writing a script for doing the first 3 steps, but I have not been able to find a solution for steps 4-5. It has to be loaded to PowerBI because that's what clients are most familiar with. The ideal result would be a script that I can then package as an executable file, send to a non-technical person so they can run it at their own leisure.
Although I'd prefer a solution in python, if it's possible in SQL or R, I'd also be very happy. I've tried all the in-PowerBI options for using python scripts, but I have found them limited and difficult to use. I've packaged all my ETL functions into a py script, then imported it to PowerBI and ran functions therein, but it's still not really fully automated, and wouldn't land well with non-technical folks. Thanks in advance! (I am working on PC, PowerBI Desktop)

Hosting Jupyter Notebook Python App on a Webpage

I've made a simple python GUI application using ipywidgets, ipycanvas, and numpy. I made the program on Jupyter notebook as an ipynb file. I would now like to take my application and put it on a webpage. What is the best way to take this Jupyter notebook app and host it on the web?
I've looked a bit into Binder and Django, but I can't seem to find enough resources or documentation on the net to help me learn how to do this.

If you already have it working as a Jupyter notebook (.ipynb file), I'd suggest that sticking with that as the core item for now. I'd suggest getting running via MyBinder.org based on either this example repo or this one. Or a combination of the two.
This video is recent and a good reference for many of the steps of setting up a repo with your content.
You essentially make a copy of the Binder templates under your control and then edit them to have your content. You adapt the URLs that trigger launches so that when you share the link, they launch a session via MyBInder.org with your content. Most often the steps can be performed right in the Github browser-based interface without you needing to use git or work locally. If you have something fancier you need, you may have to move to using more complex configuration file set-ups and those may necessitate some use of git and local editing.
If you hit some technical road blocks, post your questions here using the 'questions' category as suggested in this post about 'Debugging your Binder'.
Maybe once the basics of sharing the notebook or appmode version are working with your own content, you may want to check out Voila or some of the other ways you can share a jupyter notebook-based app discussed here.

Jupyter itself is made with Tornado web framework.
There are many bindings to another popular web frameworks.
I once tried on it, and I found that pyramid-notebook is easy to use.

For a quick build I recommend Binder. This is how you can quickly set up Binder with voila:
Checkout this Git Repo: https://github.com/lschmiddey/book_recommender_voila
In combination with this blogpost: https://lschmiddey.github.io/fastpages_/2020/09/28/Build-binder-app-Part4.html

Is there a way for two People to work on one Jupyter Notebook

A friend of mine and me are doing some field research for our Physics degree. And we are using jupyter notebook to analyse the data we get. We usually sit together working at two different copies of the same file that in the end will be drag and dropped together using jupyter lab. This is obviously not ideal, so i thought is there any way for just two people to work on one document in Jupyter, sadly Google Colab has been Deprecated and CoCalc is expensive. So i thought id ask here if there is a way to make one person run a Jupyter notebook and the other one just being able to access it over peer to peer aswell so we could write in the same file at the same time.
Do you guys know something that makes me do this maybe a workaround that i can do.
Thanks for answers in advance

CoCalc is expensive.
Fortunately, we also provide a complete free easy to install open source version of CoCalc, which you can run on any computer that supports Docker. For example, here's how to run it on Google cloud.
(I have put too many years of my life into making realtiime collaboration work for Jupyter via CoCalc... In any case, the open source code has been battle tested in production for a while now and is working well finally. I hope it can solve your problem...)

You can upload your notebook to Deepnote. It provides a hosted environment, where you and your colleague can connect at the same time and work on the same notebook in real-time (the same way you'd do in Google Docs).
Colab is also good, but writing at the same time will result in conflicts.

Notebook itself doesn't support to collaborate simultaneously, but you can use GitHub to manage your python script and upload it into Colab separately. This way Github can help manage the file history and solve the conflicts.

JupyterLab 3.1.0a7 introduced real time collaboration.
There is a screencast showing it in action.
Key thing to note is the new top-level menu item called Share, to the right of Settings & Help.
You can click on launch binder here or here to try it now.
"Once you see the JupyterLab interface, there's a new top-level menu item called "Share"; click that, grab and share that URL, and you're done!"-SOURCE: Step #5 here
There's a gist here that seems to be updated regularly with how to activate the feature.
There's a detailed walk-through here if you want to add the ability into your own repositories that can launch via MyBinder.org. Although if that repo falls behind the gist, you'll probably want to consult the gist for the current best practices once you have the idea from the detailed walk-through.
Closely related question with an answer by #krassowski, is here. You may want to look there for some additional details.

While you can use github for this it can get messy, many people clear output cells when committing to git to avoid conflict issues. Which would defeat the object of your review work.
You should try Curvenote (which we're building for that reason) it doesn't offer compute as its a collaborative writing tool, works on top of Jupyter via a chrome extenson and gives you real time versioning, commenting and diffs.

Google Colab has been Deprecated and CoCalc is expensive
Noteable.io is 100% free for all users including storage, compute, RAM. For your purposes, it will be ideal as you will get Google Drive like collaboration (commenting, #mentioning, Annotating data points), versioning, sharing, interactive visualizations, choice of using Python and SQL in the same notebook and a ton of other features.
Here are good example notebooks on Noteable:
Climate Change: An analysis of Dew Point for the city of Toronto
Healthcare Sector Employee Attrition Exploratory Data Analysis
Exploratory Data Analysis Using SQL and Python - Online Retailer Orders

Hosting interactive jupyter notebook on private website

I currently run a personal website using Wordpress (but hosted on siteground) that is a set of engineering study guides. I would like to move towards making these study guides interactive (i.e. refreshing graphics based on sliders, doing basic calculations to indicate if a design works or not, so I need numpy). A friend recommended that I utilize Jupyter notebooks for this purpose, as you can both render LaTeX (which I'm currently using Mathjax with Wordpress to do), as well as have the types of interactive graphics I want using either Bokeh or Plotly.
While I've seen tutorials for sharing notebooks on specific servers, what I'm after is being able for others to run my notebook in their browser (read-only), where the notebook is privately hosted.
I'm still not sure if Jupyter is the correct avenue to accomplish what I want, so I'm open to other suggestions (someone also recommended using Julia, but I've seen fewer examples of this).

I agree with your friend that Jupyter Notebooks is an excellent approach. And while it's by no means the only method to accomplish what you're after, I'm hard-pressed to come up with an immediate alternative that doesn't require significant work to set up.
I can think of three primary methods of using Jupyter Notebooks which suit your needs:
1. Azure Notebooks
Microsoft has a new service called Azure Notebooks, which is (currently) totally free.
Azure Notebooks boasts the complete functionality of Jupyter Notebooks, and in addition to Python, users can also program cells in R and F#. As for typical usage of the service, here's a snippet from their FAQ:
Jupyter (formerly IPython), is a multi-lingual REPL on steroids. This is a free service that provides Jupyter notebooks along with supporting packages for R, Python and F# as a service. This means you can just login and get going since no installation/setup is necessary. Typical usage includes schools/instruction, giving webinars, learning languages, sharing ideas, etc. The service is provided by the Python team # Microsoft, which is part of the Data Group.
2. nbviewer
The top banner of the main Jupyter site contains a link link to an application called nbviewer.
Evidently, you can create your markdown / Jupyter syntax as a discrete page somewhere else, feed the URL to your page into nbviewer, and it'll render it for you right there in the results. If I were going to use this, I would either;
Create a discrete WordPress page for my Jupyter syntax, then feed that into nbviewer; or, more likely
Use GitHub to host my Jupyter Notebook pages (mainly for posterity and version control, over the Gist option), and use the raw text link as the source to feed into nbviewer.
3. Hosting Your Own Solution
If you're technically savy enough, I'd recommend this approach over nbviewer.
When you launch Jupyter Notebooks on your own machine, you access it through your browser using the default URL of http://localhost:8888. That means there must exist some mechanism to expose that port to external users, and allow them to have access to your Notebook, using the exact same interface. Two methods of doing so:
Using Jupyter Notebooks public server
Remotely accessing your normal Jupyter Notebook
Hope that helps! I'm curious to know if any of these options works out for you.

The Iodide Project (and subsequently, Pyodide) are two projects that aim to allow this. They're still in development, but might be worth looking into.

You can try to use Mercury framework. It allows you to transform notebooks into web applications (with interactive widgets). You need to add YAML header to the beginning of the notebook. Based on YAML the widgets will be generated. Your users can change widgets values and click Run button to execute the notebook with new inputs. You can decide whether to show or hide code for your users. You can serve multiple notebooks with Mercury on single server. It is based on Django so can be easily deployed on any server/cloud.
The example notebook:
The generated application for the above notebook:
The screenshot of app/notebooks gallery in the Mercury:

Creating projects based on jupyter notebooks

Having learned to program in Python with Jupyter Notebooks, I've found this to be a very practical way of analyzing data, writing simple programs and even interacting with databases.
However, if one is working on a bigger project (with program and database that run automatically and should also eventually be deployed), is it still possible / reasonable to run the code from Notebooks? If so, do you have any advice on that?
Otherwise, I would of course resort to an IDE. Thanks!

Jupyter Notebooks are awesome tools, and great for certain types of collaborative development, and creating interactive web apps.
The purpose of Jupyter Notebooks is to provide a framework for combining rich text elements and code together (perfect for data science projects, tutorials or interactive dashboards). Jupyter notebooks also allow you to run code in multiple languages, which is a neat feature if your project requires it.
You probably wouldn't want to run a large scale production application from a Jupyter notebook, but you could certainly use them to help you develop it.
Check out this presentation for using Jupyter notebooks with multiple users.
https://www.slideshare.net/mbussonn/jupyter-a-platform-for-data-science-at-scale
If you specified the type of project you were considering it might be easier to suggest alternatives to help complete it.