We Keep Coding

How do I control where SQLAlchemy places a sqlite db file

I plan to (attempt) to upload a flask application to a web-host that does not provide ssh access. I have confirmed that it (the webhost) will run Flask applications, and I can create one that works, when it has no database, but I am getting errors when attempting to create the database. I can't work out how to control where it is trying to place the database. My code looks like this:
from flask_sqlalchemy import SQLAlchemy
# create the extension
db = SQLAlchemy()
# create the app
app = Flask(__name__)
# configure the SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///flaskapp.db"
# initialize the app with the extension
db.init_app(app)
In my development machine, using Geany, running db.create_all() places the database in "var/app-instance/". Using PyCharm, on the same machine it places it in "instance/".
Some variable presumably dictates what this path is, but so far I haven't worked out what, or how to influence it. My application works as expected on my development server, using either development environment (Geany or Pycharm), but this does not work on the webhost I am trying to use, as described below.
As well as 'googling', I have grepped through the sqlalchemy files, and I found the def create_all(..), in schema.py but can't work out where it gets the information on what directory-structure to create.
I am not able to use "os" in the web-host, a suggestion made in other answers and tutorials.
I tried creating a path in various forms, and on my development machine, this, for example, works:
"sqlite:////tmp/flaskapp.db" , but I don't have access to /tmp on the web host, and I was unable to find an absolute path that the webhost would accept (ie without complaining that I don't have access to write to the directory). I can't 'pwd' on the webhost either.
Using "sqlite://instance/flaskapp.db" on my development machine produces an error pointing out that:
Valid SQLite URL forms are:
sqlite:///:memory: (or, sqlite://)
sqlite:///relative/path/to/file.db
sqlite:////absolute/path/to/file.db
However, if I try, a relative path, for example, "sqlite:///instance/flaskapp.db", I get "sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file -(Background on this error at: https://sqlalche.me/e/20/e3q8)", even if I create the directory myself (ie relative to the app.py root directory). [In this case write permissions are the same as for all other parts of the project].
That link, in the error output, says "This error is a DBAPI Error and originates from the database driver (DBAPI), not SQLAlchemy itself". Unfortunately I am not clear how to proceed from that.
If someone could help direct me to information that would help me understand and resolve the issue, that would be great, thanks!
I would like to be able to explicitly state where the database will be stored, relative to the route of my application directory
I am using Linux (Arch), in case that is important, too.
The question linked by Sam below shows the use of "url.make_url()". When I use this as shown, I get
>>> import sqlalchemy.engine.url as url
>>> url.make_url('sqlite:///flaskcw.db')
sqlite:///flaskcw.db
which is what I would expect (and what I want). But this is not what happens when I run db.create_all()
>>> from main import db
database binding
sqlite:///flaskcw.db
Engine(sqlite:////home/user/PycharmProjects/cwflaskapp/instance/flaskcw.db)
whereas I would expect it to place the database in the root of the project (in this case cwflaskapp/, as ..cwflaskapp/flaskcw.db) - given that is where main.py is - rather than in 'projroot'/instance/, a directory that is created in the process. (Or in the case of Geany in 'projroot'/var/app-instance/ - also created only on creating the database as above).
What am I missing?

I found another question this code isnt creating my site.db file in directory with an answer that allows me to specify a folder on my development server, without a dependency on "os".
To briefly repeat the relevant part of the answer, I can do this:
app.config['SQLALCHEMY_DATABASE_URI'] = f"sqlite:///{app.root_path}/mydir/site.db"
and this puts the database in 'projroot'/mydir/site.db . I am going to confirm that I can do the same on the webhost, and will re-edit accordingly

Linking python file into functioning HTML/CSS website

I'm giving myself a project to better learn these languages which I already know a lot of it's just syncing them together I need to get better with. This project is a pretty basic "SIM" game, generate some animals into your profile with login/logout. So far I've got the website aspect with HTML/CSS done and functioning with all the pages I currently need all of which is local host on my desktop. Now I'm moving on to working with Python and possibly some PHP aspects into this to get the login/logout and generate a new animal into your account.
Everything I've done with python so far has been done in IDEL, I'm wondering how to link my python document to my HTML document. Like you would CSS? Or is that not possible if not then how do I connect the two to have python interact with the HTML/CSS that has been created? I'm guessing to need MySQL for a database setup but seeing how much I can get as a simple local host without hosting online?

If you want to setup a localhost with PHP and MYSQL I can recommend XAMP (https://www.apachefriends.org/). In order for your webapp to talk to your Python scripts you will either need to use FLASK or Django to create a python webserver, or use PHP to run python scripts. Either way, you will need to make AJAX requests to an API to get this done.
Edit: Forgot to mention this, but you will need JavaScript in order to do this

Implementing own scrapyd service

I want to create my own service for scrapyd API, which should return a little more information about running crawler. I get stuck at very beginning: where I should place the module, which will contain that service. If we look at default "scrapyd.conf" it's has a section called services:
[services]
schedule.json = scrapyd.webservice.Schedule
cancel.json = scrapyd.webservice.Cancel
addversion.json = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json = scrapyd.webservice.ListSpiders
delproject.json = scrapyd.webservice.DeleteProject
delversion.json = scrapyd.webservice.DeleteVersion
listjobs.json = scrapyd.webservice.ListJobs
so this is the absolute paths to each service in scrapyd package, which placed in dist-packages folder. Is there any way to place my own module, containing service not in dist-packages folder?
upd.
Realized that question may be unclear. Scrapy is a framework for parsing data from websites. I have a simple django site from where I can start/stop crawlers for specific region etc (http://54.186.79.236 it's in russian). Manipulating with crawlers occurs through scrapyd API. In default it has a little API's only for start/stop/list crawlers and their logs etc. This APIs are listed in this doc's http://scrapyd.readthedocs.org/en/latest/api.html
So above was a little intro, to the question now. I want extend existing API to retrieve more info from running crawler and render it in my website mentioned above. For this I need inherit existing scrapyd.webservice.WsResource and write a service. Its ok with that part if I place that service module in one of 'sys.path' paths. But I want to keep this service containing module in scrapy project folder (for some aesthetic reason). So if I keep it there it argues(predictably) 'No module named' on scrapyd launch.

So, I solve my problem according to this.

how to create a web2py app from command line?

I am trying to create a web2py app from command line with -S ... However, I would like to create an app without the "No app, Create one?" question. Is there someway I can create it in a forceful manner.
I have tried app_create() from gluon.admin but there I face the problem of the Request argument. None value for request doesn't seem to work.
Help would really be appreciated. Thanks!

I see Massimo Di Pierro answered your question here.
Although it is not what you asked about, I'd like to add this answer to hopefully help those who come here for a related problem. I.E. how to create a new web2py app WITHOUT browser IDE, in windows?
Notice in the web2py book chapter 4 it says, "The tar gzipped scaffolding app that ship with web2py is welcome.w2p". Therefore:
copy welcome.w2p to mytempdir
rename welcome.w2p welcome.gz
using winzip[gui] or other extract welcome.gz to mytempdir
mytempdir will contain a new file named welcome.w2p
rename welcome.w2p welcome.tar
using winzip[gui] or other extract welcome.tar to mytempdir/appname
copy mytempdir/appname into web2py/applications

How can I periodically run a Python script to import data into a Django app?

I have a script which scans an email inbox for specific emails. That part's working well and I'm able to acquire the data I'm interested in. I'd now like to take that data and add it to a Django app which will be used to display the information.
I can run the script on a CRON job to periodically grab new information, but how do I then get that data into the Django app?
The Django server is running on a Linux box under Apache / FastCGI if that makes a difference.
[Edit] - in response to Srikar's question When you are saying " get that data into the Django app" what exactly do you mean?...
The Django app will be responsible for storing the data in a convenient form so that it can then be displayed via a series of views. So the app will include a model with suitable members to store the incoming data. I'm just unsure how you hook into Django to make new instances of those model objects and tell Django to store them.

I think Celery is what you are looking for.

You can write custom admin command to load data according to your need and run that command through cron job. You can refer Writing custom commands
You can also try existing loaddata command, but it tries to load data from fixture added in your app directory.

I have done the same thing.
Firstly, my script was already parsing the emails and storing them in a db, so I set the db up in settings.py and used python manage.py inspectdb to create a model based on that db.
Then it's just a matter of building a view to display the information from your db.
If your script doesn't already use a db it would be simple to create a model with what information you want stored, and then force your script to write to the tables described by the model.

Forget about this being a Django app for a second. It is just a load of Python code.
What this means is, your Python script is absolutely free to import the database models you have in your Django app and use them as you would in a standard module in your project.
The only difference here, is that you may need to take care to import everything Django needs to work with those modules, whereas when a request enters through the normal web interface it would take care of that for you.
Just import Django and the required models.py/any other modules you need for it work from your app. It is your code, not a black box. You can import it from where ever the hell you want.
EDIT: The link from Rohan's answer to the Django docs for custom management commands is definitely the least painful way to do what I said above.

When you are saying " get that data into the DJango app" what exactly do you mean?
I am guessing that you are using some sort of database (like mysql). Insert whatever data you have collected from your cronjob into the respective tables that your Django app is accessing. Also insert this cron data into the same tables that your users are accessing. So that way your changes are immediately reflected to the users using the app as they will be accessing the data from the same table.

Best way?
Make a view on the django side to handle receiving the data, and have your script do a HTTP POST on a URL registered to that view.
You could also import the model and such from inside your script, but I don't think that's a very good idea.

Have your script send an HTTP Post request like so. This is the library Requests
>>> files = {'report.xls': open('report.xls', 'rb')}
>>> r = requests.post(url, files=files)
>>> r.text
then on the receiving side you can use web.py to process the info like this
x = web.input()
then do whatever you want with x
On the receiving side of the POST request import web and write a function that handles the post
for example
def POST(self):
x = web.input()

If you dont want to use HTTP to send messages back and forth you could just have the script write the email info to a .txt file and then have your django app open the file and read it.
EDIT:
You could set your CRON job to read the e-mails at say 8AM then write it to a text file info.txt. The in your code write something like
import time
if '9' == time.strftime("%H"):
file = open(info.txt)
info = file.read()
that will check the file at 9AM untill 10AM. if you want it to only check one time just add the minutes too the if statement as well.