Here is what I have done so far,
1) I installed Django, and started a project using:
django-admin.py startproject helloworld
2) I downloaded django non-rel, djangotoolbox and django-mongodb-engine
I installed these using
sudo python setup.py install
3) Added djangotoolbox to INSTALLED_APPS, and added django_mongodb_engine as database back end engine.
4) I have a sample model like this:
from django.db import models
from djangotoolbox import *
class Post(models.Model):
title = models.CharField()
text = models.TextField()
tags = ListField()
comments = ListField()
5) Started python repl using: python manage.py shell
6) And did this:
from hello.models import Post
post = Post.objects.create(
... title='Hello MongoDB!',
... text='Just wanted to drop a note from Django. Cya!',
... tags=['mongodb', 'django']
... )
I got this error, I don't know how to fix this
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 149, in create
return self.get_query_set().create(**kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 416, in create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 546, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 650, in save_base
result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 215, in _insert
return insert_query(self.model, objs, fields, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 1675, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 237, in get_compiler
return connection.ops.compiler(self.compiler)(self, connection, using)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/__init__.py", line 703, in compiler
self._cache = import_module(self.compiler_module)
File "/usr/local/lib/python2.6/dist-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/home/home/Desktop/helloworld/django_mongodb_engine/compiler.py", line 18, in <module>
from djangotoolbox.db.basecompiler import (
File "/usr/local/lib/python2.6/dist-packages/djangotoolbox-1.4.0-py2.6.egg/djangotoolbox/db/basecompiler.py", line 9, in <module>
from django.db.models.sql.constants import LOOKUP_SEP, MULTI, SINGLE
ImportError: cannot import name LOOKUP_SEP
What am i doing wrong here? how do i fix it?
LOOKUP_SEP was here in Django 1.4, but has been removed in 1.5, so my guess is you are using an olfer version of djangotoolbox.
You probably need to update your packages, try to follow the instructions here: http://django-mongodb-engine.readthedocs.org/en/latest/topics/setup.html
Related
I am working with django-tenant-schemas and when I try to use "migrate_schemas" command I encounter an error. I've seen similar questions here but they didn't help at all. I've tried this on two different apps but the result is the same. Does anybody know how to fix this?
Traceback (most recent call last):
File "C:\DjangoNew\tenancy\manage.py", line 22, in <module>
main()
File "C:\DjangoNew\tenancy\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 401, in execute_from_command_line
utility.execute()
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\base.py", line 322, in run_from_argv
parser = self.create_parser(argv[0], argv[1])
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\base.py", line 296, in create_parser
self.add_arguments(parser)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\site-packages\tenant_schemas\management\commands\migrate_schemas.py", line 20, in add_arguments
command.add_arguments(parser)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\commands\migrate.py", line 28, in add_arguments
help='Skip system checks.',
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\argparse.py", line 1373, in add_argument
return self._add_action(action)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\argparse.py", line 1736, in _add_action
self._optionals._add_action(action)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\argparse.py", line 1577, in _add_action
action = super(_ArgumentGroup, self)._add_action(action)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\argparse.py", line 1387, in _add_action
self._check_conflict(action)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\argparse.py", line 1526, in _check_conflict
conflict_handler(action, confl_optionals)
File "C:\Users\asyey\AppData\Local\Programs\Python\Python37\lib\argparse.py", line 1535, in _handle_conflict_error
raise ArgumentError(action, message % conflict_string)
argparse.ArgumentError: argument --skip-checks: conflicting option string: --skip-checks
It is a bug in django-tenant-schemas.
From reading Django documents it seems like they forgot to set requires_system_checks to False.
I believe this pull request will fix the problem.
Till this fix is merged, you can solve this by either downgrade Django to version 2 by running something like
pip install "Django~=2.2"
or move tenant-schemas at the INSTALLED_APPS list to the bottom in the settings.py file.
Error: argparse.ArgumentError: argument --email: conflicting option string: --email
If anyone here by this error from django-rest-framework
So, this error occurs because the field 'email' is required, but still in Model's REQUIRED_FIELDS list.
Just remove it from REQUIRED_FIELDS, it should work.
This scenario also possible in other attributes also.
class UserAccount(AbstractUser):
first_name = ...
last_name = ...
email = models.EmailField(_("Email address"), unique=True, blank=False)
REQUIRED_FIELDS = AbstractUser.REQUIRED_FIELDS + ['first_name', 'last_name', 'email']
class Meta(AbstractUser.Meta):
swappable = "AUTH_USER_MODEL"
verbose_name = _("UserAccount")
verbose_name_plural = _("UserAccounts")
I want to setup a machine learning pipeline that is callable by flask but I am facing some issues, these links are for the documentations I have read so far:
https://exploreflask.com/en/latest/views.html#view-decorators
https://flask.palletsprojects.com/en/1.1.x/api/#flask.Flask
Let me explain the pipeline I have in mind:
pull a dataframe from a PostgreSQL database
encode said dataframe to make it ready for most algorithms
split up the data
feed to a pipeline and determine accuracy
store the model in a pickle file
What is working so far:
All parts are working as a regular script
I can just slap all the steps into one huge flask file with one decorator and it would run as well (my emergency solution)
The File Structure
The encoder script:
#Flask main thread
#makes flask start this part as application and not as module
app = Flask('encoder_module')
#app.route('/df_encoder')
def df_encoder(rng = 4):
encoding stuff
`return df`
The Pipeline script (random forest regressor here)
app = Flask('pipeline_module')
#app.route('/pipeline_rfr')
def pipeline_rfr():
pipeline stuff
`return grid_search_rfr`
The pickle module:
app = Flask('pickle_module')
#app.route('/store_reg_pickle')
def store_pickle():
"""
Usage of a Pickle Model -Storage of a trained Model
"""
model = grid_search_rfr
#specify file name in letter strings
model_file = "regression_model"
with open(model_file, mode='wb') as m_f:
pickle.dump(model, m_f)
print(f"Model saved in: {os.getcwd()}")
return model_file
The Main Flask File
#packages
from flask import Flask
from encoder_main_thread import df_encoder
from rfr_pipeline_function import pipeline_rfr
from pickle_call import store_pickle
app = Flask(__name__.split('.')[0])
#app.route('/regression_pipe')
#df_encoder
#pipeline_rfr
#store_reg_pickle
def regression_pipe():
`return 'pipeline done`
The problem iss that the return value of the encoder cannot be a dataframe, only a string, tuple, etc.
Is there a workaround for this?
I actually want it to be a flawless passing of the dataframe to the pipeline and eventually storing it in the pickle file which is then saved in the folder.
For some reason it cannot detect the pickle file import and throws following error:
Use a production WSGI server instead.
* Debug mode: off
Traceback (most recent call last):
File "C:\ANACONDA3\Scripts\flask-script.py", line 9, in <module>
sys.exit(main())
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 967, in main
cli.main(args=sys.argv[1:], prog_name="python -m flask" if as_module else None)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\click\decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 848, in run_command
app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 305, in __init__
self._load_unlocked()
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 330, in _load_unlocked
self._app = rv = self.loader()
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 388, in load_app
app = locate_app(self, import_name, name)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 240, in locate_app
__import__(module_name)
File "C:\Users\bill-\OneDrive\Dokumente\Docs Bill\TA_files\functions_scripts_storage\flask_test\flask_regression_pipeline.py", line 18, in <module>
#store_reg_pickle
NameError: name 'store_reg_pickle' is not defined
If you wish I could upload the entire scripts but that is a lot to look through and since it is working as a long regular pice of code, the mistake needs to be somewhere with my flask setup.
I have this piece of code which is running perfectly on localhost but throws up this obscure error on GAE:
import_string() failed for 'webapp2_extras.appengine.auth.models.User' . Possible reasons are: - missing __init__.py in a package; - package or module
My import statements:
from webapp2_extras import auth
from webapp2_extras import sessions
from webapp2_extras.auth import InvalidAuthIdError
from webapp2_extras.auth import InvalidPasswordError
Usage of auth's user model:
user = self.auth.store.user_model.create_user(username, password_raw = password, email = email)
if not user[0]: #returns a tuple with [boolean, user_info]
return 'Create user error'
else:
self.set_flash("Thank you for registering. Please login!")
self.redirect(self.auth_config['login_url'])
Full code
UPDATE (Full stack trace)
import_string() failed for 'webapp2_extras.appengine.auth.models.User'. Possible reasons are:
- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;
Original exception:
ImportError: No module named ndb
Debugged import:
- 'webapp2_extras' found in '/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/__init__.pyc'.
- 'webapp2_extras.appengine' found in '/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/appengine/__init__.pyc'.
- 'webapp2_extras.appengine.auth' found in '/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/appengine/auth/__init__.pyc'.
- 'webapp2_extras.appengine.auth.models' not found.
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/data/home/apps/s~webapp-auth/1.358936463581927371/main.py", line 34, in dispatch
response = super(BaseHandler, self).dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~webapp-auth/1.358936463581927371/main.py", line 127, in post
user = self.auth.store.user_model.create_user(username, password_raw = password, email = email)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 701, in __get__
value = self.func(obj)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/auth.py", line 131, in user_model
cls = self.config['user_model'] = webapp2.import_string(cls)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1824, in import_string
return getattr(__import__(module, None, None, [obj]), obj)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/appengine/auth/models.py", line 13, in <module>
from ndb import model
Look similar to this issue which was already fixed by webapp 2.5.1
Make sure you import the latest version of webapp2, by adding those line to your app.yaml file:
libraries:
- name: webapp2
version: latest
As a workaround you can add the following lines to your application:
import sys
from google.appengine.ext import ndb
sys.modules['ndb'] = ndb
I have a problem with south migrations. I still don't understand how this did happen, and what should what path to move to resolve this
Romans-MacBook-Pro:holms$ ./manage.py migrate
cRunning migrations for accounts:
- Nothing to migrate.
- Loading initial data for accounts.
No fixtures found.
Running migrations for allocations:
- Nothing to migrate.
- Loading initial data for allocations.
No fixtures found.
Running migrations for adyen:
- Nothing to migrate.
- Loading initial data for adyen.
No fixtures found.
Running migrations for blog:
- Nothing to migrate.
- Loading initial data for blog.
No fixtures found.
Running migrations for offers:
- Nothing to migrate.
- Loading initial data for offers.
No fixtures found.
Running migrations for orders:
- Migrating forwards to 0011_update_price_fields.
> orders:0002_update_order_contact_information
Traceback (most recent call last):
File "./manage.py", line 15, in <module>
execute_manager(settings)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/management/commands/migrate.py", line 105, in handle
ignore_ghosts = ignore_ghosts,
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/__init__.py", line 191, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/migrators.py", line 221, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/migrators.py", line 292, in migrate_many
result = self.migrate(migration, database)
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/migrators.py", line 125, in migrate
result = self.run(migration)
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/migrators.py", line 99, in run
return self.run_migration(migration)
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/migrators.py", line 81, in run_migration
migration_function()
File "/Users/holms/Development/xxx/xxx/settings/../../lib/south/migration/migrators.py", line 57, in <lambda>
return (lambda: direction(orm))
File "/Users/holms/Development/xxx/migrations/0002_update_order_contact_information.py", line 29, in forwards
for o in orders:
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 107, in _result_iter
self._fill_cache()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 772, in _fill_cache
self._result_cache.append(self._iter.next())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 273, in iterator
for row in compiler.results_iter():
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 680, in results_iter
for rows in self.execute_sql(MULTI):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 234, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.DatabaseError: no such column: orders_order.pre_paid
part of migration file [0002_update_order_contact_information.py] which breaks:
# encoding: utf-8
import datetime
from south.db import db
from south.v2 import DataMigration
from django.db import models
class Migration(DataMigration):
def forwards(self, orm):
Order = models.get_model('orders', 'Order')
orders = Order.all_objects.select_related('buyer')
orders = orders.filter(first_name__isnull=True)
orders = orders.filter(buyer__isnull=False)
orders = orders.exclude(payment_status="UNFINISHED")
userfields = (
'gender', 'birth_date', 'first_name', 'last_name', 'street_number',
You should not interact with your models directly like that. You use django.models, but that version of the models are in a wrong state. You want the state of the models as they were in migration 0002. The south manual states that you should access your models through the orm parameter.
Notice that we use orm.User to access the User model - this gives us the version of User from when this migration was created, so if we want to run the migration in future, it won’t get a completely different, new, User model. source
So you should rewrite the migration like this:
orders = orm.Order.objects.all()
Or even like this:
Order = orm.Order
I've model which represents settings in my system and I use it from another part of my application so that import has 3 levels WORKING CODE <- Module <- Model
Model Variables
from django.db import models
class Variables(models.Model):
key = models.CharField(max_length = 20, verbose_name = 'Variable')
value = models.CharField(max_length = 1024)
class Meta:
app_label = 'core'
def __unicode__(self):
return '%s: %s' % (self.key, self.value,)
Here is the code I'm using it from
Module variables.py
from core.models.storage import Variables
def get_var(name):
return Variables.objects.get(key = name)
Module config.py
var = get_var('some_key')
When I use this stuff from django shell everything works well but when I call get_var function I've ImportError exception
storage.py
from django.db import models
class Variables(models.Model):
key = models.CharField(max_length = 20, verbose_name = 'Variable')
value = models.CharField(max_length = 1024)
class Meta:
app_label = 'core'
def __unicode__(self):
return '%s: %s' % (self.key, self.value,)
File "monitor_cli.py", line 19, in
print worker.get_provider()
File "/home/sultan/Project/monitor/app/worker.py", line 14, in get_provider
print Variables.objects.get(pk=1)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 132, in get
return self.get_query_set().get(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 341, in get
clone = self.filter(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 550, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 568, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1172, in add_q
can_reuse=used_aliases, force_having=force_having)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1060, in add_filter
negate=negate, process_extras=process_extras)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1226, in setup_joins
field, model, direct, m2m = opts.get_field_by_name(name)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 307, in get_field_by_name
cache = self.init_name_map()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 337, in init_name_map
for f, model in self.get_all_related_m2m_objects_with_model():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 414, in get_all_related_m2m_objects_with_model
cache = self._fill_related_many_to_many_cache()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 428, in _fill_related_many_to_many_cache
for klass in get_models():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 167, in get_models
self._populate()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 61, in _populate
self.load_app(app_name, True)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 76, in load_app
app_module = import_module(app_name)
File "/usr/local/lib/python2.6/dist-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
ImportError: No module named c
Get rid of the models directory, and put all your models in a models.py file, unless you have a VERY good reason not to. Change your imports to from core.models import Variable (rename your Variables class to Variable - django models should be named as the singular not the plural).
The problem probably has to do with the fact that your models live in namespaces other than models; ie. models.storage. The django infrastructure expects certain things to be in certain places. If you're going to put models in separate namespaces like you're doing, you should be importing them from the __init__.py file within the models module. Don't do this, again, unless you have a very good reason.
Finally, when asking questions of this nature, you should provide a lot more information. You should show the code of all of the relevant files. You should provide detail on what you've done that deviates from the convention of django (in this case, a models directory rather than a models.py file). You should also show relevant settings from settings.py, in this case, your INSTALLED_APPS setting. You should also tell us what version of django and python you are using; this is sometimes relevant. More information upfront is much better than less information.