We Keep Coding

how to use select query inside a python udf for redshift?

I tried uploading modules to redshift through S3 but it always says no module found. please help
CREATE or replace FUNCTION olus_layer(subs_no varchar)
RETURNS varchar volatile AS
$$
import plpydbapi
dbconn = plpydbapi.connect()
cursor = dbconn.cursor()
cursor.execute("SELECT count(*) from busobj_group.olus_usage_detail")
d=cursor.fetchall()
dbconn.close()
return d
$$
LANGUAGE plpythonu;
–

You cannot do this in Redshift. so you will need to find another approach.
1) see here for udf constraints http://docs.aws.amazon.com/redshift/latest/dg/udf-constraints.html
2) see here http://docs.aws.amazon.com/redshift/latest/dg/udf-python-language-support.html
especially this part:
Important Amazon Redshift blocks all network access and write access
to the file system through UDFs.
This means that even if you try to get around the restriction, it won't work!
If you don't know an alternative way to get what you need, you should ask a new question specifying exactly what your challenge is and what you have tried, (leave this question ans answer here for future reference by others)

It can't connect to DB inside UDF, Python functions are scalar in Redshift, meaning it takes one or more values and returns only one output value.
However, if you want to execute a function against a set of rows try to use LISTAGG function to build an array of values or objects (if you need multiple properties) into a large string (beware of string size limitation), pass it to UDF as parameter and parse/loop inside the function.

Amazon has recently announced the support for Stored Procedures in Redshift. Unlike a user-defined function (UDF), a stored procedure can incorporate data definition language (DDL) and data manipulation language (DML) in addition to SELECT queries. Along with that, it also supports looping and conditional expressions, to control logical flow.
https://docs.aws.amazon.com/redshift/latest/dg/stored-procedure-overview.html

str object is not callable when inserting into SQLite database

I am working on a temperature sensor network using onewire temperature sensors that runs on a Raspberry Pi 2. I am following this tutorial and as I was going along, I realized that his setup is for one temperature sensor, whereas my setup needs to work with multiple sensors.
As a result of having multiple sensors, I also need to be able to differentiate the sensors from one another. To do this, I want to have 3 columns in the SQLite table. I am encountering the error when I execute the Python script that is supposed to log the readout from the sensor, the date and time, and the sensor name.
Here is the problem, when I am configuring the python script to write three values to the table, I get an error.
Here is the code that I am getting an error when executing
#!/usr/bin/env python
import sqlite3
import os
import time
import glob
# global variables
speriod=(15*60)-1
dbname='/var/www/templog.db'
# store the temperature in the database
def log_temperature(temp):
conn=sqlite3.connect(dbname)
curs=conn.cursor()
sensor1 = 'Sensor1'
curs.execute("INSERT INTO temps values(datetime('now'), (?,?))" (temp, sensor1))
# commit the changes
conn.commit()
conn.close()

"INSERT INTO temps values(datetime('now'), (?,?))" (temp, sensor1)
Breaking this down you will see that this creates a string and then the parenthesis appears to Python to be a function call. However this is nonsensical because you have a string that you are trying to call like it is a function. Hence the error you get about str not being callable, this is definitely a bit cryptic if you are not experienced with Python. Essentially you are missing a comma:
curs.execute("INSERT INTO temps values(datetime('now'), (?,?))", (temp, sensor1))
Now you will get the ? placeholders correctly filled in.
Often the "str is not callable" error will be a result of typos such as this or duplicated variable names (you think you are calling a function but the variable really contained a string), so start by looking for those problems when you see this type of error.

You have to put a , there:
curs.execute("INSERT INTO temps values(datetime('now'), (?,?))" , (temp, sensor1))
From the documentation
Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method
As you can see you need to provide the tuple as the second argument to the function execute

Commiting objects with long integers to MYSQL with SQLAlchemy

I am trying to add a large integer to a MySQL table with SQLAlchemy. As this answer explains, you cannot pass Integer a length argument like you can String. So following that answer I've defined my column with mysql.INTEGER like so:
from sqlalchemy.dialects import mysql
uniqueid = Column(mysql.INTEGER(20))
When I try to commit an object with a 14 digit uniqueid, however, I get the following error message: DataError: (DataError) (1264, "Out of range value for column 'uniqueid' at row 1"). When I try a shorter integer that is not a long, it has no problem committing the same object to the SQL database. I am running python 2.7, other discussions of the long type indicate that it should not behave any differently than int except for printing an L at the end of the number. One final piece of information is if I set the uniqueid to the same short number but make it a long, as in uniqueid = long(32423), I can still commit the object to the SQL database.

I did not solve the mystery of why the mysql.INTEGER class will not work with numbers that have to be long in python 2.7, but the practical solution is to use SQLalchemy's BigInteger class, which as the name suggests can handle big integers, including long.

Values from models from Database (Django)

I have this model
class Type(models.Model):
type = models.CharField(max_length=50)
value = models.CharField(max_length=1)
And into it, I have some data from an sql file:
INSERT INTO quest_type (type, value) VALUES ('Noun', '1');
INSERT INTO quest_type (type, value) VALUES ('Adjective', '2');
INSERT INTO quest_type (type, value) VALUES ('Duration', '3');
How do I access these values in the python shell? For example, if I know the type, how do I get the value (and vice verse)? I'm not sure how the syntax works.

you should be able to get that with
Type.objects.filter(type=typeImInterestedIn)
A couple of things to be leary of:
-you probably want to avoid manually writing to a DB that you're using an ORM in. It just creates potential for mismatches.
-naming an object Type is little problematic since it's so close to the python native function type.

It's unclear from your question how much about databases you understand, so I apologize if this answer is too basic for you (if so, please edit your question to include information about what actual database engine you're using and show some sample code trying to read from the database).
The SQL file you have is not the same as an SQL database. It is a series of commands that will create records in an SQL database. First you must install and configure a database engine on your machine then "run" that .sql file so that the records are created in the database.
After you have an actual database, you will have to configure Django so that it knows what kind of SQL engine you're using and the name and location of the database.
Finally, once the database is created and Django configured to talk to the engine, you will write python code to instantiate an instance of the Type class, read a record from the database, and inspect the values.
Also, let me point out that Type is a really, really bad name for a class in any programming language, and type and value are both bad names for columns in SQL databases.

If you are using python shell from django (python manage.py shell) firstly You have to import to your namespace your model, so type from my_app.models import Type.
Now if You want to get only one object from db syntax is:
result = Type.objects.get(type='your_query')
If you want to fetch more then one object syntax goes like this:
result = Type.objects.filter(type='your_query')
second method returns list instead of single object
To loop through list after using filter write:
for item in result:
item.value #will print values from matched rows

Insert into sqlite3 a null date from pyramid framework using sqlalchemy

python 2.7
pyramid 1.3a4
sqlalchemy 7.3
sqlite3.7.9
from sqlite prompt > I can do:
insert into risk(travel_dt) values ('')
also
insert into risk(travel_dt) values(Null)
Both result in a new row with a null value for risk.travel_dt but when I try those travel_dt values from pyramid, Sqlalchemy gives me an error.
In the first case, I get sqlalchemy.exc.StatementError:
SQLite Date type only accepts python date objects as input
In the second case, I get Null is not defined. When I use "Null", I get the first case error
I apologize for another question on nulls: I have read a lot of material but must have missed something simple. Thanks for any help
Clemens Herschel

While you didn't provide any insight into the table definition you're using or any example code, I am guessing the issue is due to confusing NULL (the database reserved word) and None (the Python reserved word).
The error message is telling you that you need to call your SQLA methods with valid python date objects, rather than strings such as "Null" or ''.
Assuming you have a Table called risk containing a Column called travel_dt, you should be able to create a row in that table with something sort of like:
risk.insert().values(travel_dt=None)
Note that this is just a snippet, you would need to execute such a call within an engine context like that defined in the SA Docs SQL Expression Language Tutorial.