We Keep Coding

Why does COUNT(*) in my SQL query return several values? How to get one single value for total rows found?

I have a Django class getting some order data from a PostreSQL database. I need to get the number of rows found in a query.
Trying to get the found rows count from the following query with COUNT(*):
When I print the result from the query above, I get a lot of data:
I only want to get a single number, the count of the total rows found and loaded by the select query above. How do I achieve this?
Keep in mind that I'm pretty new to SQL, so I might be missing something obvious to you.
Thanks!

COUNT(expr) will return a count of the number of non-null values of expr in the rows that are retrieved by the SELECT.
In your case, you're grouping a bunch togheter and it returns the count for each grouped result row.
You'll probably get the result you're looking for, wrapping it in a subselect.
For example:
SELECT COUNT(*)
FROM (SELECT o.* FROM your_table o ....)

Python - Regex pattern matching issue after executing a sqlite query

I am trying to extract some information from one table and store it in another table using Sqlite and Python. Table 1 contains a list of websites in the form of (www.abc.com). I am trying to extract the (abc) part from each row and store it in Table 2 which also maintains a count for each site. If the site already exist in Table 2, then it just increment the count.
Here the code I have:
p = re.compile('^.+\.([a-zA-Z]+)\..+$')
for row in c.execute('SELECT links FROM table1'):
link = p.match(row[0])
if link.group(1):
print(link.group(1))
c.execute('SELECT EXISTS(SELECT 1 FROM table2 WHERE site_name = ?)', (link.group(1), ))
When I run the script, it will only execute once, then I get:
Traceback (most recent call last):
File "test.py", line 43, in <module>
link = p.match(row[0])
TypeError: expected string or buffer
If I comment out the c.execute line, all the site names are printed properly. I am new to Python and Sqlite, so I am not sure what the problem is.
Any help will be great, thanks in advance.

The problem is that you're iterating over a cursor whose rows contain a single string:
for row in c.execute('SELECT links FROM table1'):
… but then, in the middle of the iteration, you change it into a cursor whose rows consist of a single number:
c.execute('SELECT EXISTS(SELECT 1 FROM table2 WHERE site_name = ?)', (link.group(1), ))
So, when you get the next row, it's going to be [1] instead of ['http://example.com'], so p.match(row[0]) is passing the number 1 to match, and it's complaining that 1 is not a string or buffer.
For future reference, it's really helpful to debug things by looking at the intermediate values. Whether you run in the debugger, or just add print(row) calls and the like to log what's going on, you'd know that it works the first time through the loop, but that it fails the second time, and that row looked like [1] when it failed. That would make it much easier for you to track down the problem (or allow you to ask a better question on SO, because obviously you still won't be able to find everything yourself.)
You could fix this in (at least) three ways, in increasing order of "goodness if appropriate":
Fetch all of the values from the first query, then loop over those, so your second query doesn't get in the way.
Use a separate cursor for each query, instead of reusing the same one.
Don't make the second query in the first place—it's a SELECT query and you aren't doing anything with the rows, so what good is it doing?

The inner execute is probably stepping on your cursor iterator state. Try creating a second cursor object for that query.

Is sqlite3 fetchall necessary?

I just started using sqlite3 with python .
I would like to know the difference between :
cursor = db.execute("SELECT customer FROM table")
for row in cursor:
print row[0]
and
cursor = db.execute("SELECT customer FROM table")
for row in cursor.fetchall():
print row[0]
Except that cursor is <type 'sqlite3.Cursor'> and cursor.fetchall() is <type 'list'>, both of them have the same result .
Is there a difference, a preference or specific cases where one is more preferred than the other ?

fetchall() reads all records into memory, and then returns that list.
When iterating over the cursor itself, rows are read only when needed.
This is more efficient when you have much data and can handle the rows one by one.

The main difference is precisely the call to fetchall(). By issuing fetchall(), it will return a list object filled with all the elements remaining in your initial query (all elements if you haven't get anything yet). This has several drawbacks:
Increments memory usage: by storing all the query's elements in a list, which could be huge
Bad performance: filling the list can be quite slow if there are many elements
Process termination: If it is a huge query then your program might crash by running out of memory.
When you instead use cursor iterator (for e in cursor:) you get the query's rows lazily. This means, returning one by one only when the program requires it.
Surely that the output of your two code snippets are the same, but internally there's a huge perfomance drawback between using the fetchall() against using only cursor.
Hope this helps!

how to count value returned from select query in python

I want to count the total number of rows returned by query. I am able to retrieved rows returned by query but what if i need to work in case when no data exits. that is when query returns no va from database.
the code i used to solve this problem is :
try:
cur.execute(query)
id = cur.fetchone()[0]
if(id is None):
return '-1'
else:
return id
But this doenst help when no values is returned selected from query.(when condition doesnt meet criteria defined in select statement)

cur.fetchall() will give you a sequence of all the rows. You can look at the length of that sequence to see if any rows were returned. This works for small result sets, but may not be ideal for queries that return large amounts of data.
Alternatively, you can look at cur.rowcount. Rowcount will return the number of rows in the query, or -1 if the number cannot be determined. It is up to the implementation to set rowcount; several popular python database modules (most notably sqlite3), do not set rowcount for all queries. For modules that do not set rowcount, the only way to count the number of result rows is to load the full result set into memory.

Python: Number of rows affected by cursor.execute("SELECT ...)

How can I access the number of rows affected by:
cursor.execute("SELECT COUNT(*) from result where server_state='2' AND name LIKE '"+digest+"_"+charset+"_%'")

Try using fetchone:
cursor.execute("SELECT COUNT(*) from result where server_state='2' AND name LIKE '"+digest+"_"+charset+"_%'")
result=cursor.fetchone()
result will hold a tuple with one element, the value of COUNT(*).
So to find the number of rows:
number_of_rows=result[0]
Or, if you'd rather do it in one fell swoop:
cursor.execute("SELECT COUNT(*) from result where server_state='2' AND name LIKE '"+digest+"_"+charset+"_%'")
(number_of_rows,)=cursor.fetchone()
PS. It's also good practice to use parametrized arguments whenever possible, because it can automatically quote arguments for you when needed, and protect against sql injection.
The correct syntax for parametrized arguments depends on your python/database adapter (e.g. mysqldb, psycopg2 or sqlite3). It would look something like
cursor.execute("SELECT COUNT(*) from result where server_state= %s AND name LIKE %s",[2,digest+"_"+charset+"_%"])
(number_of_rows,)=cursor.fetchone()

From PEP 249, which is usually implemented by Python database APIs:
Cursor Objects should respond to the following methods and attributes:
[…]
.rowcount
This read-only attribute specifies the number of rows that the last .execute*() produced (for DQL statements like 'select') or affected (for DML statements like 'update' or 'insert').
But be careful—it goes on to say:
The attribute is -1 in case no .execute*() has been performed on the cursor or the rowcount of the last operation is cannot be determined by the interface. [7]
Note:
Future versions of the DB API specification could redefine the latter case to have the object return None instead of -1.
So if you've executed your statement, and it works, and you're certain your code will always be run against the same version of the same DBMS, this is a reasonable solution.

The number of rows effected is returned from execute:
rows_affected=cursor.execute("SELECT ... ")
of course, as AndiDog already mentioned, you can get the row count by accessing the rowcount property of the cursor at any time to get the count for the last execute:
cursor.execute("SELECT ... ")
rows_affected=cursor.rowcount
From the inline documentation of python MySQLdb:
def execute(self, query, args=None):
"""Execute a query.
query -- string, query to execute on server
args -- optional sequence or mapping, parameters to use with query.
Note: If args is a sequence, then %s must be used as the
parameter placeholder in the query. If a mapping is used,
%(key)s must be used as the placeholder.
Returns long integer rows affected, if any
"""

In my opinion, the simplest way to get the amount of selected rows is the following:
The cursor object returns a list with the results when using the fetch commands (fetchall(), fetchone(), fetchmany()). To get the selected rows just print the length of this list. But it just makes sense for fetchall(). ;-)
print len(cursor.fetchall)
# python3
print(len(cur.fetchall()))

To get the number of selected rows I usually use the following:
cursor.execute(sql)
count = len(cursor.fetchall())

when using count(*) the result is {'count(*)': 9}
-- where 9 represents the number of rows in the table, for the instance.
So, in order to fetch the just the number, this worked in my case, using mysql 8.
cursor.fetchone()['count(*)']