We Keep Coding

Update SQL database registers based on JSON

I have a table with 30k clients, with the ClientID as primary key.
I'm getting data from API calls and inserting them into the table using python.
I'd like to find a way to insert rows with new clients and, if the ClientID that comes with the API call already exists in the table, update the existing register with the updated information of this client.
Thanks!!

A snippet of code would be nice to show us what exactly you are doing right now. I presume you are using an ORM like SqlAlchemy? If so, then you are looking at doing an UPSERT type of an operation.
That is already answered HERE
Alternatively, if you are executing raw queries without an ORM then you could write a custom procedure and pass required parameters. HERE is a good write up on how that is done in MSSQL under high concurrency. You could use this as a starting point for understanding and then re-write it for PostgreSQL.

Creating Hypertables through SQL Alchemy

Our current project relies heavily on SQL Alchemy for table creation/data insertion. We would like to switch to timescaledb's hypertables, but it seems the recommended way to create hypertables is by executing a
create_hypertable
command. I need to be able to dynamically create tables, and so manually doing this for every table created is not really an option. One way of handling the conversion is to run a python script sending psycopg2 commands to convert all newly-created tables into hypertables, but this seems a little clumsy. Does timescaledb offer any integration with SQL Alchemy with regards to creating hypertables?

We currently do not offer any specific integrations with SQL Alchemy (broadly or specifically for creating hypertables). We are always interested in hearing new feature requests, so if you wanted to post your issue/use case on our Github it would help us keep better track of it for future work.
One thing that might work for your use case is to create an event trigger that executes on table creation. You'd have to check that it's in the correct schema since TimescaleDB creates its own chunk tables dynamically and you don't want to have them converted to hypertables.
See this answer for more info on event triggers:
execute a trigger when I create a table

Here is a practical example of using event trigger to create a hyper table:
from sqlalchemy import Column, Integer, DateTime, event, DDL, orm
Base = orm.declarative_base()
class ExampleModel(Base):
__tablename__ = 'example_model'
id = Column(Integer, primary_key=True)
time = Column(DateTime)
event.listen(
ExampleModel.__table__,
'after_create',
DDL(f"SELECT create_hypertable('{ExampleModel.__tablename__}', 'time');")
)

In Python multiple mysql db communication using ORM

Good day to All,
I need to communicate to DBs like here is the sample using python & peewee.
First i need to put an entry in the types table of the MainDb. only the Name & type am having & the ID will get auto increment after the entryin the table, then using the auto incremented id(MainID), i need to put a data entry in respective type table of the respective type db.
then i need to do some operation using the select queries with MainID in the python code and finally i need to delete the entries of the MainID in the MainDb as well as in the respective type db, if the operations in the python code gets executed successfully or not (throws any error or exception).
Currently i am using the simple db (MySQLdb) connection & close with cursor to execute the queries & to get the last auto incremented id am using lastrowid of the cursor(cursor.lastrowid).
And also i referred the question in stackoverflow as per my understanding its ok if i have an single db (like main db alone) but what am i need to do for my situation.
OS: Windows7 64bit,
Db: MySQL Db v5.7,
Python : v2.7,
peewee : v2.10
Thanks in advance

SqlAlchemy Reflection of Oracle Table Not Owned

I'm working with SQLAlchemy to run SQL queries against an Oracle database. I have read access to the database, but the user I have does not own any of the tables I'm working with.
The database updates on a regular basis, so rather than explicitly listing the MetaData, I was hoping to use reflection. I found this question, that describes an issue similar to what I'm having. However, I don't have a way to change ownership of the tables, nor modify the database in any way. I just have read access.
Is there a way to reflect Oracle tables in SQLAlchemy if I don't have ownership of those tables?
(Edit)
Example Code:
engine = create_engine('ORACLE CONNECTION STRING')
metadata = MetaData()
students = Table('students', metadata, autoload=True, autoload_with=engine)
I receive an exception of sqlalchemy.exc.NoSuchTableError: students
However, when I run the following:
results = engine.execute('SELECT * FROM students')
for r in results:
print(r)
I receive the output that I expected from the table, which is a tuple of all the fields for each row.
So instead of trying to reflect a single table, I try to reflect all of them:
metadata.reflect(bind=engine)
print(metadata.tables)
The output is immutabledict({}).
So essentially it's nothing. All of these tables are owned by user A where as I'm logging in with a read-only of user B.

You might have better luck reflecting someone else's tables if you specify the schema (account) you're targeting:
metadata.reflect(bind=engine, schema='userA')
This way, you'll reflect all readable tables belonging to 'userA'. I'm not sure why you're able to query students using engine.execute, though.

django postgresql query not working

I have a postgreSQL database that has a table foo that I've created outside of django. I used manage.py inspectdb to build the model for table foo for me. This technique worked fine when I was using MySQL but with PostgreSQL it is failing MISERABLY. The table is multiple gigabytes and I build it from a text file with PostgreSQL 'COPY'.
I can run raw queries on table foo and everything executes and expected.
For example
foo.objects.raw('bar_sql')
executes as expected.
But running queries like:
foo.objects.get(bar=bar)
throw
ProgrammingError column foo.id does not exist LINE 1: SELECT "foo"."id", "foo"."bar1", "all_...
foo doesn't innately have an id field. As I understand it django is suppose to create one. Have I some how subverted this step when creating the tables outside of django?
Queries run on models whose table was populated threw django run as expected in all cases.
I'm missing something very basic here and any help would be appreciated.
I'm using django 1.6 with postgreSQL 9.3.

Django doesn't modify your existing database tables. It only creates new tables. If you have existing tables, it usually doesn't touch them at all.
"As I understand it django is suppose to create one." --> It only adds a primary key to a table when it creates it, which means you don't need to specify that explicitly in your model, but it won't do anything to an existing table.
So if for example you later on decide to add fields to your models, you have to update your databases manually.
What you need to do in your case is that by doing manual database administration make sure that your table has a primary key, and also that the name of the primary key is "id" (although I am not sure if this is necessary, it is better to do it.) So use a database administration tool, modify your table and add the primary key, and name it id. Then it should start working.