We Keep Coding

Undo Table Conversion with Astropy

I have a FITS file with a BinTableHDU that has many entries that have been converted from digital numbers various units like Volts and Currents. I would like to turn off this conversion and access the original digital number value that was stored in the table.
This table makes use of TTYPE, TSCAL, TZERO, and TUNIT keys in the header to accomplish the conversions. So I could use these header keys to undo the conversion manually.
undo = (converted - tzero ) / tscal
Since the astropy.table.Table and astropy.table.QTable class automatically interpret these fields I can't help but think I'm overlooking a way to use astropy functions to undo the conversion. Is there an attribute or function in the astropy QTable or Table class that could let me undo the conversion automatically. (or perhaps another easier way) to undo this conversion?
EDIT: Thanks to the answer by Tom Aldcroft below.
In my situation a FITS binary table can be grabbed and put into a pandas.DataFrame either converted or unconverted using the following code:
import pandas as pd
from astropy.table import Table
from astropy.io import fits
# grab converted data in various scaled values and units volts / current
converted = Table(fits.getdata(file, 2)).to_pandas()
# grab unconverted data in its original raw form
unconverted = Table(fits.getdata(file, 2)._get_raw_data()).to_pandas()

You'll need to be using the direct astropy.io.fits interface instead of the high-level Table interface. From there something like this might work:
from astropy.io import fits
with fits.open('data.fits') as hdus:
hdu1 = hdus[1].data
raw = hdu1._get_raw_data()
See https://github.com/astropy/astropy/blob/645a6f96b86238ee28a7057a4a82adae14885414/astropy/io/fits/fitsrec.py#L1022

How to put the dates in chronological order in python CSV

So I am working on processing some data and after running my file, everything is fine except that the date are not in order. I used the code below to try putting them in order but didnt work. by the way 'updated_at' is the column I am trying to put in chronological
df = df.sort_values(by=["updated_at"], ascending=True)
Please let me know how I can make this work. I have attached a picture to for better understanding of my question.
"updated_at" column pic

We are missing a bit of the context, but it could be that the column "updated_at" is not a datetime column, but a simple string, since it looks to me it is sorted alfabetically. Check with df["updated_at"].dtype and if it's not a datetime type (it will probably be of type "object") then do
df["update_at"] = pd.to_datetime(df["update_at"])

How to set the column text format of QTableWidget?

I want to show some data in table form. I took QTableWidget for it having multiple columns. One column of it will contain time(hh:mm format).
I also want user to edit any item of table but with corresponding format.
I was able to add data in QTableWidget but i couldn't set text format of time column.
This i want to achieve so that user can edit time only in hh:mm format.
If possible please write your answer code in python.

Since you want the user enter time date, I would suggest to reuse the already existing QDateTimeEdit class in the following way:
dateTime = QDateTimeEdit();
dateTime.setDisplayFormat("hh:mm");
dateTime.setFrame(False);
myTableWidget.setCellWidget(row, column, dateTime);
The user will be able to edit the "time data" this way in your table widget. Moreover, it will be also convenient due to the steps that can be applied.
If you really insist on reinventing this yourself, you can use a QLineEdit with custom validator againt the desired hh::mm format.
dateTime = QLineEdit();
dateTime.setValidator(QRegExpValidator(QRegExp("^([0-9]|0[0-9]|1[0-9]|2[0-3]):[0-5][0-9]$")));
myTableWidget.setCellWidget(row, column, dateTime);

MySQL data type for movie times

I have chapter times in the form of HH:MM:SS. I am parsing them from a document, and I will have times as a string in the format of '00:12:14'. How would I store this in a mysql column, and then retrieve it in the required format to be able to:
1) order by time;
2) convert to a string in the above format.

I suggest you look at the MySQL time type. It will allow you to sort and format as you wish.
http://dev.mysql.com/doc/refman/5.0/en/time.html

Use the TIME type.
It allows "time values to be represented in several formats, such as quoted strings or as numbers, depending on the exact type of the value and other factors." In addition, you can perform various functions to manipulate the time.

If I have such a simple task, I choose a simple solution: I would choose the python datetime.time module (see: datetime.time) and store a TIME object using strftime.
Loading it back in is a little painful as you would have to split your string at : and then pass the values to the time constructor. Example:
def load(timestr):
hours,minutes,seconds = timestr.split(":")
return datetime.time(hours,minutes,seconds)
Hope this helps.

Modify column output for sqlform.grid() in Web2py

I have started using web2py for a web application and try to use SQLFORM.grid(...) to display a paginated listing of one of my db-table's data like in the following minimal example.
grid=SQLFORM.grid(query,
links=links,
fields=[db.example.date,db.example.foo, db.example.bar])
The db.example.date field contains a Python datetime.datetime object in UTC. At the moment it is displayed just plainly like that. However, I want to have more control about the actual output in a way that I can set the local timezone and modify the output string to have something like "2 hours ago".
As seen in another question[0] I can use the links to insert new columns. Unfortunately I can't seem to sort the rows by a field I have inserted in such way. Also, they are inserted on the right instead of actually replacing my first column. So that does not seem to be a solution.
To sum it up: How do I gain control about the way db.example.date is printed out in the end?
[0] Calculated Fields in web2py sqlgrid

You can achieve your goal when you define the table in your model. The represent parameter in the Field constructor that you used in define_table will be recognized by the SQLFORM.grid. For example, if you wanted to just print the date with the month name you could put the following in your model.
Field('a_date', type='date', represent=lambda x, row: x.strftime("%B %d, %Y")),
your function could also convert to local time.

You need to use prettydate to change the datetime arid format in a humanized string, and call it in the represent parameter of your Field() descriptor. For example :
from gluon.tools import prettydate
db.example.date.represent = lambda v,r: prettydate(r.date)
That way, any display of the db.example.date would be displayed humanized, including through SQLFORM.grid

If you don't want to have the date always represented in this way as per David Nehme's answer. Just before your grid creation, you can set the db.table.field.represent in the controller.
db.example.date.represent = lambda value, row: value.strftime("%B %d, %Y")
followed by.
grid = SQLFORM.grid(query,....
I use this often when I join tables. If there is a row.field in the represent from the model file it breaks because it then must be more specific, row.table.field.