16 bit FITS image negative values misinterpreted

[clkotnik]

I have some images which, when calibrated, have some negative pixel values.  When these are displayed in AIJ, the ADU counts are very high instead of the expected low values as though the value is 64K plus the actual value.  These images have BITPIX=16 with BZERO and BSCALE set appropriately.  The astropy routines do the actual image generation.  The odd thing is that when the same image is simply converted to BITPIX=-32 with all the same pixel values, AIJ seems to work just fine showing negative values.

Is there a way to get AIJ to load the 16 bit images correctly?  Here is an example:
240323.bz2

thanks,
Cliff

[karenacollins]

Hi Cliff,

Integer fits files normally cover the range 0 to 64K. but get truncated at zero. When you say BZERO is set appropriately, does that mean it is set to 32768 (see below)? I assume BSCALE is either 1.0 or not present. If not, those could be the problem.

I tried downloading your linked image twice and it didn't finish either time. Even if it did finish, I'm not sure I am set up to decompress a "bz2" file. Could you please upload an example of a plain fits file?

Could you try opening your file in an alternate fits reader such as DS9 and see if you get the same result as AIJ? If so, then something is not getting written correctly from astropy.

Karen

[clkotnik]

Karen,
Sorry for the difficulty with BZ2.  I only used it because it was able to compress the image below the 5MB limit set by this forum.  I have placed the uncompressed image on the anonymous FTP server aavsonet.aavso.org.  Simply drop into the "aij" subdirectory and get it - 240323.0134.fits.  Once there please also find 240323.0134.f32.fits which is exactly the same image stored with BITPIX=-32.  You will see what I hope to be able to have AIJ display and process with the 16-bit image.  Note this image is an extreme case where negative values exist.  

Regarding the header values:
BITPIX      =                                   16
BZERO       =                     4459.62939453125
BSCALE      =                   0.1513604966744362
This image not only has low pixel values, but it also has a narrow range.  Hence the small BSCALE value.  All of these are set by astropy under the covers.

You asked about other programs processing this image.  I am in the process of verifying with various photometry programs.  The AAVSO Vphot, Lesve Photometry and Mira handle it ok.  MaximDL has a very similar issue to AIJ if a special setting is not turned on.  That setting is located under File-Settings on the Files tab.  It specifically mentions 16 bit images with negative values.  Once set it adds a constant to each pixel to make them non-negative.  AIJ is widely used within the AAVSO because of your exoplanet work with us, so I wanted to make sure these images worked in it.  Thanks for the reminder about DS9.  I will add it to my tests.

Thanks again,
Cliff

[clkotnik]

A quick check with DS9 shows both the 16 and 32 images to have a min of -500 and a max of 9419.  Our calibration routine clips negative values at -500.

Cliff

[karenacollins]

Thank you for pointing out this problem. It has been fixed in builds 5.3.4 and later.

Karen

[clkotnik]

Thank you for the rapid response.
Cliff