Stack Aligner does not work in a macro with AstroImageJ V5

[GeorgeF]

Previously 3.4. version I used this extract (from a very large macro which processes all my data in batch mode)

print ("Loading Sequence");
                        // I have preset that stack aligning will be done by WCS
                        call("ij.Prefs.set","stackAligner.useWCSOnly","true");
  run("Image Sequence...", "open=&file number=i starting=1 increment=1 scale=100 file=[] or=[] sort");
                        print("Images loaded..");
                        run("Align Stack", "first=1 last=i radius=8 inner=14 outer=21 use show");
                        print("Waiting for Align stack to complete");


Decided that it was probably time to upgrade to V5 and had a few false starts but started to get things working - however I am attempting to run with the latest daily build v5 and tells me "Align Stack" not found and sure enough it is not in the Astromony list of commands. I believe the correct name has always been Stack Aligner however using Macro record for running this just returned  run("Stack Aligner") - no parameters, ok, BUT if you run in a macro it STOPS and wants an OK button to be pressed. Not much use in a batch macro which will want to do this many times during the course of a run. Up to this point the transition to V5 was working but I have no work around for this problem?
Regards
George

[GeorgeF]

This idea is merely speculation on my part as I am no expert in this area.

Have been checking the source code for 3.4 vs 5 for Stack Aligner and the difference which might be causing the problem is the 3.4 is using GenericDialog - an ImageJ class which exposes details which can be used for macro recording and play back while the 5 version uses GenericSwingDialog which probably doesn't? Does this make sense?

Regards

George

[karenacollins]

Hi George,

Indeed your analysis led you to the correct conclusion that our switch to genericswingdialog has caused the problem. We need genericswingdialog to provide a more flexible panel design, but didn't realize that it did not include macro support. We will try to remedy this in the next few days and issue a daily build update.

Karen

[karenacollins]

George,

We have added macro and macro recorder support to the new Stack Aligner and Multi-Aperture panels. We are not ImageJ macro power users, so we'd appreciate any feedback from your implementation. Grab the latest daily build (v5.2.0.13 or later) to test the new capability.

For Checkboxes, you will need to use True and False as arguments.

Karen

[GeorgeF]

Hi Karen
Tried latest build  .13 and the macro recorder now produces

run("Stack Aligner", "first=1 last=i radius=8 inner=14 outer=21 use=false use_0=false use_1=false allow=false use_2=true remove=false align=false show=false");

However the process does not happen - it is as though the OK button has not been 'pressed' within the macro, so everything sits waiting indefinitely. Suspect something subtle is missing?

The Multi aperture does not produce any macro recording from the tool line selection but does for the Multiaperture selected under the Astronomy list - however it is as before

run("MultiAperture ");

as I run this by ij.Prefs.set calls this seems to work okay - but tests are not complete as this is after the Stack Aligner in the macro!

Regards

George

[keastrid]

Hi George,

Using .13 and the interactive interpreter, I cannot reproduce the issue with stack aligner. Are you not seeing any movement within the stack?

var i = 100;
run("Stack Aligner", "first=1 last=i radius=8 inner=14 outer=21 use=false use_0=false use_1=false allow=false use_2=true remove=false align=false show=false");

Kevin

[GeorgeF]

Hi Kevin

Sorry about time taken to reply - I have been setting up part of the main macro to test - yes run as a simple statement it works - however I kept adding code back up the macro and reached a point which caused it to fail.

MasterFlats = "X\:\\Master\\Flats\\" +  Fdate + "\\" + Filter + "\\";
                        MasterFlatImage = MasterFlats + "\\mflat.fits";  // need to get stats from this file
                        open(MasterFlatImage);
                        // get max values
                        //other work on master flat
                        // ... but removed for this test
                        close("*");
                       
  file = CalEntry;
                       
            list = getFileList(CalEntry);
  i = list.length;
                        print ("Loading Sequence");
                    //    file = "X:\\Images\\Variable Stars\\R Mon (VSX18829)\\2023-03-07\\B\\gain_300\\Calibrated";
                        // file = "X:\\Images\\R Mon (VSX18829)\\2023-03-07\\B\\gain_300\\Calibrated";
                        print ("Sequence File " + file);
                        // I have preset that stack aligning will be done by WCS
                        call("ij.Prefs.set","stackAligner.useWCSOnly","true");
  run("Image Sequence...", "open=&file number=i starting=1 increment=1 scale=100 file=[] or=[] sort");
                        print("Images loaded..");
                       
                        selectWindow("Calibrated");
                        run("Stack Aligner", "first=1 last=i radius=8 inner=14 outer=21 use=false use_0=false use_1=false allow=false use_2=true remove=false align=false show=false");
                               
                        print("Waiting for Align stack to complete");
                        Alignwait = 0;
  while ((isOpen("Aligned_") == false) && (Alignwait < 20)){ // again we have an asynchronous process we have to wait for
                        wait(5000);
            print("waiting...");
                        Alignwait = Alignwait + 1;
      }

THIS PART OF THE MACRO FAILS

aij.log

Exception in thread "pool-1-thread-1" java.lang.NullPointerException: Cannot invoke "ij.ImagePlus.getProcessor()" because "this.imp" is null
        at astroj.AstroStackWindow.setAstroProcessor(AstroStackWindow.java:5425)
        at astroj.AstroStackWindow.setAstroProcessor(AstroStackWindow.java:5421)
        at Astronomy.Astronomy_Listener.lambda$imageUpdated$0(Astronomy_Listener.java:72)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Exception in thread "pool-1-thread-2" java.lang.NullPointerException: Cannot invoke "ij.ImagePlus.setRoi(ij.gui.Roi)" because "this.imp" is null
        at astroj.AstroStackWindow.getBiSliderStatistics(AstroStackWindow.java:5363)
        at astroj.AstroStackWindow.updatePanelValues(AstroStackWindow.java:5505)
        at astroj.AstroStackWindow.setAstroProcessor(AstroStackWindow.java:5465)
        at astroj.AstroStackWindow.setAstroProcessor(AstroStackWindow.java:5421)
        at Astronomy.Astronomy_Listener.lambda$imageUpdated$0(Astronomy_Listener.java:72)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)


Reason appears to be the
 
open(MasterFlatImage);
close("*");

remove this and it appears to work???
Regards

George

[keastrid]

Hi George,

Is your macro writing MasterFlatImage? Opening an existing file on disk seems to work.

Kevin

[GeorgeF]

Hi Kevin

No at this point I am opening the FlatImage to read some fo the stats from the file.

I then close it - in fact my latest test shows astroimagej hanging at this point now - if run the macro installed it sometimes works other time not.

my simplest test macro looks like this

 CalEntry = "path to a folder with wcs images 32 bit";
                        print ("Calibration directory " + CalEntry);
                       
                       
                       
                        MasterFlats = "path to a single 32 bit fits image no wcs";
                       
                        open(MasterFlatImage);
                       
                       
                        close("*");
                       
  file = CalEntry;
                       
                        list = getFileList(CalEntry);
  i = list.length;
                  // these are just to we are got to this point!
                        print ("Loading Sequence");
                        print ("Sequence File " + file);
       
        run("Image Sequence...", "open=&file number=i starting=1 increment=1 scale=100 file=[] or=[] sort");
                       
        // I have preset that stack aligning will be done by WCS
        call("ij.Prefs.set","stackAligner.useWCSOnly","true");
        run("Stack Aligner", "first=1 last=i radius=8 inner=14 outer=21 use=false use_0=false use_1=false allow=false use_2=true remove=false align=false show=false");


end of test macro

Key question is where does it fail?

I have had failures in my latest tests astroimagej hanging at the close("*"); statement and at the stack aligner statement
BUT if I remove the open and close of that file it works - Stack Aligner does its thing.

Checking the macro functions on ImageJ site for open and close

open(path)
Opens and displays a tiff, dicom, fits, pgm, jpeg, bmp, gif, lut, roi, or text file. Displays an error message and aborts the macro if the specified file is not in one of the supported formats, or if the file is not found. Displays a file open dialog box if path is an empty string or if there is no argument. Use the File>Open command with the command recorder running to generate calls to this function. With 1.41k or later, opens images specified by a URL, for example open("../../images/clown.gif"). With 1.49v or later, opens a folder of images as a stack. Use open("path/to/folder","virtual") to open a folder of images as a virtual stack.

Another difference between V3.4 and V5.2 is the version of ImageJ - is the single file in a stack not just an image window and does this make a difference to the close("*") function?

Interestingly this is the only time in the full macro where I use this open command to open an image file!

Time for some research on the imagej forum as well?

However I think Stack Aligner needs to check for a NULL imp pointer to catch this exception and work out a recovery or present a clear failure message?

Regards

George

[karenacollins]

Hi George,

Instead of using close("*"), does the problem go away if you close the specific master flat image window that you are intending to close? Maybe there is a race condition and the * is actually killing the CalEntry imp as well? Kevin may have better feedback.

We both poked at this yesterday and couldn't duplicate the problem. If the above is not the solution or if Kevin does not have any better ideas, we may need some minimum required subset of the files you are using that presents the problem, along with a minimum macro that shows the problem using the files you provide.

Karen

[karenacollins]

Another question, are your images on a local drive or a network drive? If local, is it SSD or a magnetic hard drive?

Karen

[GeorgeF]

Hi Karen

I have tried all kinds of close combinations * , named flat window and () it hangs most of time!

Remove close, it sometimes works, remove flat image load and close - it works most of the time - still feel there is an element of randomness but that might me getting tired.

The all data is stored on network drive on a file server with gigabit performance.

I will send you the data files and simplfied macro via email

Regards

George

[GeorgeF]

Hi Karen

I had one last thought  and I put a wait(10000); before close of flat image and wait(10000); after image sequence load and it appears to work - all these commands are running async in threads - really messing up macros.
With ImageJ macros I can see someone has lost the plot (open and close async?? ). Really need a flag on astroimagej plugins which use threads async in normal use, then have a single thread capability when running in a macro?

regards

George