Downloading incorrect restituted orbit when running topsStack

Added by Cheryl Tay 5 months ago

Hi ISCE developers,

I encountered this error when running the topsStack processor, run_4_geo2rdr_resample, where IW{i}.xml files were missing from the slaves directory for just one of the dates (20200424):

Traceback (most recent call last):
 File "/shared/isce2/share/isce2/topsStack/SentinelWrapper.py", line 251, in <module>
  main(args.start,args.end)
 File "/shared/isce2/share/isce2/topsStack/SentinelWrapper.py", line 242, in main
  cfgParser.runCmd()
 File "/shared/isce2/share/isce2/topsStack/SentinelWrapper.py", line 54, in runCmd
  func_modules.main(self.funcParams[section])
 File "/shared/isce2/share/isce2/topsStack/geo2rdr.py", line 226, in main
  slave = ut.loadProduct(os.path.join(inps.slave, 'IW{0}.xml'.format(swath)))
 File "/shared/isce2/share/isce2/topsStack/s1a_isce_utils.py", line 26, in loadProduct
  obj = pm.loadProduct(xmlname)
 File "/shared/isce2/lib/python3.8/site-packages/isce/components/iscesys/Component/ProductManager.py", line 25, in loadProduct
  self.load(filename)
 File "/shared/isce2/lib/python3.8/site-packages/isce/components/iscesys/Component/Configurable.py", line 1407, in load
  tmpProp, tmpFact, tmpMisc = FP.parse(filename)
 File "/shared/isce2/lib/python3.8/site-packages/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
  root = ET.parse(filename)
 File "/shared/isce2/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
  tree.parse(source, parser)
 File "/shared/isce2/lib/python3.8/xml/etree/ElementTree.py", line 584, in parse
  source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/shared/stack_singapore_p171/slaves/20200424/IW2.xml'

Upon checking the run_2_unpack_slave_slc log for 20200424, there seems to be an issue with the restituted orbit downloaded for this date:

Setting IPF version to : 003.20
Extracting orbit from Orbit File: /shared/stack_singapore_p171/orbits/20200424/S1A_OPER_AUX_RESORB_OPOD_20200424T163824_V20200424T112525_20200424T144255.EOF
Time stamp (2020-04-24 11:25:22.578436) falls outside of the interpolation interval [2020-04-24 11:25:25.481888:2020-04-24 11:26:45.481888]

The affected SAFE file is: S1A_IW_SLC__1SDV_20200424T112520_20200424T112547_032268_03BBB1_7546.SAFE
Downloaded restituted orbit is: S1A_OPER_AUX_RESORB_OPOD_20200424T163824_V20200424T112525_20200424T144255.EOF

Could the "falls outside of the interpolation interval" be because the SAFE file start and end times are not completely within the orbit file start and end times i.e. SAFE file start time is before the orbit file start time? For the orbit download process (please correct me if I got this wrong):

Here is a short list of orbit files from https://qc.sentinel1.eo.esa.int/aux_resorb/?validity_start=2020&validity_start=2020-04&validity_start=2020-04-22..2020-04-25&sentinel1__mission=S1A&page=3, where the most recent in the list passed the condition and was downloaded, and there is also an earlier version which will capture the SAFE file start and end times:
S1A_OPER_AUX_RESORB_OPOD_20200424T163824_V20200424T112525_20200424T144255 --> downloaded
S1A_OPER_AUX_RESORB_OPOD_20200424T153918_V20200424T112525_20200424T144255
S1A_OPER_AUX_RESORB_OPOD_20200424T151904_V20200424T112525_20200424T144255
S1A_OPER_AUX_RESORB_OPOD_20200424T135320_V20200424T094640_20200424T130410 --> desired?

I have attached the logs for run_2_unpack_slave_slc and run_4_geo2rdr_resample for reference.

After which, I tried replacing the 20200424 orbit file with the 'desired' one and managed to rerun the topsStack processor from start to end successfully with no errors. Please may I check if:
  • The orbit file is indeed the cause of the error?
  • And if so, would a stricter filter for orbit downloads using both SAFE file start and end times help or lead to other repercussions further down the stack processing?

Thank you so much in advance!

Best,
Cheryl

stack_run2p12.log - Logs for 20200424 run_2_unpack_slave_slc (5.1 kB)

stack_run4p12.log - Logs for 20200424 run_4_geo2rdr_resample (2.5 kB)


Replies (3)

RE: Downloading incorrect restituted orbit when running topsStack - Added by Piyush Agram 5 months ago

Orbit reader adds a margin of 60 secs before and after for reliable interpolation.

https://github.com/isce-framework/isce2/blob/fb7ad09211b6cd2fc51b8da9e87eb5f4aab77824/components/isceobj/Sensor/TOPS/Sentinel1.py#L828

If you use add a similar margin or even +/- 2 mins when looking for files to download that should solve the problem.

RE: Downloading incorrect restituted orbit when running topsStack - Added by Cheryl Tay 5 months ago

Hi Piyush,

Thanks for your suggestion and quick reply! May I ask if this fix related to extractPreciseOrbit is indeed only for downloading precise orbits and will not affect the downloading of restituted orbits? I'm asking about this because the error I encountered was only for a recent Sentinel-1 scene where precise orbits are not yet available. Downloads for precise orbits for the rest of the earlier scenes are good.

I tried changing the hardcoded margin=60.0 to margin=120.0 as suggested, and ran stackSentinel.py again. Still the same error encountered due to an unsuitable restituted orbit file.

Best,
Cheryl

RE: Downloading incorrect restituted orbit when running topsStack - Added by Piyush Agram 5 months ago

I think you are making changes in the wrong script. You want to update the download script, not the reader.

(1-3/3)