ALOS2 stackStripMap processing questions

Added by HY Chen 5 months ago

Hi all,
I tried to use the Stripmap stack processor to process ALOS2 data. I followed the basic workflow introduced in Readme.txt file under StripmapStack folder:
step 1. Because I unzip the ALOS*.zip first, therefor I using unpackFrame_ALOS2.py.
step 2. fixImageXml.py -i demLat_N26_N28_Lon_E104_E106.dem.wgs84 -f
step 3. stackStripMap.py -s RSLC -x '26.475 26.616 105.067 105.227 ' -d demLat_N26_N28_Lon_E104_E106.dem.wgs84 -m 20191124 -t 500 -b 300 -a 2 -r 2 -W slc -u snaphu –nofocus

However, There are some errors:
1. After the step3, the error :"the TypeError: object of type 'NoneType' has no len()" occurs,
then I refer to the topic http://earthdef.caltech.edu/boards/4/topics/3261?r=3263#message-3263,I add the code: obj.extractDoppler() to unpackFrame_ALOS2.py.
2. the error
Traceback (most recent call last):
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/stackStripMap.py", line 358, in <module>
main()
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/stackStripMap.py", line 328, in main
if inps.sensor.lower() 'uavsar_stack': # don't try to calculate baselines for UAVSAR_STACK data
AttributeError: 'NoneType' object has no attribute 'lower'

After I annotated the corresponding lines in stackStripMap.py (line328-330,I added it below), It alos works and the run_file file was generated. #################################################################################################
if inps.sensor.lower() 'uavsar_stack': # don't try to calculate baselines for UAVSAR_STACK data
pairs = selectPairs(inps,stackMasterDate, slaveDates, acquisitionDates,doBaselines=False)
else:
pairs = selectPairs(inps,stackMasterDate, slaveDates, acquisitionDates,doBaselines=True)
print ('number of pairs: ', len(pairs)) ###################################################################################################

3. Firstly, I executed ./run_files/run_1* , the "RSLC_crop" folder was generated and I check the .slc using the mdx.py, It works good.
However, When I executed ./run_files/run_2
, a new error occurred: ###################################################################################################
Traceback (most recent call last):
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/stripmapWrapper.py", line 155, in <module>
main(args.start,args.end)
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/stripmapWrapper.py", line 146, in main
cfgParser.runCmd()
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/stripmapWrapper.py", line 51, in runCmd
func_modules.main(self.funcParams[section])
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/createWaterMask.py", line 116, in main
geo2radar(geo_file, inps.outfile, inps.latName, inps.lonName)
File "/data/users/CHDHPC/chenhengyi/anaconda3/share/isce2/stripmapStack/createWaterMask.py", line 103, in geo2radar
sw.toRadar(geo_file, lat_file, lon_file, rdr_file)
File "/data/users/CHDHPC/chenhengyi/anaconda3/lib/python3.7/site-packages/isce/components/contrib/demUtils/SWBDStitcher.py", line 105, in toRadar
latim.load(latin + '.xml')
File "/data/users/CHDHPC/chenhengyi/anaconda3/lib/python3.7/site-packages/isce/components/iscesys/Component/Configurable.py", line 1407, in load
tmpProp, tmpFact, tmpMisc = FP.parse(filename)
File "/data/users/CHDHPC/chenhengyi/anaconda3/lib/python3.7/site-packages/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
root = ET.parse(filename)
File "/data/users/CHDHPC/chenhengyi/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
tree.parse(source, parser)
File "/data/users/CHDHPC/chenhengyi/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 587, in parse
source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/data/users/CHDHPC/chenhengyi/tq_alos2_isce/unzip/RSLC_crop/geom_master/lat.rdr.xml' ###################################################################################################
I found the topic in http://earthdef.caltech.edu/boards/4/topics/3255,
However, my .netrc file is clearly checked and the the automatic downloads operation seems correct(401 Unauthorized).

So, how to solve this problem?

Looking forward to your answer. Thank you!

fix1.PNG (27.3 kB)

2.PNG (94.4 kB)

3.PNG (61.5 kB)


Replies (6)

RE: ALOS2 stackStripMap processing questions - Added by Joaquin Escayo 5 months ago

Hi there!

I'm also trying to do a strimap processing and had some similar issues as yours. For the run_2 I think it's failing on the watermask creation. It is looking for lat and lon files in a wrong directory, it also happens with Envisat processing. To fix it check the file configs/config_master_XXXXXX and in the second function CreateWaterMask you must include directory "merged" before geom_master in lat_file, lon_file and output. For example you must change from this:

[Function-2]
createWaterMask :
dem_file : /home/subsidence/procesados/isce_test/dem/demLat_N27_N30_Lon_W019_W017.dem.wgs84
lat_file : /home/subsidence/procesados/isce_test/geom_master/lat.rdr
lon_file : /home/subsidence/procesados/isce_test/geom_master/lon.rdr
output : /home/subsidence/procesados/isce_test/geom_master/waterMask.rdr

to this one:

[Function-2]
createWaterMask :
dem_file : /home/subsidence/procesados/isce_test/dem/demLat_N27_N30_Lon_W019_W017.dem.wgs84
lat_file : /home/subsidence/procesados/isce_test/merged/geom_master/lat.rdr
lon_file : /home/subsidence/procesados/isce_test/merged/geom_master/lon.rdr
output : /home/subsidence/procesados/isce_test/merged/geom_master/waterMask.rdr

After that you should be able to finish step 2 but you'll not have the waterMask.rdr in geom_master folder, so you can do it by calling gdal as in the function 1 (you can copy from the terminal the command for other files like lat.rdr and change input and output). I don't know if this behaviour is a bug, but this seems to solve it.

Hope it helps

Joaquín

RE: ALOS2 stackStripMap processing questions - Added by HY Chen 5 months ago

Thanks a lot! It works !

RE: ALOS2 stackStripMap processing questions - Added by francesco menniti 2 months ago

HI!
I am also having a similar problem. I'm trying to process Cosmoskymed images with stackStripmap and then use STAMPS.
I started processing from .h5 images and I have used unpackFrame_CSK to convert them to SLC.

Afterwards when I run the stackStripMap.py -s /root/elaborazone/slc_output -d ....... demLat*.dem.wgs84 -t 250 -b 600 -x '....' --nofocus -z

I get this error

['20190626', '20190712', '20190728', '20190813', '20190829', '20190930', '20191117', '20191203', '20200205', '20200221', '20200324', '20200511', '20200612']
Traceback (most recent call last):
File "/root/isce2/contrib/stack/stripmapStack/stackStripMap.py", line 363, in <module>
main()
File "/root/isce2/contrib/stack/stripmapStack/stackStripMap.py", line 333, in main
if inps.sensor.lower() == 'uavsar_stack': # don't try to calculate baselines for UAVSAR_STACK data
AttributeError: 'NoneType' object has no attribute 'lower'

can you kindly help me to solve this problem?
Thanks a lot!

RE: ALOS2 stackStripMap processing questions - Added by Joaquin Escayo 2 months ago

Hi there Francesco!

I had this same problem processing Envisat data. If I remember well the problem is because no sensor or looks are defined. Since there are no CSK sensor defined in stackStripMap.py try to define the -a and -r parameters using values that produces square pixels.

Hope it helps

RE: ALOS2 stackStripMap processing questions - Added by francesco menniti 2 months ago

thank you for quick reply!
now works!
I used stackStripMap.py -s /root/elaborazone/slc_output -d ....... demLat*.dem.wgs84 -t 250 -b 600 -x '....' --nofocus -z -a 1 -r 1 -S'CSK'

I entered -S 'CSK' because it didn't work without it. I know that cosmo is not a defined sensor.

however, after processing, I noticed that in the merged/slc folder the files for each date weigh only 80 kb. In fact the .slc .vrt .full files are missing etc .. with the exception of a single folder containing master immage .hdr .slc .vrt data and the foldersMasterShelve and slaaveShelve.

do you have any idea how i could solve it?

thank you so much.

RE: ALOS2 stackStripMap processing questions - Added by francesco menniti 2 months ago

I just realized that there is a error in run 6

root/ELABORAZIONE_ASC/SLC_OUTPUT3/refineSlaveTiming/pairs/20200511_20200612
['_MutableMapping__marker', '__abstractmethods__', '__class__', '__contains__', '__del__', '__delattr__', '__delitem__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__setitem__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__weakref__', '_abc_cache', '_abc_negative_cache', '_abc_negative_cache_version', '_abc_registry', '_protocol', 'cache', 'clear', 'close', 'dict', 'get', 'items', 'keyencoding', 'keys', 'pop', 'popitem', 'setdefault', 'sync', 'update', 'values', 'writeback']
Traceback (most recent call last):
File "/root/isce2/contrib/stack/stripmapStack/invertMisreg.py", line 207, in <module>
main()
File "/root/isce2/contrib/stack/stripmapStack/invertMisreg.py", line 147, in main
A, B, Laz, Lrg = design_matrix(pairDirs)
File "/root/isce2/contrib/stack/stripmapStack/invertMisreg.py", line 124, in design_matrix
azOff, rgOff = extract_offset(pairDirs[ni])
File "/root/isce2/contrib/stack/stripmapStack/invertMisreg.py", line 74, in extract_offset
return azCoefs.flatten(0), rgCoefs.flatten(0)
ValueError: Non-string object detected for the array ordering. Please pass in 'C', 'F', 'A', or 'K' instead

do you have any idea how i could solve it? I could be a problem with the config files?

(1-6/6)