Chapter 1. Introduction

This document provides an introductive description of GRASP software package for general users. At the same time, it is expected that the users interested in the package are familiar with the field of atmospheric remote sensing and have, at least, basic of understanding of the problematic. The document does not include the details description of the scientific algorithm. The relevant scientific content can be found in publication following the citations provided in the text below. This document does not include the elaborated technical details either (for developers). That information can be found on GRASP technical documentation, provided on line (www.grasp-open.com/tech-doc). This description is aimed to explain how to handle the software package and orient the qualified user regarding actions needed for adapting the software to a specific application and for contributing into the evolution and the improvement of the GRASP software and the overall concept. Additionally, for extra services such as code adaptations, specific developments or code optimization for specfic purposes, GRASP SAS company can be contacted (www.grasp-sas.com/tech-doc).

1.1. Scientific background and heritage

GRASP (Generalized Retrieval of Atmosphere and Surface Properties), introduced by Dubovik et al. (2014), is the first unified algorithm and a software package developed for retrieving atmospheric properties from wide variety of remote sensing observations including satellite, ground-based and airborne passive and active measurements of atmospheric radiation and their combinations.

GRASP relies on the heritage of retrieval advances [Dubovik and King, 2000, Dubovik et al. 2000, 2002a,b. 2009] implemented for AERONET (see Holben et al., 1998) a worldwide network of over 300 radiometer sites that generate the data used to validate nearly all satellite observations of atmospheric aerosols. The AERONET retrievals derive detailed aerosol properties (Dubovik et al. 2002a) including absorption, providing information of vital importance for reducing uncertainty in assessments of climate change. The concept of GRASP was proposed in the recent efforts by Dubovik et al. (2011) to develop the algorithm for improved aerosol retrieval from the French Space Agency’s PARASOL imager (see Tanre et al. 2011) over bright surfaces like deserts where high surface reflectance dwarfs the signal from aerosols. In these efforts several principles used in AERONET retrieval concept were used and applied for PARASOL satellite retrieval. Then, in return, several new features of newly developed PARASOL retrieval by Dubovik et al. (2011) appeared to be useful for improving ground-based measurements interpretation by combining observations of radiometer and lidar [Lopatin et al., 2013]. Finally, the structure of the algorithm was adapted to convenient and efficient application with diverse remote sensing observations and their combinations.

1.2. Generalized aspects of GRASP algorithm and package

The important feature of the GRASP is that both the core scientific algorithms and the whole package are based on several generalization principles with the idea of developing a scientifically rigorous, versatile, practically efficient, transparent, and accessible algorithm.

There are several “layers” of complimentary generalizations used in the GRASP designs, that are outlined in the diagram shown in Figure 1.1, “Structure of the GRASP software package”

Figure 1.1. Structure of the GRASP software package

Structure of the GRASP software package


1.2.1. Generalized approach of numerical inversion

The corner stone of the GRASP is the used approach for implementing numerical inversion that is highly elaborated and is highly abstract in the sense that it is not linked in any way to specific measurement type. Here are the key elements of the inversion approach employed at the GRASP package:

  • The main part of the inversion concept is formulated for inverting abstract "indirect measurements" independent of the physical nature;

  • The used mathematical inversion formalism (Dubovik 2004, Dubovik et al. 2011, etc.) complementarily unites advantages of a variety of practical inversion approaches of known mathematical inversion procedures;

  • The specific numerical procedure is defined as Multi-Term LSM (Least Square Method) statistically optimized fitting of "positively redundant [1] set of observations". The Multi-Term LSM follows general Least Square concept, however it explicitly considers all inverted data (both the actual observations and used a priori constraints) as "observations" known with different levels of accuracies. As a result, the concept is highly practical:

    1. it benefits from numerous known fundamental developments relevant to LSM (i.e. no need to invent new fundamental principles of optimized inversion);

    2. the methodology is highly suitable for inverting combined data (both observations and a priori constraints).

1.2.2. Practical generalization of the algorithm for atmospheric remote sensing

Though the numerical inversion is highly abstract of the measurement type, and can be applied to any indirect measurements, the GRASP package was developed for application in the field of atmospheric remote sensing with pursuing the following generalization ideas:

  • Making GRASP instrument-independent algorithm, as a result GRASP can be applied to ground-based, satellite and airborne, passive and active measurements and the spectral, angular, polarization, etc. specifications can be changed flexibility within applicability of GRASP "forward model";

  • Implementing "forward modeling" simulation of the measurements using accurate approach with minimum dependence of the algorithm on the a priori assumptions (atmospheric radiation is calculated on-line without using look-up-tables);

  • Applicable to the combined data from the same instrument: i.e., the data can be obtained both in the exactly same location at different observation times or/and at different (e.g. neighboring) locations at the same or different time moments;

  • Applicable to multi-instrument retrievals, i.e. both single observations and extended data sets of observations by different instruments can be processed simultaneously (in highly synergetic way). Since the remote sensing observations (especially from space) are often composing images, Dubovik et al. (2011) proposed improving retrieval using multi-pixel principle when a group of pixels is inverted simultaneously under additional inter-pixel constraints. This principle was realized using rigorous approach of inversion optimization (see more explanations below and related articles) and may significantly improve the retrieval results. However, applying multi-pixel retrieval requires specific elaborated data preparation. Therefore, the additional "service" pieces of software were developed as part of GRASP package that significantly simplified practical application of the GRASP multi-pixel approach to real observations. Thus, GRASP realizes processing of global (or regional) time series of instrument observations as illustrated in Figure 1.2, “Structure of the GRASP software package”. The entire image is divided by the user into geo-located grid composed by rather large data sets that are called "tiles", each tile is composed by "segments" groups of adjacent segments of the observation "pixels". Correspondingly, GRASP can be set to process: (i) only one pixel, (ii) only one segment, (iii) tiles, i.e group of segments.

Figure 1.2. Structure of the GRASP software package

Structure of the GRASP software package


1.2.3. Adaptation of GRASP for general user

In addition to elaborated theoretical concepts (numerical inversion approach, modeling of atmospheric radiation and image processing approach) GRASP was designed with the idea of making convenient "research tool" that can be used by a user possessing general knowledge in remote sensing retrieval but not familiar with the details of source code routines. Therefore, the following user-oriented principles were realized in GRASP:

  • Possibility for user to construct custom retrieval using GRASP, i.e. user set up the inversion of the same observations in many different ways by choosing:

    • different sets (in terms of number and type) of the retrieved and a priori known (and/or even fixed) parameters;
    • different approaches in making forward simulations;
    • different assumption for noise distribution in the inverted data;
    • different sets of a priori constraints for retrieved parameters;
    • different standard procedures used in numerical inversion (i.e. for solving linear systems);
    • different approaches for data processing: inverting each pixel independently, inverting large images (see Fig.2), combining available independent co-incident and/or co-located observations;
    • etc.

  • User independence of specific program realization, because the management of GRASP inputs is done using command lines, not symbols, that represent the "names" of parameters or procedures and can be understood by a user that has general knowledge in the atmospheric retrieval.

1.3. Concept of GRASP software package

The key component of the software package is the scientific GRASP core. This is a code that implements actual inversion of remote sensing observations following the retrieval procedure assumed by scientific algorithm. Initially, the scientific GRASP core was used directly for processing observations by reading the input data from files and providing retrieval output on the screen or/and in the file. However, in order to achieve the highly optimized processing of large volumes of data, such as satellite observations, the scientific GRASP core has been complemented by the development of the control unit – software package that manages the preparations of observations, implementation of actual retrievals by scientific code and the output of results. The utilization of the control unit allows an implementation of the retrieval without generating intermediate input/output files and a number of other optimizations of applying scientific core . In other words the communication of user or processing routing with the scientific core passes via the control unit as illustrated in Figure 1.3, “Structure of the GRASP software package”.

Figure 1.3. Structure of the GRASP software package

Structure of the GRASP software package


Such set up of the scientific package was designed for simplifying the processing of large satellite images by GRASP. Thus, while original GRASP scientific core could provide the retrieval for only one pixel or segment (see Figure 1.2, “Structure of the GRASP software package”), using the control unit manages application of retrieval to the groups of segments (a tile). This approach provides a number of conveniences in employing GRASP for processing the actual observations. Indeed, now the GRASP can read directly a raw data archive and perform data preparation on the fly and without generating intermediate files. The code can manage large volumes of data: the control unit organizes input data, implements multiple calls of the scientific core, obtains the output for all archives, may manage the display of the results, etc.

In addition, the development of the control unit resulted in many convenient features of managing scientific core, not only at the level of the tiles processing of observation data, but also in implementing the retrievals of the segments and even single pixels. For example, the input texts files were replaced by settings files in YAML format that significantly improves the user-interface:

  • all inputs can be provided in a standard format instead of a specific one developed only for GRASP;
  • the settings are driven by the text commands that can be organized in any order (only the content is important);
  • information redundancy was decreased because structure and size of arrays are automatically adapted to the input defined by the user.
  • All settings are auto-documented: every parameter has a description directly in the code that can be viewed calling help command.

It should be noted that the current version of the GRASP software conserves the possibility of running scientific core without using the control unit. While this option may be of some interest for the GRASP developers, the utilization of the entire package (including control unit) is the recommended approach for general GRASP users.



[1] i.e. formally redundant in the sense that the number of inverted observations exceeds the number of retrieved parameters