Recent advances in very high resolution (VHR) earth observation (EO) techniques have led to a massive increase in data volumes to be processed. These remote sensing (RS) processing steps are complex and heterogeneous and an optimized use of these algorithms in both High Processing Computer (HPC) and Cloud platforms are still an important and opened study field for RS data providers with actual and future EO missions. The goal of our study is to identify and develop a new architecture to deal with large and increasing quantity of data and versatile production profiles. In this study, we took the example of the actual and complete processing pipeline used in Pleiades production to deliver perfect sensor image. This pipeline is composed of heterogeneous radiometric and geometric processing steps. In the first part, we study five main big data framework solutions. As result of this study, we identify Apache Spark as the best framework to use due to its performance, great development maturity, and data resilience certification. In the second part, and to develop the new processing pipeline, we redesign the processing pipeline with separation of the metadata management and the core processing. These good practices help us to develop and reuse legacy algorithms to an operational processing pipeline compatible with big data paradigm. As result of this development, we successfully identify a generic way to develop new processes and reuse legacy algorithms with large data paradigm and by keeping great performance and, more importantly, gaining platform flexibility.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.