Integrated design-sense-plan architecture for autonomous geometric-semantic mapping with UAVs

Rui Pimentel De Figueiredo; Jonas Le Fevre Sejersen; Jakob Grimm Hansen; Martim Brandão

doi:https://doi.org/10.3389/frobt.2022.911974

Integrated design-sense-plan architecture for autonomous geometric-semantic mapping with UAVs

Rui Pimentel De Figueiredo, Jonas Le Fevre Sejersen, Jakob Grimm Hansen, Martim Brandão

Aarhus University

Research output: Contribution to journal › Article › peer-review

49 Downloads (Pure)

Abstract

This article presents a complete solution for autonomous mapping and inspection tasks, namely a lightweight multi-camera drone design coupled with computationally efficient planning algorithms and environment representations for enhanced autonomous navigation in exploration and mapping tasks. The proposed system utilizes state-of-the-art Next-Best-View (NBV) planning techniques, with geometric and semantic segmentation information computed with Deep Convolutional Neural Networks (DCNNs) to improve the environment map representation. The main contributions of this article are the following. First, we propose a novel efficient sensor observation model and a utility function that encodes the expected information gains from observations taken from specific viewpoints. Second, we propose a reward function that incorporates both geometric and semantic probabilistic information provided by a DCNN for semantic segmentation that operates in close to real-time. The incorporation of semantics in the environment representation enables biasing exploration towards specific object categories while disregarding task-irrelevant ones during path planning. Experiments in both a virtual and a real scenario demonstrate the benefits on reconstruction accuracy of using semantics for biasing exploration towards task-relevant objects, when compared with purely geometric state-of-the-art methods. Finally, we present a unified approach for the selection of the number of cameras on a UAV, to optimize the balance between power consumption, flight-time duration, and exploration and mapping performance trade-offs. Unlike previous design optimization approaches, our method is couples with the sense and plan algorithms. The proposed system and general formulations can be be applied in the mapping, exploration, and inspection of any type of environment, as long as environment dependent semantic training data are available, with demonstrated successful applicability in the inspection of dry dock shipyard environments.

Original language	English
Article number	911974
Journal	Frontiers in Robotics and AI
Volume	9
DOIs	https://doi.org/10.3389/frobt.2022.911974
Publication status	Published - 8 Sept 2022

Access to Document

https://doi.org/10.3389/frobt.2022.911974

Integrated design-sense-plan_2022_GOLD VoR (CC BY)Final published version, 2.74 MBLicence: CC BY

Cite this

@article{ba251ca5d8704ecbb39cb303f63473ab,

title = "Integrated design-sense-plan architecture for autonomous geometric-semantic mapping with UAVs",

abstract = "This article presents a complete solution for autonomous mapping and inspection tasks, namely a lightweight multi-camera drone design coupled with computationally efficient planning algorithms and environment representations for enhanced autonomous navigation in exploration and mapping tasks. The proposed system utilizes state-of-the-art Next-Best-View (NBV) planning techniques, with geometric and semantic segmentation information computed with Deep Convolutional Neural Networks (DCNNs) to improve the environment map representation. The main contributions of this article are the following. First, we propose a novel efficient sensor observation model and a utility function that encodes the expected information gains from observations taken from specific viewpoints. Second, we propose a reward function that incorporates both geometric and semantic probabilistic information provided by a DCNN for semantic segmentation that operates in close to real-time. The incorporation of semantics in the environment representation enables biasing exploration towards specific object categories while disregarding task-irrelevant ones during path planning. Experiments in both a virtual and a real scenario demonstrate the benefits on reconstruction accuracy of using semantics for biasing exploration towards task-relevant objects, when compared with purely geometric state-of-the-art methods. Finally, we present a unified approach for the selection of the number of cameras on a UAV, to optimize the balance between power consumption, flight-time duration, and exploration and mapping performance trade-offs. Unlike previous design optimization approaches, our method is couples with the sense and plan algorithms. The proposed system and general formulations can be be applied in the mapping, exploration, and inspection of any type of environment, as long as environment dependent semantic training data are available, with demonstrated successful applicability in the inspection of dry dock shipyard environments.",

author = "{De Figueiredo}, {Rui Pimentel} and {Le Fevre Sejersen}, Jonas and Hansen, {Jakob Grimm} and Martim Brand{\~a}o",

note = "Funding Information: This work was supported by UKRI/EPSRC THuMP (EP/R033722/1) and the Smart Industry Program (European Regional Development Fund and Region Midtjylland, grant no.: RFM-17-0020). Publisher Copyright: Copyright {\textcopyright} 2022 Pimentel de Figueiredo, Le Fevre Sejersen, Grimm Hansen and Brand{\~a}o.",

year = "2022",

month = sep,

day = "8",

doi = "https://doi.org/10.3389/frobt.2022.911974",

language = "English",

volume = "9",

journal = "Front. Robot. AI",

issn = "2296-9144",

publisher = "Frontiers Media S.A.",

}

TY - JOUR

T1 - Integrated design-sense-plan architecture for autonomous geometric-semantic mapping with UAVs

AU - De Figueiredo, Rui Pimentel

AU - Le Fevre Sejersen, Jonas

AU - Hansen, Jakob Grimm

AU - Brandão, Martim

N1 - Funding Information: This work was supported by UKRI/EPSRC THuMP (EP/R033722/1) and the Smart Industry Program (European Regional Development Fund and Region Midtjylland, grant no.: RFM-17-0020). Publisher Copyright: Copyright © 2022 Pimentel de Figueiredo, Le Fevre Sejersen, Grimm Hansen and Brandão.

PY - 2022/9/8

Y1 - 2022/9/8

N2 - This article presents a complete solution for autonomous mapping and inspection tasks, namely a lightweight multi-camera drone design coupled with computationally efficient planning algorithms and environment representations for enhanced autonomous navigation in exploration and mapping tasks. The proposed system utilizes state-of-the-art Next-Best-View (NBV) planning techniques, with geometric and semantic segmentation information computed with Deep Convolutional Neural Networks (DCNNs) to improve the environment map representation. The main contributions of this article are the following. First, we propose a novel efficient sensor observation model and a utility function that encodes the expected information gains from observations taken from specific viewpoints. Second, we propose a reward function that incorporates both geometric and semantic probabilistic information provided by a DCNN for semantic segmentation that operates in close to real-time. The incorporation of semantics in the environment representation enables biasing exploration towards specific object categories while disregarding task-irrelevant ones during path planning. Experiments in both a virtual and a real scenario demonstrate the benefits on reconstruction accuracy of using semantics for biasing exploration towards task-relevant objects, when compared with purely geometric state-of-the-art methods. Finally, we present a unified approach for the selection of the number of cameras on a UAV, to optimize the balance between power consumption, flight-time duration, and exploration and mapping performance trade-offs. Unlike previous design optimization approaches, our method is couples with the sense and plan algorithms. The proposed system and general formulations can be be applied in the mapping, exploration, and inspection of any type of environment, as long as environment dependent semantic training data are available, with demonstrated successful applicability in the inspection of dry dock shipyard environments.

AB - This article presents a complete solution for autonomous mapping and inspection tasks, namely a lightweight multi-camera drone design coupled with computationally efficient planning algorithms and environment representations for enhanced autonomous navigation in exploration and mapping tasks. The proposed system utilizes state-of-the-art Next-Best-View (NBV) planning techniques, with geometric and semantic segmentation information computed with Deep Convolutional Neural Networks (DCNNs) to improve the environment map representation. The main contributions of this article are the following. First, we propose a novel efficient sensor observation model and a utility function that encodes the expected information gains from observations taken from specific viewpoints. Second, we propose a reward function that incorporates both geometric and semantic probabilistic information provided by a DCNN for semantic segmentation that operates in close to real-time. The incorporation of semantics in the environment representation enables biasing exploration towards specific object categories while disregarding task-irrelevant ones during path planning. Experiments in both a virtual and a real scenario demonstrate the benefits on reconstruction accuracy of using semantics for biasing exploration towards task-relevant objects, when compared with purely geometric state-of-the-art methods. Finally, we present a unified approach for the selection of the number of cameras on a UAV, to optimize the balance between power consumption, flight-time duration, and exploration and mapping performance trade-offs. Unlike previous design optimization approaches, our method is couples with the sense and plan algorithms. The proposed system and general formulations can be be applied in the mapping, exploration, and inspection of any type of environment, as long as environment dependent semantic training data are available, with demonstrated successful applicability in the inspection of dry dock shipyard environments.

UR - http://www.scopus.com/inward/record.url?scp=85138532513&partnerID=8YFLogxK

U2 - https://doi.org/10.3389/frobt.2022.911974

DO - https://doi.org/10.3389/frobt.2022.911974

M3 - Article

SN - 2296-9144

VL - 9

JO - Front. Robot. AI

JF - Front. Robot. AI

M1 - 911974

ER -

Integrated design-sense-plan architecture for autonomous geometric-semantic mapping with UAVs

Abstract

Access to Document

Other files and links

Fingerprint

Cite this