Autonomous robotic exploration using a utility function based on Rényi’s general theory of entropy

Abstract

In this paper we present a novel information-theoretic utility function for selecting actions in a robot-based autonomous exploration task. The robot’s goal in an autonomous exploration task is to create a complete, high-quality map of an unknown environment as quickly as possible. This implicitly requires the robot to maintain an accurate estimate of its pose as it explores both unknown and previously observed terrain in order to correctly incorporate new information into the map. Our utility function simultaneously considers uncertainty in both the robot pose and the map in a novel way and is computed as the difference between the Shannon and the Rényi entropy of the current distribution over maps. Rényi’s entropy is a family of functions parameterized by a scalar, with Shannon’s entropy being the limit as this scalar approaches unity. We link the value of this scalar parameter to the predicted future uncertainty in the robot’s pose after taking an exploratory action. This effectively decreases the expected information gain of the action, with higher uncertainty in the robot’s pose leading to a smaller expected information gain. Our objective function allows the robot to automatically trade off between exploration and exploitation in a way that does not require manually tuning parameter values, a significant advantage over many competing methods that only use Shannon’s definition of entropy. We use simulated experiments to compare the performance of our proposed utility function to these state-of-the-art utility functions. We show that robots that use our proposed utility function generate maps with less uncertainty and fewer visible artifacts and that the robots have less uncertainty in their pose during exploration. Finally, we demonstrate that a real-world robot using our proposed utility function is able to successfully create a high-quality map of an indoor office environment.

Appendices

Appendix

1.1 A.1 Properties of Rényi’s Entropy

Entropy is a measure of the uncertainty of a random variable (Shannon and Weaver 1949; Rényi 1960; Jumarie 1990). A proper definition of entropy should comply with a set of axioms that guarantee a coherent way of accounting for uncertainty. A widely accepted set of axioms was developed by Aczél and Daróczy (1975). Feinstein (1958) also developed an earlier and more succinct set.

The first attempt to mathematically define entropy was from Hartley (1928). The second definition was developed by Shannon and Weaver (1949), and is the most widely-known and commonly-used definition. Finally Rényi (1960, 1970) created a family of entropy functions, of which the entropies of Hartley and Shannon are special cases. This family of functions, parameterized by $\alpha$, is defined as:

$$\begin{aligned} {{\mathrm{\mathbb {H}_{\alpha }}}}[P(\mathbf {x})] = \frac{1}{1-\alpha }~{{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}^{\alpha } \right) \end{aligned}$$

(A.1)

where $p_{i} = P(\mathbf {x}= \mathbf {x}_i)$ is an element of the probability distribution of a discrete random variable $\mathbf {x}$, so that $p_{i} \ge 0, \quad \forall i$ and $\sum _{i=1}^{n} p_{i} = 1$. The variable $\alpha$ is a free parameter in the range $[0, 1) \cup (1, \infty )$.

1.2 A.2 ${{\mathrm{\mathbb {H}_{\alpha }}}}~\text {at}~\alpha = 0$

Plugging $\alpha = 0$ in to (A.1) yields $\mathbb {H}_0[P(\mathbf {x})] = {{\mathrm{\log _{2}}}}n$, which is the Hartley entropy.

1.3 A.3 ${{\mathrm{\mathbb {H}_{\alpha }}}}~\text {as}~\alpha \rightarrow 1$

Note from the definition of Rényi’s entropy in (A.1) that it is undefined at $\alpha = 1$. Thus to define $\mathbb {H}_{1}[P(\mathbf {x})]$ we must look at the limit as $\alpha \rightarrow 1$:

$$\begin{aligned} \mathbb {H}_{1}[P(\mathbf {x})] = \lim _{\alpha \rightarrow \ 1} \mathbb {H}_{\alpha }[P(\mathbf {x})]. \end{aligned}$$

(A.2)

Applying the limit directly, we obtain a canonical indeterminate form $\frac{0}{0}$. Applying l’Hôpital’s rule we see that:

$$\begin{aligned} \mathbb {H}_{1}[P(\mathbf {x})]&= \lim _{\alpha \rightarrow \ 1} \frac{\left( \sum _{i=1}^{n} p_{i}^{\alpha } \right) ^{-1} \left( \sum _{i=1}^{n} p_{i}^{\alpha } {{\mathrm{\log _{2}}}}(p_{i}) \right) }{-1}\nonumber \\&= \quad -\,\sum _{i=1}^{n} p_{i} {{\mathrm{\log _{2}}}}(p_{i}) \nonumber \\&= \mathbb {H}[P(\mathbf {x})]. \end{aligned}$$

(A.3)

In other words, in the limit as $\alpha \rightarrow 1$ Rényi’s entropy becomes equal to Shannon’s entropy.

1.4 A.4 ${{\mathrm{\mathbb {H}_{\alpha }}}}~\text {as}~\alpha \rightarrow \infty$

Attempting to compute the limit at infinity of ${{\mathrm{\mathbb {H}_{\alpha }}}}[P(\mathbf {x})]$ directly yields infinity as a result. However, we can obtain the true value using the squeeze theorem. We start by defining $p_{i'} = \max (p_{i})$ and recall that $i \in \lbrace 1 \ldots n \rbrace$, $0 \le p_{i} \le 1$, and $\sum _{i=1}^{n} p_{i} = 1$. Hence, for $1< \alpha < \infty$, the following inequality stands:

$$\begin{aligned} p_{i'}^{\alpha } \le \sum _{i=1}^{n} p_{i}^{\alpha } \le n~p_{i'}^{\alpha } \end{aligned}$$

(A.4)

If we take the binary logarithm of (A.4), divide by $1-\alpha$, and rearrange terms, we obtain a more familiar inequality:

$$\begin{aligned} {{\mathrm{\log _{2}}}}(p_{i'}^{\alpha })&\le {{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}^{\alpha } \right) \le {{\mathrm{\log _{2}}}}(n~p_{i'}^{\alpha }) \nonumber \\ \alpha {{\mathrm{\log _{2}}}}(p_{i'})&\le {{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}^{\alpha } \right) \le {{\mathrm{\log _{2}}}}(n~p_{i'}^{\alpha }) \nonumber \\ \frac{\alpha }{1-\alpha }~{{\mathrm{\log _{2}}}}(p_{i'})&\!\ge \! {{\mathrm{\mathbb {H}_{\alpha }}}}[P(\mathbf {x})] \ge ~\frac{{{\mathrm{\log _{2}}}}(n)}{1-\alpha } \!+\! \frac{\alpha }{1-\alpha }~{{\mathrm{\log _{2}}}}(p_{i'}) \end{aligned}$$

(A.5)

Computing the limit as $\alpha \rightarrow \infty$ with l’Hôpital’s rule, we see that both sides yield the same value of $-{{\mathrm{\log _{2}}}}(p_{i'})$. Hence, according to the squeeze theorem, we can compute the desired limit:

$$\begin{aligned} \mathbb {H}_\infty [P(\mathbf {x})] = \lim _{\alpha \rightarrow \infty } {{\mathrm{\mathbb {H}_{\alpha }}}}[P(\mathbf {x})] = -{{\mathrm{\log _{2}}}}(\max _{i} p_{i}) \end{aligned}$$

(A.6)

1.5 A.5 Monotonicity with respect to $\alpha$

We seek to show that Rényi’s entropy monotonically decreases with increasing $\alpha$. To do this, we take the derivative with respect to $\alpha$ and show that it is non-positive. Let $q_{i} = p_{i}^{\alpha } / \sum _{j} p_{j}^{\alpha }$, and note that this defines a probability distribution $Q(\mathbf {x})$.

Taking the derivative of (A.1) with respect to $\alpha$ yields:

$$\begin{aligned} \frac{d}{d\alpha } {{\mathrm{\mathbb {H}_{\alpha }}}}[P(\mathbf {x})]&= \frac{d}{d\alpha } \frac{1}{1-\alpha }~{{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}^{\alpha } \right) \nonumber \\&= \frac{(1-\alpha ) (\sum _{j} p_{j}^{\alpha })^{-1} (\sum _{i} p_{i}^{\alpha } \log p_{i}) + \log (\sum _{j} p_{j}^{\alpha })}{(1-\alpha )^2} \nonumber \\&= \frac{(1-\alpha ) (\sum _{i} q_{i} \log p_{i}) + \sum _{i} (q_i \log (\sum _{j} p_{j}^{\alpha }))}{(1-\alpha )^2} \nonumber \\&= \frac{\sum _{i} q_{i} \log p_{i} - q_{i} \log p_{i}^{\alpha } + q_i \log (\sum _{j} p_{j}^{\alpha })}{(1-\alpha )^2} \nonumber \\&= \frac{\sum _{i} q_{i} \log p_{i} - q_{i} \log q_{i}}{(1-\alpha )^2} \nonumber \\&= \quad -\, \frac{\sum _{i} q_{i} \log \frac{q_{i}}{p_{i}}}{(1-\alpha )^2} \nonumber \\&= \quad -\, \frac{\text {KL}[Q(\mathbf {x}) \, || \, P(\mathbf {x})]}{(1-\alpha )^2}. \end{aligned}$$

(A.7)

Since both $(1-\alpha )^2$ and the Kullback–Leibler divergence $\text {KL}[Q(\mathbf {x}) \, || \, P(\mathbf {x})]$ are non-negative (Cover and Thomas 2012) the derivative is non-positive. Thus we conclude that Rényi’s entropy monotonically decreases in $\alpha$ for $\alpha \in (1, \infty )$.

1.6 A.6 Useful inequalities of ${{\mathrm{\mathbb {H}_{\alpha }}}}$

Let us consider two values for the free parameter of the Rényi entropy, $\alpha$ and $\alpha '$, such that $1 \le \alpha \le \alpha '$. For these two values of the free parameter, we can show that:

• 1. $\mathbb {H}[P(\mathbf {x})] \ge \mathbb {H}_{\alpha }[P(\mathbf {x})], \; \forall \alpha \ge 1$

• 2. $\mathbb {H}[P(\mathbf {x})] \ge \mathbb {H}_{\alpha }[P(\mathbf {x})] \ge \mathbb {H}_{\alpha '}[P(\mathbf {x})], \; 1 \le \alpha \le \alpha '$.

1.6.1 A.6.1 $\mathbb {H}[P(\mathbf {x})] \ge \mathbb {H}_{\alpha }[P(\mathbf {x})]$

The function $-{{\mathrm{\log _{2}}}}(z)$ is convex and a non-negative weighted sum operation does not affect the convexity of a function Boyd and Vandenberghe (2004, Ch. 3), hence using the $p_{i}$ as weights, the function $-\sum _{i=1}^{n} p_{i}{{\mathrm{\log _{2}}}}(z_{i})$ is still convex. Applying Jensen’s inequality we see that:

$$\begin{aligned} -{{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i} z_{i} \right) \le -\sum _{i=1}^{n} p_{i}{{\mathrm{\log _{2}}}}(z_{i}) \end{aligned}$$

(A.8)

where it is understood that $\sum _{i=1}^{n} p_{i} = 1$. Letting $z_{i} = p_{i}^{\alpha - 1}$, (A.8) becomes:

$$\begin{aligned} -{{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}^{\alpha }\right)\le & {} -(\alpha - 1)\sum _{i=1}^{n} p_{i}{{\mathrm{\log _{2}}}}(p_{i}) \end{aligned}$$

(A.9)

$$\begin{aligned} \frac{1}{1-\alpha }{{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}^{\alpha }\right)\le & {} -\sum _{i=1}^{n} p_{i}{{\mathrm{\log _{2}}}}(p_{i}) \end{aligned}$$

(A.10)

$$\begin{aligned} \mathbb {H}_{\alpha }[P(\mathbf {x})]\le & {} \mathbb {H}[P(\mathbf {x})]. \end{aligned}$$

(A.11)

1.6.2 A.6.2 $\mathbb {H}[P(\mathbf {x})] \ge \mathbb {H}_{\alpha }[P(\mathbf {x})] \ge \mathbb {H}_{\alpha '}[P(\mathbf {x})]$

It is known Hardy et al. (1952, Theorem 16) that

$$\begin{aligned} \left( \sum _{i=1}^{n} w_{i}x_{i}^{\beta } \right) ^{\frac{1}{\beta }} \end{aligned}$$

(A.12)

is a monotone increasing function of $\beta$. Rewriting (A.1) as

$$\begin{aligned} {{\mathrm{\mathbb {H}_{\alpha }}}}[P(\mathbf {x})] = {{\mathrm{\log _{2}}}}\left( \sum _{i=1}^{n} p_{i}\left( \frac{1}{p_{i}}\right) ^{1-\alpha } \right) ^{\frac{1}{1-\alpha }} \end{aligned}$$

(A.13)

and letting $\beta = 1 - \alpha$, we may apply (A.12) to conclude that (A.1) is a monotone decreasing function of $\alpha$ and the inequality holds.

B Probability simplex

Let $P(\mathbf {x})$ be a probability distribution of a discrete random variable $\mathbf {x}$, where $p_{i} = P(\mathbf {x}= \mathbf {x}_i)$ is an element of the distribution, $p_{i} \ge 0, \forall i$, and $\sum _{i=1}^{N} p_{i} = 1$. The probability simplex $\Delta _N$ is a $N-1$ dimensional manifold in a N-dimensional space where all the possible probabilities distributions of a N multidimensional random variable $\mathbf {x}$ lives (Principe 2010). That is:

$$\begin{aligned} \Delta _N \!=\! \left\{ p \!= \!(p_1, \ldots , p_N) \in \mathcal {R}^{N} , p_{i} \!\ge \! 0,\sum _{i=1}^{N} p_{i} = 1, \quad \forall i \right\} \end{aligned}$$

(A.14)

Where any point p in the probability simplex has a natural interpretation as a discrete probability distribution (Calafiore and Ghaoui 2014).

As an example consider the probability simplex in $\mathcal {R}^{3}$ for three variables $p_1, p_2, p_3$ where all possible distributions lives inside the equilateral triangle with vertices at (1, 0, 0), (0, 1, 0) and (0, 0, 1). Figure 13 shows an illustration of the above.

Fig. 13

The probability simplex in $\mathcal {R}^{3}$ is shown as the blue triangle with vertices at (1, 0, 0), (0, 1, 0) and (0, 0, 1). Any point on the simplex represents a probability distribution over three variables $p_1, p_2, p_3$ (Color figure online)