Operator norm

In mathematics, the operator norm measures the "size" of certain linear operators by assigning each a real number called its operator norm. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces. Informally, the operator norm $\|T\|$ of a linear map $T:X\to Y$ is the maximum factor by which it "lengthens" vectors.

Introduction and definition

Given two normed vector spaces $V$ and $W$ (over the same base field, either the real numbers $\mathbb {R}$ or the complex numbers $\mathbb {C}$), a linear map $A:V\to W$ is continuous if and only if there exists a real number $c$ such that $$\|Av\|\leq c\|v\|\quad {\text{ for all }}v\in V.$$

The norm on the left is the one in $W$ and the norm on the right is the one in $V$. Intuitively, the continuous operator $A$ never increases the length of any vector by more than a factor of $c.$ Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of $A,$ one can take the infimum of the numbers $c$ such that the above inequality holds for all $v\in V.$ This number represents the maximum scalar factor by which $A$ "lengthens" vectors. In other words, the "size" of $A$ is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of $A$ as $$\|A\|_{\text{op}}=\inf\{c\geq 0:\|Av\|\leq c\|v\|{\text{ for all }}v\in V\}.$$

The infimum is attained as the set of all such $c$ is closed, nonempty, and bounded from below.

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces $V$ and $W$.

Examples

Every real $m$-by-$n$ matrix corresponds to a linear map from $\mathbb {R} ^{n}$ to $\mathbb {R} ^{m}.$ Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all $m$-by-$n$ matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both $\mathbb {R} ^{n}$ and $\mathbb {R} ^{m},$ then the matrix norm given to a matrix $A$ is the square root of the largest eigenvalue of the matrix $A^{*}A$ (where $A^{*}$ denotes the conjugate transpose of $A$). This is equivalent to assigning the largest singular value of $A.$

Passing to a typical infinite-dimensional example, consider the sequence space $\ell ^{2},$ which is an L^p space, defined by $$\ell ^{2}=\left\{(a_{n})_{n\geq 1}:\;a_{n}\in \mathbb {C} ,\;\sum _{n}|a_{n}|^{2}<\infty \right\}.$$

This can be viewed as an infinite-dimensional analogue of the Euclidean space $\mathbb {C} ^{n}.$ Now consider a bounded sequence $s_{\bullet }=\left(s_{n}\right)_{n=1}^{\infty }.$ The sequence $s_{\bullet }$ is an element of the space $\ell ^{\infty },$ with a norm given by $$\left\|s_{\bullet }\right\|_{\infty }=\sup _{n}\left|s_{n}\right|.$$

Define an operator $T_{s}$ by pointwise multiplication: $$\left(a_{n}\right)_{n=1}^{\infty }\;{\stackrel {T_{s}}{\mapsto }}\;\ \left(s_{n}\cdot a_{n}\right)_{n=1}^{\infty }.$$

The operator $T_{s}$ is bounded with operator norm $$\left\|T_{s}\right\|_{\text{op}}=\left\|s_{\bullet }\right\|_{\infty }.$$

This discussion extends directly to the case where $\ell ^{2}$ is replaced by a general $L^{p}$ space with $p>1$ and $\ell ^{\infty }$ replaced by $L^{\infty }.$

Equivalent definitions

Let $A:V\to W$ be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition $V\neq \{0\}$ then they are all equivalent:

${\begin{alignedat}{4}\|A\|_{\text{op}}&=\inf &&\{c\geq 0~&&:~\|Av\|\leq c\|v\|~&&~{\text{ for all }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|\leq 1~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|<1~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|\in \{0,1\}~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|=1~&&~{\mbox{ and }}~&&v\in V\}\;\;\;{\text{ this equality holds if and only if }}V\neq \{0\}\\&=\sup &&{\bigg \{}{\frac {\|Av\|}{\|v\|}}~&&:~v\neq 0~&&~{\mbox{ and }}~&&v\in V{\bigg \}}\;\;\;{\text{ this equality holds if and only if }}V\neq \{0\}.\\\end{alignedat}}$

If $V=\{0\}$ then the sets in the last two rows will be empty, and consequently their supremums over the set $[-\infty ,\infty ]$ will equal $-\infty$ instead of the correct value of $0.$ If the supremum is taken over the set $[0,\infty ]$ instead, then the supremum of the empty set is $0$ and the formulas hold for any $V.$

Importantly, a linear operator $A:V\to W$ is not, in general, guaranteed to achieve its norm $\|A\|_{\text{op}}=\sup\{\|Av\|:\|v\|\leq 1,v\in V\}$ on the closed unit ball $\{v\in V:\|v\|\leq 1\},$ meaning that there might not exist any vector $u\in V$ of norm $\|u\|\leq 1$ such that $\|A\|_{\text{op}}=\|Au\|$ (if such a vector does exist and if $A\neq 0,$ then $u$ would necessarily have unit norm $\|u\|=1$). R.C. James proved James's theorem in 1964, which states that a Banach space $V$ is reflexive if and only if every bounded linear functional $f\in V^{*}$ achieves its norm on the closed unit ball. It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If $A:V\to W$ is bounded then $$\|A\|_{\text{op}}=\sup \left\{\left|w^{*}(Av)\right|:\|v\|\leq 1,\left\|w^{*}\right\|\leq 1{\text{ where }}v\in V,w^{*}\in W^{*}\right\}$$ and $$\|A\|_{\text{op}}=\left\|{}^{t}A\right\|_{\text{op}}$$ where ${}^{t}A:W^{*}\to V^{*}$ is the transpose of $A:V\to W,$ which is the linear operator defined by $w^{*}\,\mapsto \,w^{*}\circ A.$

Properties

The operator norm is indeed a norm on the space of all bounded operators between $V$ and $W$. This means $$\|A\|_{\text{op}}\geq 0{\mbox{ and }}\|A\|_{\text{op}}=0{\mbox{ if and only if }}A=0,$$ $$\|aA\|_{\text{op}}=|a|\|A\|_{\text{op}}{\mbox{ for every scalar }}a,$$ $$\|A+B\|_{\text{op}}\leq \|A\|_{\text{op}}+\|B\|_{\text{op}}.$$

The following inequality is an immediate consequence of the definition: $$\|Av\|\leq \|A\|_{\text{op}}\|v\|\ {\mbox{ for every }}\ v\in V.$$

The operator norm is also compatible with the composition, or multiplication, of operators: if $V$, $W$ and $X$ are three normed spaces over the same base field, and $A:V\to W$ and $B:W\to X$ are two bounded operators, then it is a sub-multiplicative norm, that is: $$\|BA\|_{\text{op}}\leq \|B\|_{\text{op}}\|A\|_{\text{op}}.$$

For bounded operators on $V$, this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

Table of common operator norms

By choosing different norms for the codomain, used in computing $\|Av\|$, and the domain, used in computing $\|v\|$, we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in $N^{2}$ operations (for an $N\times N$ matrix), with the exception of the $\ell _{2}-\ell _{2}$ norm (which requires $N^{3}$ operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

Computability of Operator Norms

		Co-domain
		$\ell _{1}$	$\ell _{2}$	$\ell _{\infty }$
Domain	$\ell _{1}$	Maximum $\ell _{1}$ norm of a column	Maximum $\ell _{2}$ norm of a column	Maximum $\ell _{\infty }$ norm of a column
	$\ell _{2}$	NP-hard	Maximum singular value	Maximum $\ell _{2}$ norm of a row
	$\ell _{\infty }$	NP-hard	NP-hard	Maximum $\ell _{1}$ norm of a row

The norm of the adjoint or transpose can be computed as follows. We have that for any $p,q,$ then $\|A\|_{p\rightarrow q}=\|A^{*}\|_{q'\rightarrow p'}$ where $p',q'$ are Hölder conjugate to $p,q,$ that is, $1/p+1/p'=1$ and $1/q+1/q'=1.$

Operators on a Hilbert space

Suppose $H$ is a real or complex Hilbert space. If $A:H\to H$ is a bounded linear operator, then we have $$\|A\|_{\text{op}}=\left\|A^{*}\right\|_{\text{op}}$$ and $$\left\|A^{*}A\right\|_{\text{op}}=\|A\|_{\text{op}}^{2},$$ where $A^{*}$ denotes the adjoint operator of $A$ (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix $A$).

In general, the spectral radius of $A$ is bounded above by the operator norm of $A$: $$\rho (A)\leq \|A\|_{\text{op}}.$$

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator $A$ has spectrum $\{0\}.$ So $\rho (A)=0$ while $\|A\|_{\text{op}}>0.$

However, when a matrix $N$ is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that $$\rho (N)=\|N\|_{\text{op}}.$$

This formula can sometimes be used to compute the operator norm of a given bounded operator $A$: define the Hermitian operator $B=A^{*}A,$ determine its spectral radius, and take the square root to obtain the operator norm of $A.$

The space of bounded operators on $H,$ with the topology induced by operator norm, is not separable. For example, consider the Lp space $L^{2}[0,1],$ which is a Hilbert space. For $0<t\leq 1,$ let $\Omega _{t}$ be the characteristic function of $[0,t],$ and $P_{t}$ be the multiplication operator given by $\Omega _{t},$ that is, $$P_{t}(f)=f\cdot \Omega _{t}.$$

Then each $P_{t}$ is a bounded operator with operator norm 1 and $$\left\|P_{t}-P_{s}\right\|_{\text{op}}=1\quad {\mbox{ for all }}\quad t\neq s.$$

But $\{P_{t}:0<t\leq 1\}$ is an uncountable set. This implies the space of bounded operators on $L^{2}([0,1])$ is not separable, in operator norm. One can compare this with the fact that the sequence space $\ell ^{\infty }$ is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.

See also

• Banach–Mazur compactum – Concept in functional analysis
• Continuous linear operator
• Contraction (operator theory) – Bounded operators with sub-unit norm
• Discontinuous linear map
• Dual norm – Measurement on a normed vector space
• Matrix norm – Norm on a vector space of matrices
• Norm (mathematics) – Length in a vector space
• Normed space – Vector space on which a distance is definedPages displaying short descriptions of redirect _targets
• Operator algebra – Branch of functional analysis
• Operator theory – Mathematical field of study
• Topologies on the set of operators on a Hilbert space
• Unbounded operator – Linear operator defined on a dense linear subspace

References

• Aliprantis, Charalambos D.; Border, Kim C. (2007), Infinite Dimensional Analysis: A Hitchhiker's Guide, Springer, p. 229, ISBN 9783540326960.
• Conway, John B. (1990), "III.2 Linear Operators on Normed Spaces", A Course in Functional Analysis, New York: Springer-Verlag, pp. 67–69, ISBN 0-387-97245-5
• Diestel, Joe (1984). Sequences and series in Banach spaces. New York: Springer-Verlag. ISBN 0-387-90859-5. OCLC 9556781.
• Rudin, Walter (1991). Functional Analysis. International Series in Pure and Applied Mathematics. Vol. 8 (Second ed.). New York, NY: McGraw-Hill Science/Engineering/Math. ISBN 978-0-07-054236-5. OCLC 21163277.