The maximal analytic continuation domain across a plane boundary for solutions of general elliptic boundary value problems (Hörmander's $\Omega^*$ problem)

Hörmander (Acta Mathematica, 1958) proved that all solutions of an elliptic boundary value problem with constant coefficients on a domain with a plane boundary piece $\omega$ admit analytic continuations across $\omega$ into one fixed domain $\Omega^*$ that does not depend on the solution, and posed the problem of determining the size of the largest such domain, noting that in the classical case (Laplace equation, vanishing Dirichlet data) $\Omega^*$ is obtained by geometric reflection and that F. John had answered a special case. The intervening literature resolved substantial fragments — John's 1957 characterization of the constant-coefficient equations that possess reflection principles for Dirichlet data; explicit polyharmonic reflection formulas and continuation domains (Poritsky, Duffin, Huber, Nystedt); and modern studies of reflection operators (Farwig, Aberra–Savina, Savina, Kounchev–Render, Matevossian) — but the general question remains open: for a general elliptic operator with general elliptic boundary conditions (in Hörmander's sense of condition (3.2)), no characterization, computation, or sharp estimate of the maximal common analytic continuation domain is known. The problem asks for such a characterization: a determination of the size of the largest domain $\Omega^*$ to which all solutions have analytic continuations.

Background

Setting. Let $\Omega \subset \mathbb{R}^n$ ($n \geq 2$) be a bounded connected open set whose boundary contains a relatively open piece $\omega$ of a hyperplane (take $\omega \subset \{x_1 = 0\}$, with $\Omega$ on the side $x_1 > 0$ near $\omega$). Let $P(D)$ be a scalar elliptic differential operator with constant coefficients of order $2m$ (elliptic: the principal symbol — the homogeneous polynomial in the frequency variables $\xi$ formed from the highest-order derivatives — satisfies $P_{2m}(\xi) \neq 0$ for real $\xi \neq 0$), and let $Q_1, \dots, Q_m$ be differential operators with constant coefficients of orders $< 2m$. Hörmander calls the boundary conditions $Q_\nu u = 0$ on $\omega$ elliptic with respect to $P$ if the determinant $C(\xi)$ of the associated symbol matrix (the Lopatinskii determinant of the problem) satisfies his algebraic condition (3.2): there is a constant $M$ with $C(\xi) \neq 0$ whenever $|\mathrm{Re}\,\xi| \geq M(1 + |\mathrm{Im}\,\xi|)$; for constant coefficients on a flat boundary this is equivalent to the Lopatinskii–Shapiro (complementing) condition. For the boundary problem $P(D)u = 0$ in $\Omega$, $Q_\nu(D)u = 0$ on $\omega$, Hörmander's Theorem 3.2 proves regularity of solutions, and the proof of his Lemma 6.1 shows that all solutions $u \in C^{2m}(\Omega \cup \omega)$ are real-analytic in $\Omega$ and continue analytically across $\omega$ into a domain $\Omega^* \supseteq \Omega \cup \omega$ depending only on $P$, the $Q_\nu$, and $\Omega$ (the constant in his analytic a priori estimate (6.6) is independent of $u$) — but the proof gives no usable information on how far $\Omega^*$ reaches.

Classical prototype. For $P = \Delta$ with vanishing Dirichlet data, the classical Schwarz reflection principle continues a harmonic function vanishing on $\omega$ to the union of $\Omega$ with its mirror image, so the largest $\Omega^*$ is the geometric double of $\Omega$ — the maximal possible answer.

John's special case. Hörmander credits F. John with 'a special case of this question' — his footnote [7] cites John's 1956 Nancy colloquium note on prolongement and reflection, of which the 1957 Bulletin article referenced here is the expanded journal version. John studied constant-coefficient equations of even order $2m$ with vanishing normal derivatives (Dirichlet-type data) on a plane. He called the boundary data adequate for continuation if every analytic solution on one side continues locally across the plane, and said a reflection principle holds for $P$ if some $\varepsilon > 0$ depending only on $P$ makes every solution analytic in the cylinder $\{x \in D,\ 0 \leq x_0 < h\}$ continue into the strip $\{x \in D,\ -\varepsilon h < x_0 \leq 0\}$. By constructing solutions for which the plane is a natural boundary (a hypersurface across which no analytic continuation is possible), he proved necessary conditions for adequacy; and, via singular solutions built from complex characteristics, he proved that reflection principles exist only when the symbol is of one of three types: (17a) $\eta_0^{2m}$; (17b) $\prod_{k=1}^{2m}(\eta_0 - a_k L(\eta))$ with $L$ a real linear form and real $a_k$ of which $m$ are positive and $m$ negative; (17c) $\prod_{k=1}^{m}(\eta_0^2 - a_k^2\, q(\eta))$ with $q$ a real quadratic form and $a_k > 0$. For these types he gave explicit reflection formulas; for all other operators his singular solutions obstruct any scale-invariant reflection. He conjectured (open in his paper) that his necessary condition for adequacy is also sufficient.

Polyharmonic line. For the polyharmonic equation $\Delta^p u = 0$, Poritsky (1946; the biharmonic case $p = 2$ in two dimensions), Duffin (1955; biharmonic in three dimensions, and spherical boundaries), and Huber (1955; general $p$, Dirichlet data) gave explicit finite-order differential reflection formulas continuing solutions into the geometrically reflected domain; Nystedt (1982) handled general linear homogeneous boundary conditions, proving continuation into an explicit domain $\tilde{\Omega}_0 = \Omega \cup \omega \cup \Omega' \cup \Omega''$ (with $\Omega'$ obtained from $\Omega$ by convexifying along normals to $\omega$ and $\Omega''$ its geometric reflection), and noting explicitly that Hörmander's theorem 'does not tell anything about the extent of the continuation'.

Modern line. Farwig (1994) proved reflection principles for the biharmonic equation and the Stokes system (the linear stationary equations of a viscous incompressible fluid) in a half-space under several distinct boundary conditions and applied them to uniqueness in weighted $L^q$ spaces. Aberra–Savina (2000) gave a Schwarz reflection formula for polyharmonic functions in $\mathbb{R}^n$, including nonhomogeneous data. Savina (2010) classified how the structure of the biharmonic reflection operator depends on the boundary conditions: point-to-point reflections for $u = \partial_n u = 0$, $u = \Delta u = 0$, $\partial_n u = \partial_n \Delta u = 0$, but point-to-a-continuum for $u = \partial_n \Delta u = 0$ and $\partial_n u = \Delta u = 0$. Savina (2012) showed that for second-order elliptic equations with analytic (non-constant) coefficients point-to-point reflection almost always fails and is replaced by a non-local, point-to-compact-set reflection formula. Kounchev–Render (2017) proved symmetry (oddness) properties of polyharmonic functions on strips vanishing on equidistant hyperplanes. Matevossian (2025a, 2025b) studies the (generalized) Farwig problem — uniqueness and solution-space dimension for polyharmonic and biharmonic problems in half-spaces and exterior domains under weighted Dirichlet-integral bounds. None of these addresses the maximal continuation domain for general elliptic boundary problems.

Problem Statement

Let $n \geq 2$. Let $\Omega \subset \mathbb{R}^n$ be a bounded connected open set and let $\omega \subset \partial \Omega$ be a connected relatively open piece of the hyperplane $\{x_1 = 0\}$ such that $\Omega$ lies in the half-space $\{x_1 > 0\}$ near every point of $\omega$. Let $P(D)$ be an elliptic differential operator with constant coefficients of order $2m$, and let $Q_1, \dots, Q_m$ be differential operators with constant coefficients of orders $< 2m$ whose boundary-condition determinant $C(\xi)$ satisfies Hörmander's condition (3.2): there exists $M$ such that $C(\xi) \neq 0$ whenever $|\mathrm{Re}\,\xi| \geq M(1 + |\mathrm{Im}\,\xi|)$. Consider the solution class

$$\mathcal{S} = \{u \in C^{2m}(\Omega \cup \omega) : P(D)u = 0 \text{ in } \Omega,\ Q_\nu(D)u = 0 \text{ on } \omega,\ \nu = 1, \dots, m\}.$$

Every $u \in \mathcal{S}$ is real-analytic in $\Omega$. For $u \in \mathcal{S}$ let $D(u) \supseteq \Omega \cup \omega$ be the largest connected open set to which $u$ extends as a single-valued real-analytic function, and define the maximal common continuation domain

$$\Omega^*_{\max}(P, Q, \Omega) = \mathrm{int}\, \bigcap_{u \in \mathcal{S}} D(u).$$

Hörmander (1958) guarantees that $\Omega^*_{\max}$ extends strictly across $\omega$ (there is a solution-independent continuation domain), but nothing sharper is known in general. Determine $\Omega^*_{\max}$: give an effective description of $\Omega^*_{\max}(P, Q, \Omega)$ in terms of the algebraic data (the symbols of $P$ and the $Q_\nu$, e.g. the configuration of the characteristic roots of $P(\lambda, \eta)$ as the tangential frequency vector $\eta$ ranges over real vectors) and the geometry of $\Omega$. Here $P(\lambda, \eta)$ is the symbol of $P$ with $\lambda$ the frequency variable dual to the normal direction $x_1$ and $\eta$ the frequency variables dual to the coordinates along $\omega$, and the characteristic roots are the roots $\lambda$ of $P(\lambda, \eta) = 0$. Since $\Omega^*_{\max}$ is defined as the intersection over all of $\mathcal{S}$, the requested description must be asserted for all of $\mathcal{S}$; a result valid only for a subclass with growth or regularity restrictions does not determine $\Omega^*_{\max}$.

A determination of $\Omega^*_{\max}$ is consistent with the classical special cases: the Laplace–Dirichlet case gives the geometric double of $\Omega$; the polyharmonic Dirichlet case recovers the Poritsky–Duffin–Huber reflection formulas; operators of John's types (17a)–(17c) with Dirichlet data admit the uniform-strip reflection principles he constructed; and the determination does not contradict John's singular counterexamples or his natural-boundary constructions. Hörmander's original framework also covers determined elliptic systems; extending an answer to that setting is a legitimate further generalization, but the scalar case is already of full interest.

References

1. L. Hörmander, 'On the regularity of the solutions of boundary problems', Acta Mathematica 99 (1958), 225–264. DOI: 10.1007/BF02392427, https://doi.org/10.1007/BF02392427. [Primary source of the open question.]
2. F. John, 'Continuation and reflection of solutions of partial differential equations', Bulletin of the American Mathematical Society 63 (1957), no. 6, 327–344. DOI: 10.1090/S0002-9904-1957-10138-4, https://doi.org/10.1090/S0002-9904-1957-10138-4.
3. F. John, 'Prolongement et réflexion des solutions des équations aux dérivées partielles', Colloques Internationaux du Centre National de la Recherche Scientifique LXXI (Nancy, April 1956), Paris, 103–109. zbMATH: 0075.27601, https://api.zbmath.org/v1/document/3125468. [Hörmander's reference [7].]
4. L. Nystedt, 'On polyharmonic continuation by reflection formulas', Arkiv för Matematik 20 (1982), 201–247. DOI: 10.1007/BF02390510, https://doi.org/10.1007/BF02390510.
5. H. Poritsky, 'Application of analytic functions to two-dimensional biharmonic analysis', Transactions of the American Mathematical Society 59 (1946), 248–279. DOI: 10.1090/S0002-9947-1946-0015630-0, https://doi.org/10.1090/S0002-9947-1946-0015630-0.
6. R. J. Duffin, 'Continuation of biharmonic functions by reflection', Duke Mathematical Journal 22 (1955), 313–324. DOI: 10.1215/s0012-7094-55-02233-x, https://doi.org/10.1215/s0012-7094-55-02233-x.
7. A. Huber, 'The reflection principle for polyharmonic functions', Pacific Journal of Mathematics 5 (1955), 433–439. DOI: 10.2140/pjm.1955.5.433, https://doi.org/10.2140/pjm.1955.5.433.
8. R. Farwig, 'A note on the reflection principle for the biharmonic equation and the Stokes system', Acta Applicandae Mathematicae 37 (1994), 41–51. DOI: 10.1007/BF00995128, https://doi.org/10.1007/BF00995128.
9. D. Aberra and T. Savina, 'The Schwarz reflection principle for polyharmonic functions in $\mathbb{R}^n$', Complex Variables: Theory and Application 41 (2000), no. 1, 27–44. DOI: 10.1080/17476930008815235, https://doi.org/10.1080/17476930008815235.
10. T. Savina, 'On the dependence of the reflection operator on boundary conditions for biharmonic functions', Journal of Mathematical Analysis and Applications 370 (2010), no. 2, 716–725. DOI: 10.1016/j.jmaa.2010.04.036, https://doi.org/10.1016/j.jmaa.2010.04.036. Preprint arXiv:1008.1565.
11. T. Savina, 'On non-local reflection for elliptic equations of the second order in $\mathbb{R}^2$ (the Dirichlet condition)', Transactions of the American Mathematical Society 364 (2012), no. 5, 2443–2460. DOI: 10.1090/s0002-9947-2012-05462-6, https://doi.org/10.1090/s0002-9947-2012-05462-6.
12. O. Kounchev and H. Render, 'A symmetry property for polyharmonic functions vanishing on equidistant hyperplanes', Mathematische Nachrichten 290 (2017), no. 7, 1087–1096 (published online 12 December 2016). DOI: 10.1002/mana.201500378, https://doi.org/10.1002/mana.201500378. Preprint arXiv:1510.02299.
13. H. A. Matevossian, 'On the generalized Farwig problem for a polyharmonic equation', Complex Variables and Elliptic Equations 70 (2025), no. 7, 1153–1160 (published online 3 July 2024). DOI: 10.1080/17476933.2024.2373292, https://doi.org/10.1080/17476933.2024.2373292.
14. H. A. Matevossian, 'Farwig problem for the biharmonic equation in a half-space', Lobachevskii Journal of Mathematics 46 (2025), no. 6, 2909–2916. DOI: 10.1134/S199508022560832X, https://doi.org/10.1134/S199508022560832X.