The inverse problem of the calculus of variations for systems of second-order ordinary differential equations in arbitrary dimension $n \geq 3$ (completing the Helmholtz-Douglas solution)

The inverse problem of the calculus of variations, posed by Helmholtz (1887) and cast in multiplier form by Hirsch (1898), asks whether the $2n$-parameter family of integral curves of a given system $y_i'' = F_i(x, y_j, y_j')$ of second-order ordinary differential equations coincides with the extremals of some regular variational integral $\int\varphi\,dx$, and, if so, asks for all such Lagrangians. Jesse Douglas's 1939 PNAS note announced a complete solution of this problem and referred to a forthcoming detailed account; that account appeared as Trans. Amer. Math. Soc. 50 (1941) 71-128 and is accepted by the later literature as completely solving only the case $n = 2$, through a four-case classification by the Jacobi endomorphism. What remains open is a complete solution in Douglas's sense — existence decision, construction of all Lagrangians, and exhaustive case classification — for arbitrary dimension, with only specific subcases solved to date (the Jacobi endomorphism a multiple of the identity, and diagonalisable with distinct eigenvalues with all, or all but one, eigen co-distributions integrable). The problem remains open and actively researched.

Background

Suppose one is given a system of $n$ second-order ordinary differential equations (SODEs) $y_i'' = F_i(x, y_j, y_j'), \qquad i = 1, \dots, n,$ whose general solution is a family of curves depending on $2n$ arbitrary constants (a family of $\infty^{2n}$ curves) in the $(n+1)$-dimensional space of $(x, y_1, \dots, y_n)$. The calculus of variations singles out such families as the extremals of a functional $\int \varphi(x, y_j, y_j')\,dx,$ where the Lagrangian $\varphi$ is required to be regular, meaning that its Hessian $\varphi_{ij} = \partial^2\varphi/\partial y_i'\partial y_j'$ with respect to the velocities is invertible; the extremals satisfy the Euler-Lagrange equations, the second-order system obtained by setting the variational derivative of $\varphi$ to zero. The inverse problem of the calculus of variations, posed by Helmholtz in 1887 and given its multiplier form by Hirsch in 1898, runs opposite to this direct computation: decide whether the given family of curves is exactly the totality of extremals of some regular $\varphi$, and, in the affirmative case, find all such $\varphi$ (any two differ only by the addition of an exact differential, which does not affect the extremals). For a single equation ($n = 1$) the problem was settled in the nineteenth century and is textbook material; Douglas's note says so explicitly. For $n \geq 2$ — a containing space of three or more dimensions — the problem becomes genuinely obstructed: not every family of curves is an extremal family. The standard modern formulation is the multiplier version: the system is variational if and only if there exists an invertible multiplier matrix $g_{ab}(t, x^c, x'^c)$ such that $g_{ab}(x''^b - F^b)$ is identically an Euler-Lagrange expression. The necessary and sufficient conditions on $g_{ab}$ — the Helmholtz conditions, in the form fixed by Sarlet (1982) — read $g_{ab} = g_{ba}, \qquad \Gamma(g_{ab}) = g_{ac}\Gamma^c{}_b + g_{bc}\Gamma^c{}_a, \qquad g_{ac}\Phi^c{}_b = g_{bc}\Phi^c{}_a, \qquad \frac{\partial g_{ab}}{\partial x'^c} = \frac{\partial g_{ac}}{\partial x'^b},$ where $\Gamma^a{}_b = -\tfrac{1}{2}\partial F^a/\partial x'^b$ are the coefficients of the nonlinear connection determined by the system (a splitting of directions along solutions into horizontal and vertical parts), $\Gamma = \partial/\partial t + x'^a\,\partial/\partial x^a + F^a\,\partial/\partial x'^a$ is total differentiation along solutions, and $\Phi^a{}_b$, the Jacobi endomorphism, is the $(1,1)$-tensor field on the space of positions and velocities that measures the curvature-type asymmetry of the system. The pointwise algebraic type of $\Phi$ (its eigenvalues and their multiplicity, and whether it is diagonalisable) controls the entire analysis. A Lagrangian is recovered from a multiplier by integrating $\partial^2 L/\partial x'^a \partial x'^b = g_{ab}$. Before Douglas, D. R. Davis (1928) had reduced the problem to the determination of multipliers but could solve the resulting linear differential system only in three special examples. Douglas's PNAS note of December 1939 announced a complete solution: it reduces the problem to a fundamental overdetermined linear differential system $S$ for the Hessian $\varphi_{ij}$ of the sought Lagrangian, announces an existence-theoretic solution of $S$ by the Riquier-Janet-Thomas theory (the classical theory that decides the consistency of a system of partial differential equations and the arbitrary constants and functions entering its general solution, by reduction to involution), exhibits the first concrete non-extremal families (for instance $y'' = y^2 + z^2$, $z'' = y$), gives a tensor form of the solution with covariant differentiation adapted to the path space, and derives as a corollary that every path space whose Weyl tensor (a projective-curvature invariant of differential geometry) vanishes is the geodesic family of a Finsler space — a space whose length element $ds = \psi(x, dx)$ is homogeneous of the first, rather than the second, degree in the displacement. The detailed account announced in that note appeared, with a change of venue, as Douglas's memoir in Trans. Amer. Math. Soc. 50 (1941) 71-128 (with an intermediate announcement in PNAS 26 (1940) 215-221). There Douglas gave a complete solution for the case $n = 2$ — the three-dimensional $(x, y, z)$ space — dividing all systems into four primary cases according to the properties of $\Phi$. The modern literature credits Douglas's memoir with the two-dimensional case only; his announced complete solution for general $n$ is not accepted as such. Since the 1980s the problem has been pursued with geometric and exterior-differential-systems (EDS) methods — EDS being the framework that encodes a differential system as a differential ideal of forms with an independence condition and analyzes it by Cartan-type involution theory. Henneaux (1982) proposed an algorithm for the general problem and tied it to quantization; Sarlet (1982) put the Helmholtz conditions into the form above; Anderson and Thompson (1992) applied EDS theory and solved the first subcase for arbitrary $n$ ($\Phi$ a multiple of the identity); Crampin, Sarlet, Martinez, Byrnes and Prince (1994) reinterpreted Douglas's $n = 2$ analysis geometrically; Sarlet, Thompson and Prince (2002) and Aldridge, Prince, Sarlet and Thompson (2006) advanced the arbitrary-$n$ analysis; and Do and Prince (2016, 2021) solved a further arbitrary-$n$ subcase, proposed the first classification scheme for arbitrary dimension, and applied it to dimension 3. The remaining territory — a complete solution for arbitrary dimension — is the subject of the problem stated below.

Problem Statement

Give a complete solution, in Jesse Douglas's sense, of the inverse problem of the calculus of variations for systems of second-order ordinary differential equations in arbitrary dimension — equivalently, for every $n \geq 3$, since $n = 1$ was settled in the nineteenth century and $n = 2$ was completely solved by Douglas's 1941 memoir. For a given system $x''^a = F^a(t, x^b, x'^b)$, $a = 1, \dots, n$, the required solution jointly comprises: (i) an effective decision criterion — necessary and sufficient conditions for the existence of a non-degenerate multiplier matrix $g_{ab}(t, x, x')$ satisfying the Helmholtz conditions, equivalently for the $\infty^{2n}$-parameter family of integral curves to be the totality of extremals of some regular Lagrangian $\int L\,dt$; (ii) the construction of all corresponding Lagrangians when they exist — the full set of solutions of the multiplier problem, unique up to the addition of a total derivative, together with the number of arbitrary functions parametrizing them; and (iii) an exhaustive classification of the possible cases for arbitrary $n$, generalizing Douglas's cases I-IV and covering every pointwise algebraic type of the Jacobi endomorphism $\Phi$, including non-diagonalisable $\Phi$ and repeated eigenvalues. These three parts are jointly posed as one scientific objective — the source's own definition of a complete solution is an existence-theoretic one (determining the consistency of the fundamental system $S$ and of the arbitrary constants or functions in its general integral), combined with finding all corresponding Lagrangians and enumerating the possible types. The problem concerns general systems; the already-solved subcases ($\Phi$ a multiple of the identity; $\Phi$ diagonalisable with distinct eigenvalues with all, or all but one, eigen co-distributions integrable) are not the target.

References

1. Jesse Douglas, "Solution of the Inverse Problem of the Calculus of Variations", Proceedings of the National Academy of Sciences 25(12) (1939) 631-637, DOI 10.1073/pnas.25.12.631, https://doi.org/10.1073/pnas.25.12.631
2. Jesse Douglas, "Theorems in the Inverse Problem in the Calculus of Variations", Proceedings of the National Academy of Sciences 26(3) (1940) 215-221, DOI 10.1073/pnas.26.3.215, https://doi.org/10.1073/pnas.26.3.215
3. Jesse Douglas, "Solution of the inverse problem of the calculus of variations", Transactions of the American Mathematical Society 50(1) (1941) 71-128, DOI 10.1090/S0002-9947-1941-0004740-5, https://doi.org/10.1090/S0002-9947-1941-0004740-5
4. David R. Davis, "The inverse problem of the calculus of variations in higher space", Transactions of the American Mathematical Society 30(4) (1928) 710-736, DOI 10.1090/s0002-9947-1928-1501455-2, https://doi.org/10.1090/s0002-9947-1928-1501455-2
5. H. von Helmholtz, "Ueber die physikalische Bedeutung des Princips der kleinsten Wirkung", Journal für die reine und angewandte Mathematik 100 (1887) 137-166, DOI 10.1515/crll.1887.100.137, https://doi.org/10.1515/crll.1887.100.137
6. A. Hirsch, "Die Existenzbedingungen des verallgemeinerten kinetischen Potentials", Mathematische Annalen 50 (1898) 429-441, DOI 10.1007/bf01448077, https://doi.org/10.1007/bf01448077
7. M. Henneaux, "On the inverse problem of the calculus of variations", Journal of Physics A: Mathematical and General 15(3) (1982) L93-L96, DOI 10.1088/0305-4470/15/3/002, https://doi.org/10.1088/0305-4470/15/3/002
8. W. Sarlet, "The Helmholtz conditions revisited. A new approach to the inverse problem of Lagrangian dynamics", Journal of Physics A: Mathematical and General 15(5) (1982) 1503-1517, DOI 10.1088/0305-4470/15/5/013, https://doi.org/10.1088/0305-4470/15/5/013
9. Ian Anderson and Gerard Thompson, "The inverse problem of the calculus of variations for ordinary differential equations", Memoirs of the American Mathematical Society 98, No. 473 (1992), 110 pp., DOI 10.1090/memo/0473, https://doi.org/10.1090/memo/0473
10. M. Crampin, W. Sarlet, E. Martinez, G. B. Byrnes, G. E. Prince, "Towards a geometrical understanding of Douglas's solution of the inverse problem of the calculus of variations", Inverse Problems 10(2) (1994) 245-260, DOI 10.1088/0266-5611/10/2/005, https://doi.org/10.1088/0266-5611/10/2/005
11. W. Sarlet, G. Thompson, G. E. Prince, "The inverse problem of the calculus of variations: the use of geometrical calculus in Douglas's analysis", Transactions of the American Mathematical Society 354(7) (2002) 2897-2919, DOI 10.1090/s0002-9947-02-02994-x, https://doi.org/10.1090/s0002-9947-02-02994-x
12. J. E. Aldridge, G. E. Prince, W. Sarlet, G. Thompson, "An EDS approach to the inverse problem in the calculus of variations", Journal of Mathematical Physics 47(10) (2006) 103508, DOI 10.1063/1.2358000, https://doi.org/10.1063/1.2358000
13. Thoan Do and Geoff Prince, "New progress in the inverse problem in the calculus of variations", Differential Geometry and its Applications 45 (2016) 148-179, DOI 10.1016/j.difgeo.2016.01.005, https://doi.org/10.1016/j.difgeo.2016.01.005 (preprint arXiv:1412.1810)
14. Thoan Do and Geoff Prince, "The inverse problem in the calculus of variations: new developments", Communications in Mathematics 29(1) (2021) 131-149, DOI 10.2478/cm-2021-0008, https://doi.org/10.2478/cm-2021-0008 (preprint arXiv:2001.06771)
15. M. Farré Puiggalí and T. Mestdag, "The inverse problem of the calculus of variations and the stabilization of controlled Lagrangian systems", SIAM Journal on Control and Optimization 54(6) (2016) 3297-3318, DOI 10.1137/16M1060091, https://doi.org/10.1137/16M1060091 (preprint arXiv:1602.01673, https://arxiv.org/abs/1602.01673)