The $d''$ homomorphism condition: characterizing closed range of the Cauchy-Riemann operator and Hausdorff Dolbeault cohomology in Serre duality

Serre's 1955 duality theorem for a complex manifold $X$ and holomorphic vector bundle $E$ pairs $H^q(X,\Omega^p(E))$ with the compactly supported group $H_c^{n-q}(X,\Omega^{n-p}(E^*))$, and, when the groups are Hausdorff -- equivalently when the Cauchy-Riemann operator $d''=\bar\partial$ is a homomorphism in Serre's sense, i.e. has closed range -- identifies one with the topological dual of the other; the 1955 paper itself contains an example where the condition fails. Two companion questions arising from the paper's proof technique are settled in the literature: the flatness/Tor question underlying the extension of the Dolbeault resolution to arbitrary coherent analytic sheaves (settled via the division of distributions, as recorded by Ramis and Ruget, and by Suominen), and the extension to singular complex spaces and general coherent sheaves (dualizing-complex theory of Ramis-Ruget, Ramis-Ruget-Verdier, and Andreotti-Kas, with modern explicit versions). The surviving open core is the homomorphism (closed-range) condition: no geometric characterization is known of the manifolds, domains, and bundles for which $\bar\partial$ has closed range and Dolbeault cohomology is Hausdorff. Necessity of the condition is classical (Laufer), broad sufficient conditions exist ($q$-convexity, Carleman estimates, pseudoconvexity-type hypotheses), and recent counterexamples (Chakrabarti; Chakrabarti-Shaw, exhibiting non-Hausdorff $L^2$-cohomology on a smooth bounded Stein domain) show that the boundary of the property is not yet geometrically understood.

Background

Let $X$ be a complex manifold of complex dimension $n$, that is, a space locally modeled on open subsets of $\mathbb{C}^n$ with holomorphic transition maps, and let $E\to X$ be a holomorphic vector bundle (a complex vector bundle whose transition functions are holomorphic). The exterior derivative of a complex differential form splits as $d=d'+d''$, where $d''=\bar\partial$ is the Cauchy-Riemann part, built in local coordinates from the differentials $\partial/\partial\bar z_j$. The Dolbeault cohomology groups $H^q(X,\Omega^p(E))$ are computed either as the cohomology of the $\bar\partial$ complex on $C^\infty$ forms of bidegree $(p,q)$ with values in $E$, or, via the Dolbeault resolution -- the resolution of the sheaf $\Omega^p(E)$ of holomorphic $p$-forms with values in $E$ by the fine sheaves of $C^\infty$ $(p,q)$-forms ('fine' means admitting partitions of unity, which makes these sheaves acyclic so that cohomology is computed by an explicit complex) -- as sheaf cohomology. These groups are basic invariants of complex geometry: for $p=0$ they reduce to the cohomology $H^q(X,\mathcal{O}(E))$ of the sheaf of holomorphic sections.

In his 1955 paper 'Un théorème de dualité', Serre proved the analytic analogue of Poincaré duality for these groups: integration of the wedge product over $X$ defines a canonical bilinear pairing between $H^q(X,\Omega^p(E))$ and the compactly supported group $H_c^{n-q}(X,\Omega^{n-p}(E^*))$, where $E^*$ is the dual bundle. He equipped the groups with their natural function-space topologies -- quotients of Fréchet spaces of $C^\infty$ forms (a Fréchet space being a complete topological vector space whose topology is given by countably many seminorms, as for $C^\infty$ functions with uniform convergence of all derivatives on compact sets), and, in the compactly supported case, quotients of the strong duals of such spaces, the strong dual carrying the topology of uniform convergence on bounded sets -- and showed that, whenever the relevant groups are Hausdorff (in his French terminology, 'séparées'), the pairing identifies each group with the strong topological dual of the other. The Hausdorff condition is equivalent to $d''$ being a homomorphism in Bourbaki's topological-vector-space sense -- open onto its image -- which for these Fréchet and dual-Fréchet complexes is the same as $d''$ having closed range, which is in turn equivalent to the relevant Dolbeault cohomology groups being Hausdorff in the quotient topology. The condition is not vacuous: Serre's paper itself contains an example (its section 14, as later literature citing it records) of a domain in $\mathbb{C}^n$ with non-Hausdorff Dolbeault cohomology, and the closed-range hypothesis is in fact necessary for the duality statement (Laufer 1967).

Serre's proof technique raised two companion questions that the literature has since settled. First, the Dolbeault resolution is built from the sheaf of $C^\infty$ functions, and for a coherent analytic sheaf $\mathcal{F}$ -- a sheaf locally presented as the cokernel of a matrix of holomorphic functions, the natural class generalizing vector bundles -- the exactness of the tensor-product complex $0\to\mathcal{F}\to\mathcal{F}\otimes_\mathcal{O} A^{0,\bullet}$ requires the vanishing of the higher Tor groups, that is, flatness of the sheaf of $C^\infty$ functions over the structure sheaf $\mathcal{O}$ of holomorphic functions. This was settled using the division theorems for distributions, as recorded by Ramis and Ruget (1970), who attribute the difficult flatness statements to Malgrange's work on the division of distributions (Malgrange 2003 gives his own account of that theory), and by Suominen (1969), who worked out duality for coherent sheaves on analytic manifolds. Second, the extension to singular complex spaces and arbitrary coherent sheaves was accomplished by dualizing-complex theory: Ramis and Ruget (1970) constructed the dualizing complex $K_X$ of a complex space and proved a perfect pairing between the separated quotients of $H^p(X,\mathcal{F})$ and $\mathrm{Ext}^p(X;\mathcal{F},K_X)$, with $\mathrm{Ext}^p$ denoting the $p$-th derived functor of the sheaf-Hom functor, for every coherent $\mathcal{F}$, with separatedness of one equivalent to separatedness of the other; Ramis, Ruget, and Verdier (1971) and Andreotti and Kas (1973) extended the theory, and modern work constructs explicit fine sheaves of currents realizing the duality concretely (Ruppenthal, Samuelsson Kalm, and Wulcan 2017; Samuelsson Kalm 2021). In all of these generalizations the Hausdorff/separation hypothesis remains essential and is currently verifiable only through sufficient conditions such as compactness, holomorphic convexity (the property that the holomorphic hull of every compact set -- the set of points at which every holomorphic function is bounded by its supremum on the compact set -- is again compact), or $q$-convexity (the existence of a smooth exhaustion function, proper and bounded below, whose Levi form, the complex Hessian of the function, has at least $n-q+1$ positive eigenvalues outside a compact subset; smaller $q$ is a stronger hypothesis).

What remains uncharacterized is the closed-range property itself. Known sufficient conditions include finiteness and vanishing theorems for $q$-convex spaces (Andreotti and Grauert 1962), Carleman estimates -- a priori energy estimates with power-type weights that degenerate at the boundary -- giving closed range on $q$-convex and $q$-pseudoconcave (the analogous condition with concavity replacing convexity in the Levi-form inequality) manifolds (Andreotti and Vesentini 1965), and separation theorems under explicit geometric hypotheses (Laurent-Thiébaut and Leiterer 2000, 2002; Laurent-Thiébaut and Shaw 2013). Known failures in the smooth/Fréchet setting include the example in Serre's paper and non-pseudoconvex Hartogs-type domains -- Hartogs-type domains being domains of the general shape of the classical Hartogs extension figure, defined by radius conditions in some of the variables, and pseudoconvexity being the several-complex-variables analogue of convexity which, by the solution of the Levi problem, characterizes domains of holomorphy -- with explicitly computable non-Hausdorff Dolbeault cohomology (Chakrabarti 2015). In the $L^2$ realization -- a distinct setting, since the Hausdorff property of the smooth and $L^2$ complexes need not agree -- closed range can fail even on a bounded smooth Stein domain (Chakrabarti and Shaw 2015); a Stein domain is an open subset of a Stein manifold -- a complex manifold that is holomorphically convex and whose holomorphic functions separate points, the class on which the function theory of several complex variables is best behaved -- and by Steinness its smooth Dolbeault cohomology vanishes, so this $L^2$ failure is striking but is not a failure instance for the smooth/Fréchet setting of Serre's theorem. No geometric characterization of the manifolds, domains, and bundles for which $\bar\partial$ has closed range is known.

Problem Statement

Characterize those complex manifolds $X$, domains $D\subset X$, and holomorphic vector bundles $E$ for which the $d''=\bar\partial$ operator has closed range on the Fréchet spaces of $C^\infty$ forms -- for a domain $D$, forms smooth in the interior of $D$ with the topology of uniform convergence of all derivatives on compact subsets of $D$, the setting of Serre's theorem -- in the bidegrees $(p,q)$ entering Serre's duality pairing; equivalently, for which the Dolbeault cohomology groups are Hausdorff in the quotient topology and Serre's duality theorem identifies $H^q(X,\Omega^p(E))$ with the strong topological dual of $H_c^{n-q}(X,\Omega^{n-p}(E^*))$. The compact, holomorphically convex, $q$-convex, and pseudoconvex settings (in this smooth/Fréchet sense) are among the known positive cases, while the example in section 14 of Serre's paper and the non-pseudoconvex Hartogs-type domains of Chakrabarti with explicitly computable non-Hausdorff cohomology are known failures within the same setting. In the $L^2$ realization the closed-range property is a distinct question: there it can fail even on a smooth bounded Stein domain whose smooth Dolbeault cohomology vanishes (Chakrabarti and Shaw 2015); this motivates the problem but is not a failure instance of the smooth/Fréchet characterization sought here.

References

1. Jean-Pierre Serre (1955). Un théorème de dualité. Commentarii Mathematici Helvetici 29, 9-26. DOI 10.1007/BF02564268. https://doi.org/10.1007/BF02564268
2. Aldo Andreotti, Hans Grauert (1962). Théorèmes de finitude pour la cohomologie des espaces complexes. Bulletin de la Société Mathématique de France 90, 193-259. DOI 10.24033/bsmf.1581. https://doi.org/10.24033/bsmf.1581
3. Aldo Andreotti, Edoardo Vesentini (1965). Carleman estimates for the Laplace-Beltrami equation on complex manifolds. Publications Mathématiques de l'IHÉS 25, 81-130. DOI 10.1007/BF02684398. https://doi.org/10.1007/BF02684398
4. Henry B. Laufer (1967). On Serre duality and envelopes of holomorphy. Transactions of the American Mathematical Society 128, 414-436. DOI 10.1090/S0002-9947-1967-0222341-9. https://doi.org/10.1090/S0002-9947-1967-0222341-9
5. Kalevi Suominen (1969). Duality for coherent sheaves on analytic manifolds. Annales Academiae Scientiarum Fennicae Series A I, no. 424, 1-19. DOI 10.5186/aasfm.1969.424. https://doi.org/10.5186/aasfm.1969.424
6. Jean-Pierre Ramis, Gabriel Ruget (1970). Complexe dualisant et théorèmes de dualité en géométrie analytique complexe. Publications Mathématiques de l'IHÉS 38, 77-91. DOI 10.1007/BF02684652. https://numdam.org/item/PMIHES_1970__38__77_0/
7. Jean-Pierre Ramis, Gabriel Ruget, Jean-Louis Verdier (1971). Dualité relative en géométrie analytique complexe. Inventiones Mathematicae 13, 261-283. DOI 10.1007/BF01406078. https://doi.org/10.1007/BF01406078
8. Aldo Andreotti, Arnold Kas (1973). Duality on complex spaces. Annali della Scuola Normale Superiore di Pisa, Classe di Scienze, Serie III 27, no. 2, 187-263. Numdam item ASNSP_1973_3_27_2_187_0. https://numdam.org/item/ASNSP_1973_3_27_2_187_0/
9. Christine Laurent-Thiébaut, Jürgen Leiterer (2000). On Serre duality. Bulletin des Sciences Mathématiques 124, 93-106. DOI 10.1016/S0007-4497(00)00126-3. https://doi.org/10.1016/S0007-4497(00)00126-3
10. Christine Laurent-Thiébaut, Jürgen Leiterer (2002). A separation theorem and Serre duality for the Dolbeault cohomology. Arkiv för Matematik 40, 301-321. DOI 10.1007/BF02384538. https://doi.org/10.1007/BF02384538
11. Bernard Malgrange (2003). Idéaux de fonctions différentiables et division des distributions. Journées mathématiques X-UPS, 'Dans le sillage de Laurent Schwartz', 1-25 (published online by the Centre Mersenne, 2024). DOI 10.5802/xups.2003-01. https://doi.org/10.5802/xups.2003-01
12. Debraj Chakrabarti, Mei-Chi Shaw (2012). $L^2$ Serre duality on domains in complex manifolds and applications. Transactions of the American Mathematical Society 364, 3529-3554. DOI 10.1090/S0002-9947-2012-05511-5. arXiv:1006.3117. https://arxiv.org/abs/1006.3117
13. Christine Laurent-Thiébaut, Mei-Chi Shaw (2013). On the Hausdorff property of some Dolbeault cohomology groups. Mathematische Zeitschrift 274, no. 3-4, 1165-1176 (published online December 2012). DOI 10.1007/s00209-012-1111-z. arXiv:1208.6417. https://arxiv.org/abs/1208.6417
14. Debraj Chakrabarti (2015). Some non-pseudoconvex domains with explicitly computable non-Hausdorff Dolbeault cohomology. Archiv der Mathematik 105, 571-584. DOI 10.1007/s00013-015-0843-5. https://doi.org/10.1007/s00013-015-0843-5
15. Debraj Chakrabarti, Mei-Chi Shaw (2015). The $L^2$-cohomology of a bounded smooth Stein domain is not necessarily Hausdorff. Mathematische Annalen 363, 1001-1021. DOI 10.1007/s00208-015-1193-0. https://doi.org/10.1007/s00208-015-1193-0
16. Jean Ruppenthal (2015). $L^2$-Serre duality on singular complex spaces and applications. In: Complex Analysis and Geometry, Springer Proceedings in Mathematics & Statistics, 309-318. DOI 10.1007/978-4-431-55744-9_23. arXiv:1409.1382. https://arxiv.org/abs/1409.1382
17. Jean Ruppenthal, Håkan Samuelsson Kalm, Elizabeth Wulcan (2017). Explicit Serre duality on complex spaces. Advances in Mathematics 305, 1320-1355. DOI 10.1016/j.aim.2016.10.013. arXiv:1401.8093. https://arxiv.org/abs/1401.8093
18. Håkan Samuelsson Kalm (2021). The $\bar\partial$-equation, duality, and holomorphic forms on a reduced complex space. The Journal of Geometric Analysis 31, no. 2, 1786-1820 (published online 2019). DOI 10.1007/s12220-019-00325-w. arXiv:1506.07842. https://arxiv.org/abs/1506.07842