The Salem spectrum of Coxeter groups: exact range below the golden ratio and essential elements

For a Coxeter system (W,S) with its geometric representation, let $\lambda(w)$ be the spectral radius of an element $w \in W$. McMullen (2002) proved that $\lambda(w)$ is either $1$ or at least Lehmer's number $\lambda_L \approx 1.1762808$, with equality attained by the Coxeter element of the star diagram $Y_{2,3,7}$, and classified the irreducible systems whose bicolored eigenvalue $\beta(W,S)$ lies below the plastic number $\lambda_P \approx 1.324717$. The paper leaves open the precise range of Salem numbers arising as eigenvalues in Coxeter groups — including the classification of systems with $1 < \beta(W,S)$ below the golden ratio $\lambda_G \approx 1.618034$, for which the paper states the expectation that a similar classification can be completed — and whether all Coxeter elements are essential. The open questions considered here are that cluster. Adjacent reformulations of the realization question have since been resolved in large part — the even-unimodular-lattice variant by Bayer-Fluckiger and Taelman (irreducible case) and the hyperbolic growth-rate variant negatively by Kellerhals and Liechti — but the core Coxeter-group range question below the golden ratio remains open.

Background

A Coxeter system $(W,S)$ is a group $W$ with a finite generating set $S = \{s_1,\dots,s_n\}$ subject only to the relations $(s_is_j)^{m_{ij}} = 1$, where $m_{ii}=1$ and $m_{ij} = m_{ji} \in \{2,3,\dots\}$ (or $\infty$). Its Coxeter diagram is the weighted graph with an edge of weight $m_{ij} \ge 3$ joining $s_i$ and $s_j$ whenever $m_{ij} \ge 3$. The geometric (Tits) representation realizes $W$ on $V = \mathbb{R}^S$ by reflections preserving the symmetric bilinear form with $B(e_i,e_j) = -2\cos(\pi/m_{ij})$. For any Coxeter system, let $\alpha(W,S)$ be the leading eigenvalue of the Coxeter adjacency matrix $A_{ij} = 2\cos(\pi/m_{ij})$ ($i \ne j$; $A_{ii}=0$), and define the bicolored eigenvalue $\beta(W,S) \ge 1$ as the largest root of $\beta + \beta^{-1} + 2 = \alpha(W,S)^2$, with $\beta(W,S)=1$ if $\alpha(W,S) < 2$. For an irreducible system, $B$ is positive definite, positive semidefinite with nontrivial radical, or indefinite according as $\alpha(W,S) < 2$, $= 2$, or $> 2$; the indefinite (hyperbolic-or-higher-rank) case splits into hyperbolic systems of signature $(p,1)$ — one negative and $p$ positive eigenvalues of $B$, with $W$ acting discretely on hyperbolic space $\mathbb{H}^p$ — and higher-rank systems of signature $(p,q)$ with $q \ge 2$. For a hyperbolic system, $\log \lambda(w)$ equals the hyperbolic length of the closed geodesic on the orbifold $\mathbb{H}^p/W$ representing $w$: the spectral radius $\lambda(w)$ of $w$ acting on $V$ is a length-spectrum invariant of the reflection orbifold. $\beta$ is monotone for the domination order on diagrams, where a diagram dominates another when it is obtained by adding vertices and edges and/or increasing weights. An element $w \in W$ is essential if it is not conjugate into any standard parabolic subgroup $W_I$ generated by a proper subset $I \subsetneq S$. The Coxeter elements are the permuted products $s_{\sigma_1}s_{\sigma_2}\cdots s_{\sigma_n}$; when the diagram is bipartite with coloring $S = S_1 \sqcup S_2$ the products $S_1S_2$ are the bicolored Coxeter elements, all conjugate. A Salem number is a real algebraic integer $\lambda > 1$ whose other Galois conjugates all satisfy $|z| \le 1$ with at least one on the unit circle; a Pisot number has all other conjugates strictly inside the unit circle. The plastic number $\lambda_P \approx 1.324717$ (root of $x^3 = x+1$) is the smallest Pisot number; the golden ratio $\lambda_G = (1+\sqrt{5})/2 \approx 1.618034$ is the smallest accumulation point of the Pisot set; Lehmer's number $\lambda_L \approx 1.1762808$, root of $x^{10}+x^9-x^7-x^6-x^5-x^4-x^3+x+1$, is the smallest known Salem number, and Lehmer's conjecture asserts that every algebraic integer that is not a root of unity has Mahler measure — the product of the moduli of its conjugates outside the unit circle, equal to $\lambda$ itself for a Salem number — at least $\lambda_L$. Let $Y_{a,b,c}$ denote the star diagram with three arms of lengths $a,b,c$; the hyperbolic versions of the exceptional spherical diagrams $E_6, E_7, E_8$ are $Y_{3,3,4}$, $Y_{2,4,5}$, $Y_{2,3,7}$. The source paper proves: (Theorem 1.1) $\lambda(w) = 1$ or $\lambda(w) \ge \lambda_L$ for every $w \in W$; (Theorem 1.2) for essential $w$, $\lambda(w) \ge \inf_{\sigma} \lambda(s_{\sigma_1}\cdots s_{\sigma_n})$; (Theorem 1.3) $\inf_{\sigma} \lambda(s_{\sigma_1}\cdots s_{\sigma_n}) \ge \beta(W,S)$; (Theorem 1.5) there are 38 minimal hyperbolic Coxeter systems — hyperbolic systems minimal for domination, i.e. every strictly smaller system has only spherical and affine components — and $\inf \beta(W,S) = \lambda_L$ over them. In §7 it further proves: the smallest Salem numbers of degrees 6, 8, 10 are the Coxeter-element radii of $Y_{3,3,4}$, $Y_{2,4,5}$, $Y_{2,3,7}$ (so $\lambda_L = \lambda(w)$ for the Coxeter element of $Y_{2,3,7}$); the irreducible systems with $1 < \beta < \lambda_P$ are exactly $Y_{2,4,5}$ and $Y_{2,3,n}$, $n \ge 7$; $\lambda_P = \inf\{\beta(W,S) : (W,S)$ higher rank$\}$ and $\lambda_P$ is a limit of $\beta$-values (from below via $Y_{2,3,n}$, from above via the doubled diagrams $Y_{2,3,n} \vee Y_{2,3,n}$); exactly 6 Salem numbers below $1.3$ arise as eigenvalues in Coxeter groups (five as the Coxeter eigenvalues $\beta(Y_{2,3,n})$, $7 \le n \le 11$, namely $1.17628, 1.23039, 1.26123, 1.28064, 1.29349$, and the sixth, of degree 10, as $\lambda(w)$ for a non-Coxeter element of $Y_{2,3,7}$, namely $1.21639$; note $\beta(Y_{2,4,5}) = \beta(Y_{2,3,10}) = 1.28064$ is the degree-8 member of the five), out of 47 known Salem numbers below $1.3$; the second smallest known Salem number $\approx 1.188368$ arises as $\lambda(g)$ for $g \in O^+(II_{17,1})$ but not for any element of the index-two Coxeter subgroup $W \subset O^+(II_{17,1})$; the $\beta$-values are dense in $[\lambda_G,\infty)$ (via Shearer's theorem that the leading eigenvalues of trees are dense in $[\sqrt{2+\sqrt{5}},\infty)$); and, citing the classification of graphs with spectral radius between 2 and $\sqrt{2+\sqrt{5}}$ by Brouwer and Neumaier, the paper states the expectation that a similar classification can be completed for Coxeter systems with $1 < \beta < \lambda_G$. Two adjacent realization questions frame the problem: which Salem numbers are spectral radii of isometries of even unimodular lattices (Gross–McMullen; Bayer-Fluckiger–Taelman), and which Salem numbers are growth rates of cocompact hyperbolic Coxeter groups (Floyd; Parry; Kellerhals–Liechti), where the growth rate is the reciprocal of the radius of convergence of the growth series of $W$ (rational by a formula of Steinberg).

The two adjacent realization questions involve object classes distinct from the present one: isometries of even unimodular lattices and automorphisms of K3 surfaces (via their dynamical degrees), and growth-series invariants of reflection groups, rather than the spectral radius of an element of a Coxeter system.

Problem Statement

Let $(W,S)$ be a Coxeter system with its geometric representation on $V = \mathbb{R}^S$, and for $w \in W$ let $\lambda(w)$ denote the spectral radius of the action of $w$ on $V$. Call a Salem number $\lambda$ Coxeter-realized if $\lambda = \lambda(w)$ for some element $w$ of some Coxeter system $(W,S)$. For elements of hyperbolic systems (signature $(p,1)$) this is the same as requiring that $\lambda$ be an eigenvalue of $w|_V$: such an element has exactly two eigenvalues off the unit circle, namely $\lambda(w)^{\pm 1}$, and for a reducible system the off-circle eigenvalues are just the spectral radii of the components; only non-leading eigenvalues of irreducible higher-rank systems could realize additional values, and whether they do below $\lambda_G$ is part of the question. Determine the precise range of Coxeter-realized Salem numbers near 1, concretely:

(A) Sub-golden classification. Classify, up to diagram isomorphism, all irreducible Coxeter systems with bicolored eigenvalue $1 < \beta(W,S) < \lambda_G$, where $\lambda_G = (1+\sqrt{5})/2$ is the golden ratio. The source proves the classification for $1 < \beta(W,S) < \lambda_P \approx 1.324717$ (exactly $Y_{2,4,5}$ and $Y_{2,3,n}$, $n \ge 7$), proves that an irreducible system with $1 < \beta(W,S) \le \lambda_G$ has a tree diagram (Proposition 7.5), and states the expectation that a similar classification can be completed for Coxeter systems with $1 < \beta < \lambda_G$; the target is that classification, with a proof of exhaustiveness.

(B) Range of realized Salem numbers. Using (A), determine exactly which Salem numbers in $(1, \lambda_G)$ are Coxeter-realized. Known: exactly six Salem numbers below $1.3$ arise as $\lambda(w)$ (the source, Theorem 7.9 and Table 6: the values $1.17628, 1.21639, 1.23039, 1.26123, 1.28064, 1.29349$), among the $47$ Salem numbers known below $1.3$ (a list complete through degree $20$). Determine the status of every Salem number in $(1.3, \lambda_G)$, and whether any non-leading eigenvalue of an irreducible higher-rank system realizes a Salem number below $\lambda_G$ that no spectral radius does; for each realizable one exhibit the diagram and element. Since the set of Salem numbers below $\lambda_G$ is not known to be effectively enumerable (Lehmer's conjecture is open), a decision procedure over an external catalog cannot by itself settle the question.

(C) Essential elements (auxiliary). Prove or disprove: every Coxeter element of every Coxeter system is essential, i.e. not conjugate into any proper standard parabolic subgroup. The source proves $\lambda(w) \ge \inf_\sigma \lambda(s_{\sigma_1}\cdots s_{\sigma_n})$ for essential $w$ (Theorem 1.2) and poses this question in section 4; a positive answer would make every Coxeter element essential, so the essential spectrum $\inf\{\lambda(w) : w \in W$ essential$\}$ would coincide with the infimum of $\lambda$ over Coxeter elements, which by Theorem 1.3 lies at or above $\beta(W,S)$ and in the bipartite case is attained at $\beta(W,S)$ itself.

References

1. Curtis T. McMullen, Coxeter groups, Salem numbers and the Hilbert metric, Publications Mathematiques de l'Institut des Hautes Etudes Scientifiques 95 (2002), 151-183. DOI: 10.1007/s102400200001, https://doi.org/10.1007/s102400200001
2. Benedict H. Gross and Curtis T. McMullen, Automorphisms of even unimodular lattices and unramified Salem numbers, Journal of Algebra 257 (2002), no. 2, 265-290. DOI: 10.1016/s0021-8693(02)00552-5, https://doi.org/10.1016/s0021-8693(02)00552-5
3. Eva Bayer-Fluckiger and Lenny Taelman, Automorphisms of even unimodular lattices and equivariant Witt groups, Journal of the European Mathematical Society 22 (2020), no. 11, 3467-3490. DOI: 10.4171/jems/991, https://doi.org/10.4171/jems/991, arXiv:1708.05540, https://arxiv.org/abs/1708.05540
4. Eva Bayer-Fluckiger, Automorphisms of K3 surfaces, signatures, and isometries of lattices, Journal of the European Mathematical Society, first published online 30 January 2025. DOI: 10.4171/jems/1598, https://doi.org/10.4171/jems/1598, arXiv:2209.06698, https://arxiv.org/abs/2209.06698
5. Yuta Takada, Characteristic polynomials of isometries of even unimodular lattices, arXiv:2401.12620 (2024), https://arxiv.org/abs/2401.12620
6. Ruth Kellerhals and Livio Liechti, Salem numbers, spectral radii and growth rates of hyperbolic Coxeter groups, Transformation Groups 28 (2023), no. 2, 831-852 (online first 2022). DOI: 10.1007/s00031-022-09715-x, https://doi.org/10.1007/s00031-022-09715-x, arXiv:2008.10965, https://arxiv.org/abs/2008.10965
7. Ruth Kellerhals and Alexander Kolpakov, The minimal growth rate of cocompact Coxeter groups in hyperbolic 3-space, Canadian Journal of Mathematics 66 (2014), no. 2, 354-372. DOI: 10.4153/cjm-2012-062-3, https://doi.org/10.4153/cjm-2012-062-3
8. Naomi Bredon and Ruth Kellerhals, Hyperbolic Coxeter groups and minimal growth rates in dimensions four and five, Groups, Geometry, and Dynamics 16 (2022), no. 2, 725-741. DOI: 10.4171/ggd/663, https://doi.org/10.4171/ggd/663, arXiv:2008.10961, https://arxiv.org/abs/2008.10961
9. William J. Floyd, Growth of planar Coxeter groups, P.V. numbers, and Salem numbers, Mathematische Annalen 293 (1992), no. 1, 475-483. DOI: 10.1007/bf01444729, https://doi.org/10.1007/bf01444729
10. W. Parry, Growth series of Coxeter groups and Salem numbers, Journal of Algebra 154 (1993), no. 2, 406-415. DOI: 10.1006/jabr.1993.1022, https://doi.org/10.1006/jabr.1993.1022
11. Eknath Ghate and Eriko Hironaka, The arithmetic and geometry of Salem numbers, Bulletin of the American Mathematical Society 38 (2001), no. 3, 293-314. DOI: 10.1090/S0273-0979-01-00902-8, https://doi.org/10.1090/S0273-0979-01-00902-8
12. Eriko Hironaka, The Lehmer polynomial and pretzel links, Canadian Mathematical Bulletin 44 (2001), no. 4, 440-451. DOI: 10.4153/cmb-2001-044-x, https://doi.org/10.4153/cmb-2001-044-x
13. Lorenz Halbeisen and Norbert Hungerbuhler, On periodic billiard trajectories in obtuse triangles, SIAM Review 42 (2000), no. 4, 657-670. DOI: 10.1137/s0036144599355725, https://doi.org/10.1137/s0036144599355725
14. Chris Smyth, Seventy years of Salem numbers, Bulletin of the London Mathematical Society 47 (2015), no. 3, 379-395. DOI: 10.1112/blms/bdv027, https://doi.org/10.1112/blms/bdv027 (preprint: arXiv:1408.0195, https://arxiv.org/abs/1408.0195, titled 'Survey article: Seventy years of Salem numbers')
15. Vincent Emery, John G. Ratcliffe and Steven T. Tschantz, Salem numbers and arithmetic hyperbolic groups, Transactions of the American Mathematical Society 372 (2019), no. 1, 329-355. DOI: 10.1090/tran/7655, https://doi.org/10.1090/tran/7655
16. Keiji Oguiso, The third smallest Salem number in automorphisms of K3 surfaces, in Algebraic Geometry in East Asia - Seoul 2008, Advanced Studies in Pure Mathematics 60 (2010), 331-360. DOI: 10.2969/aspm/06010331, https://doi.org/10.2969/aspm/06010331, arXiv:0905.2396, https://arxiv.org/abs/0905.2396
17. Andries E. Brouwer and Arnold Neumaier, The graphs with spectral radius between 2 and sqrt(2+sqrt(5)), Linear Algebra and its Applications 114-115 (1989), 273-276. DOI: 10.1016/0024-3795(89)90466-7, https://doi.org/10.1016/0024-3795(89)90466-7