[Low-degree structure of $H^*(B\mathcal{G}_k)$]\label{thm:main}
Work $2$--locally and fix an integer $k$.  Consider the Serre spectral
sequence \eqref{eq:Serre-SS} for the fibration
$\Omega^3_0G_2\to B\mathcal{G}_k\to B\G$, and let
$M=H^*(\Omega^3_0G_2;\mathbb{F}_2)$.
\begin{enumerate}[label=\textup{(\roman*)}]
  \item In total degree $j=s+t\le 10$ the $E_2$--term is naturally
  isomorphic, as a bigraded $H^*(B\G)$--module, to
  \[
  E_2^{*,*}(k)^{\le 10} \;\cong\;
  H^*(B\G)\otimes M^{\le 10},
  \]
  with $E_2^{s,t}=H^s(B\G)\otimes M^t$.

  \item Using the low-degree homotopy groups of $G_2$ and the Hurewicz
  theorem, one has
  \[
  M^j = H^j(\Omega^3_0G_2;\mathbb{F}_2)=0\quad\text{for }1\le j\le 4,
  \qquad
  M^5\neq 0.
  \]
  In particular, there exists a nonzero class
  \[
  u_5 \in M^5\cong E_2^{0,5}(k)
  \]
  and $u_5$ is the first positive-degree generator of
  $H^*(\Omega^3_0G_2)$.

  \item In total degree $j\le 10$ the differentials
  \[
  d_r\colon E_r^{0,t}(k)\longrightarrow E_r^{r,t-r+1}(k)
  \]
  with $2\le r\le 5$ vanish on all classes in bidegree $(0,t)$, and
  the only page on which a differential with source in $E_r^{0,5}(k)$
  can be nonzero is $r=6$.  After choosing a generator
  $x_6\in H^6(B\G)\cong E_2^{6,0}(k)$ and $u_5\in M^5\cong E_2^{0,5}(k)$
  there is a uniquely determined scalar $\varepsilon(k)\in\mathbb{F}_2$ such that
  \[
  d_6(u_5) = \varepsilon(k)\, x_6
  \]
  in $E_6^{6,0}(k)$.

  \item Let $U\subset E_2^{*,*}(k)$ denote the $H^*(B\G)$--submodule
  generated by $u_5$.  In total degree $j\le 10$ all differentials
  $d_r$ with $2\le r\le 5$ vanish on $U$, and every differential
  \[
  d_6\colon E_6^{s,5}(k)\longrightarrow E_6^{s+6,0}(k)
  \]
  with $s+5\le 10$ satisfies
  \[
  d_6(x\cdot u_5) = x\cdot d_6(u_5)
  \qquad\text{for }x\in H^s(B\G).
  \]
  In particular, within $U$ the entire effect of the Serre
  differentials in total degree $\le 10$ is determined by the single
  scalar $\varepsilon(k)\in\mathbb{F}_2$.

  \item For each $j\le 10$ the spectral sequence induces a finite
  filtration of $H^j(B\mathcal{G}_k)$ whose associated graded
  contains, as a direct summand, the image of $U$ in
  $E_\infty^{*,*}(k)^j$.  This summand is explicitly computable from
  $H^*(B\G)\otimes M^{\le 10}$ together with $\varepsilon(k)$, while
  the remaining extension term depends only on $M^{\le j}$ and the
  multiplicative structure of the Serre spectral sequence.
\end{enumerate}
In particular, in total degree $j\le 10$ the dependence of
$H^j(B\mathcal{G}_k;\mathbb{F}_2)$ on the bundle class $k$ that is visible
through the first nontrivial fibre class $u_5$ is completely encoded
in the single Serre differential $d_6(u_5)=\varepsilon(k)x_6$.

\begin{theorem}[Low-degree structure of $H^*(B\mathcal{G}_k)$]\label{thm:main}
Work $2$--locally and fix an integer $k$.  Consider the Serre spectral
sequence \eqref{eq:Serre-SS} for the fibration
$\Omega^3_0\G\to B\mathcal{G}_k\to \BG$, and let
$M=H^*(\Omega^3_0\G;\Ftwo)$.
\begin{enumerate}[label=\textup{(\roman*)}]
  \item In total degree $j=s+t\le 10$ the $E_2$--term is naturally
  isomorphic, as a bigraded $H^*(\BG)$--module, to
  \[
  E_2^{*,*}(k)^{\le 10} \;\cong\;
  H^*(\BG)\otimes M^{\le 10},
  \]
  with $E_2^{s,t}=H^s(\BG)\otimes M^t$.

  \item Using the low-degree homotopy groups of $\G$ and the Hurewicz
  theorem, one has
  \[
  M^j = H^j(\Omega^3_0\G;\Ftwo)=0\quad\text{for }1\le j\le 4,
  \qquad
  M^5\neq 0.
  \]
  In particular, there exists a nonzero class
  \[
  u_5 \in M^5\cong E_2^{0,5}(k)
  \]
  and $u_5$ is the first positive-degree generator of
  $H^*(\Omega^3_0\G)$.

  \item In total degree $j\le 10$ the differentials
  \[
  d_r\colon E_r^{0,t}(k)\longrightarrow E_r^{r,t-r+1}(k)
  \]
  with $2\le r\le 5$ vanish on all classes in bidegree $(0,t)$, and
  the only page on which a differential with source in $E_r^{0,5}(k)$
  can be nonzero is $r=6$.  After choosing a generator
  $x_6\in H^6(\BG)\cong E_2^{6,0}(k)$ and $u_5\in M^5\cong E_2^{0,5}(k)$
  there is a uniquely determined scalar $\varepsilon(k)\in\Ftwo$ such that
  \[
  d_6(u_5) = \varepsilon(k)\, x_6
  \]
  in $E_6^{6,0}(k)$.

  \item Let $U\subset E_2^{*,*}(k)$ denote the $H^*(\BG)$--submodule
  generated by $u_5$.  In total degree $j\le 10$ all differentials
  $d_r$ with $2\le r\le 5$ vanish on $U$, and every differential
  \[
  d_6\colon E_6^{s,5}(k)\longrightarrow E_6^{s+6,0}(k)
  \]
  with $s+5\le 10$ satisfies
  \[
  d_6(x\cdot u_5) = x\cdot d_6(u_5)
  \qquad\text{for }x\in H^s(\BG).
  \]
  In particular, within $U$ the entire effect of the Serre
  differentials in total degree $\le 10$ is determined by the single
  scalar $\varepsilon(k)\in\Ftwo$.

  \item For each $j\le 10$ the spectral sequence induces a finite
  filtration of $H^j(B\mathcal{G}_k)$ whose associated graded
  contains, as a direct summand, the image of $U$ in
  $E_\infty^{*,*}(k)^j$.  This summand is explicitly computable from
  $H^*(\BG)\otimes M^{\le 10}$ together with $\varepsilon(k)$, while
  the remaining extension term depends only on $M^{\le j}$ and the
  multiplicative structure of the Serre spectral sequence.
\end{enumerate}
In particular, in total degree $j\le 10$ the dependence of
$H^j(B\mathcal{G}_k;\Ftwo)$ on the bundle class $k$ that is visible
through the first nontrivial fibre class $u_5$ is completely encoded
in the single Serre differential $d_6(u_5)=\varepsilon(k)x_6$.
\end{theorem}

\subsection{Gauge groups and their homotopy types}

Let $G$ be a compact, connected Lie group, let $X$ be a pointed CW
complex, and let $P\to X$ be a principal $G$--bundle.  The \emph{gauge
group} $\mathcal{G}(P)$ is the topological group of $G$--equivariant
bundle automorphisms of $P$ that cover the identity on $X$.  When $X$
is finite, Crabb--Sutherland showed that there are only finitely many
homotopy types among the gauge groups $\mathcal{G}(P)$ as $P$ ranges
over all principal $G$--bundles over $X$~\cite{CrabbSutherland}.  In
the fundamental case $X=S^4$ and $G$ simply connected and simple,
isomorphism classes of $G$--bundles are classified by
$\pi_4(BG)\cong \mathbb{Z}$, and we write $\mathcal{G}_k(G)$ for the
gauge group of the bundle classified by $k\in \mathbb{Z}$.

The problem of determining the homotopy types of these gauge groups
for specific Lie groups $G$ has received sustained attention; see
for example \cite{KonoSU2,TheriaultSp2,KishimotoTheriaultTsutayaG2}.  In many cases
the classification is governed by the order of a fundamental Samelson
product in~$G$.

The case $G=G_2$ is particularly interesting because of the exceptional
nature of $G_2$.  Kishimoto, Theriault and Tsutaya proved that the
classification for $G_2$--gauge groups reduces to determining the
order of the Samelson product
\[
\langle i_3,1\rangle\in [\Sigma^3G_2,G_2],
\]
where $i_3\colon S^3\to G_2$ is the inclusion of the bottom cell and
$1$ denotes the identity map~\cite{KishimotoTheriaultTsutayaG2}.

Very recently, Kameko~\cite{KamekoG2gauge} resolved this issue at the prime $2$ and
completed the $p$--local classification of $G_2$--gauge groups.  He
proved that the order of $\langle i_3,1\rangle$ is exactly $84$
\cite[Theorem~1.1]{KamekoG2gauge}.
Consequently, the $p$--local homotopy type of $\mathcal{G}_k(G_2)$ depends only on
the greatest common divisor $(k,84)$.  Thus the homotopy types of the
gauge groups are completely understood up to $p$--localization.  It is
natural to ask how far this information is reflected in the
(co)homology of their classifying spaces.

\subsection{Cohomology of $B\mathcal{G}_k$ and the Steenrod algebra}

Gottlieb identified the classifying space of $\mathcal{G}_k$ with a
component of a mapping space~\cite{GottliebGauge}: specifically,
$B\mathcal{G}_k$ is homotopy equivalent to $\mathrm{Map}_k(S^4,BG)$,
the connected component of the mapping space $\mathrm{Map}(S^4,BG)$
containing the classifying map $k\colon S^4\to BG$.
This identification yields a fundamental evaluation fibration
\[
\Omega^3_0 G \;\longrightarrow\; B\mathcal{G}_k
\xrightarrow{\;\mathrm{ev}\;} BG.
\]
For $G=G_2$, this expresses $B\mathcal{G}_k$ as a twisted
mapping space over the base $B\G$.

The mod~$2$ cohomology of $B\G$ is classically known: Borel showed that
\[
H^*(B\G;\mathbb{F}_2) \cong \mathbb{F}_2[x_4,x_6,x_7]
\]
is a polynomial algebra on generators in degrees $4,6,7$
\cite{BorelCohomology}.  The action of the Steenrod algebra $\mathcal{A}$ on
$H^*(B\G)$ can be described explicitly via Wu formulas, and from the
viewpoint of the hit problem one is interested in minimal
generating sets for the unstable $\mathcal{A}$--module $\mathbb{F}_2[x_4,x_6,x_7]$.

By contrast, there seems to be no published description of the full
mod~$2$ cohomology ring $H^*(B\mathcal{G}_k;\mathbb{F}_2)$ for any
nontrivial $k$, nor of its structure as an unstable $\mathcal{A}$--module.
Our aim here is to give a precise structural description of
$H^*(B\mathcal{G}_k;\mathbb{F}_2)$ in low degrees, in terms of the evaluation
fibration and the first nontrivial cohomology of the triple loop space
$\Omega^3_0G_2$.

\subsection{Main result: a low-degree description}

We work $2$--locally throughout and write $H^*(-)$ for
$H^*(-;\mathbb{F}_2)$.  Consider the Serre spectral sequence of the
evaluation fibration
\begin{equation}\label{eq:Serre-SS}
E_2^{s,t}(k)\;\cong\; H^s(B\G;H^t(\Omega^3_0G_2))
\;\Longrightarrow\; H^{s+t}(B\mathcal{G}_k).
\end{equation}
The $E_2$--term is a module over $H^*(B\G)$ and an unstable module
over the Steenrod algebra.  The twisting of this spectral sequence
depends on $k$ through the connecting map
\[
\partial_k\colon G_2 \longrightarrow \Omega^3_0 G_2,
\]
which is the triple adjoint of the Samelson product $\langle
k\cdot i_3,1\rangle$ by work of
Kishimoto--Theriault--Tsutaya~\cite{KishimotoTheriaultTsutayaG2}.
Kameko's calculation of the $2$--primary order of $\langle i_3,1\rangle$
implies that, after localization at $2$, the map $\partial_k$ depends
only on $k\bmod 4$.

The cohomology of the fibre $\Omega^3_0 G$ can in principle be
computed by iterated Eilenberg--Moore spectral sequences applied to
the path-loop fibrations of $BG$ and $G$.
Choi and Yoon conjectured that, for any simply connected finite
$H$--space $X$, the Eilenberg--Moore spectral sequences converging
to the mod $p$ homology of $\Omega^2 X$ and $\Omega^3 X$ collapse at
$E_2$~\cite{ChoiYoonEMSS}.  This conjecture was subsequently proved by
Lin~\cite{LinEMSS}; in particular, for any compact simple Lie group
$G$ and any prime $p$, the Eilenberg--Moore spectral sequences computing
$H_*(\Omega^2 G;\mathbb F_p)$ and $H_*(\Omega^3 G;\mathbb F_p)$
collapse at $E_2$.  Combined with the classical description of
$H^*(B\G)$, this implies that $H^*(\Omega^3_0G_2)$ is of finite type in
each degree and that its generators occur in bounded degrees.  For our
purposes, however, we only need to know the very first nontrivial
cohomology group of $\Omega^3_0G_2$.

Let
\[
M \;=\; H^*(\Omega^3_0G_2;\mathbb{F}_2)
\]
and write $M^{\le N}=\bigoplus_{j\le N}M^j$ for its truncation in
degrees $\le N$.  Our main theorem describes the Serre spectral
sequence \eqref{eq:Serre-SS} in total degree $\le 10$ and singles out
a distinguished scalar $\varepsilon(k)\in\mathbb{F}_2$ which already
captures the effect of $k$ on the first nontrivial fibre class.

Let $G$ be a compact, connected Lie group, let $X$ be a pointed CW
complex, and let $P\to X$ be a principal $G$--bundle. The \emph{gauge
group} $\mathcal{G}(P)$ is the topological group of $G$--equivariant
bundle automorphisms of $P$ that cover the identity on $X$. When $X$
is finite, Crabb--Sutherland showed that there are only finitely many
homotopy types among the gauge groups $\mathcal{G}(P)$ as $P$ ranges
over all principal $G$--bundles over $X$~\cite{CrabbSutherland}. In
the fundamental case $X=S^4$ and $G$ simply connected and simple,
isomorphism classes of $G$--bundles are classified by
$\pi_4(BG)\cong \mathbb{Z}$, and we write $\mathcal{G}_k(G)$ for the
gauge group of the bundle classified by $k\in \mathbb{Z}$.

Gottlieb identified the classifying space of $\mathcal{G}_k$ with a
component of a mapping space~\cite{GottliebGauge}: specifically,
$B\mathcal{G}_k$ is homotopy equivalent to $\mathrm{Map}_k(S^4,BG)$,
the connected component of the mapping space $\mathrm{Map}(S^4,BG)$
containing the classifying map $k\colon S^4\to BG$.
This identification yields a fundamental evaluation fibration
\[
\Omega^3_0 G \;\longrightarrow\; B\mathcal{G}_k
\xrightarrow{\;\mathrm{ev}\;} BG.
\]
For $G=G_2$, this expresses $B\mathcal{G}_k$ as a twisted
mapping space over the base $B\G$.

The mod~$2$ cohomology of $B\G$ is classically known: Borel showed that
\[
H^*(B\G;\mathbb{F}_2) \cong \mathbb{F}_2[x_4,x_6,x_7]
\]
is a polynomial algebra on generators in degrees $4,6,7$
\cite{BorelCohomology}. The action of the Steenrod algebra $\mathcal{A}$ on
$H^*(B\G)$ can be described explicitly via Wu formulas, and from the
viewpoint of the hit problem one is interested in minimal
generating sets for the unstable $\mathcal{A}$--module $\mathbb{F}_2[x_4,x_6,x_7]$.

We work $2$--locally throughout and write $H^*(-)$ for
$H^*(-;\mathbb{F}_2)$. Consider the Serre spectral sequence of the
evaluation fibration
\begin{equation}\label{eq:Serre-SS}
E_2^{s,t}(k)\;\cong\; H^s(B\G;H^t(\Omega^3_0G_2))
\;\Longrightarrow\; H^{s+t}(B\mathcal{G}_k).
\end{equation}
The $E_2$--term is a module over $H^*(B\G)$ and an unstable module
over the Steenrod algebra. The twisting of this spectral sequence
depends on $k$ through the connecting map
\[
\partial_k\colon G_2 \longrightarrow \Omega^3_0 G_2,
\]
which is the triple adjoint of the Samelson product $\langle
k\cdot i_3,1\rangle$ by work of
Kishimoto--Theriault--Tsutaya~\cite{KishimotoTheriaultTsutayaG2}.
Kameko's calculation of the $2$--primary order of $\langle i_3,1\rangle$
implies that, after localization at $2$, the map $\partial_k$ depends
only on $k\bmod 4$.

The cohomology of the fibre $\Omega^3_0 G$ can in principle be
computed by iterated Eilenberg--Moore spectral sequences applied to
the path-loop fibrations of $BG$ and $G$.
Choi and Yoon conjectured that, for any simply connected finite
$H$--space $X$, the Eilenberg--Moore spectral sequences converging
to the mod $p$ homology of $\Omega^2 X$ and $\Omega^3 X$ collapse at
$E_2$~\cite{ChoiYoonEMSS}. This conjecture was subsequently proved by
Lin~\cite{LinEMSS}; in particular, for any compact simple Lie group
$G$ and any prime $p$, the Eilenberg--Moore spectral sequences computing
$H_*(\Omega^2 G;\mathbb F_p)$ and $H_*(\Omega^3 G;\mathbb F_p)$
collapse at $E_2$. Combined with the classical description of
$H^*(B\G)$, this implies that $H^*(\Omega^3_0G_2)$ is of finite type in
each degree and that its generators occur in bounded degrees. For our
purposes, however, we only need to know the very first nontrivial
cohomology group of $\Omega^3_0G_2$.

Let
\[
M \;=\; H^*(\Omega^3_0G_2;\mathbb{F}_2)
\]
and write $M^{\le N}=\bigoplus_{j\le N}M^j$ for its truncation in
degrees $\le N$. Our main theorem describes the Serre spectral
sequence \eqref{eq:Serre-SS} in total degree $\le 10$ and singles out
a distinguished scalar $\varepsilon(k)\in\mathbb{F}_2$ which already
captures the effect of $k$ on the first nontrivial fibre class.

\begin{equation}\label{eq:Serre-SS}
E_2^{s,t}(k)\;\cong\; H^s(\BG;H^t(\Omega^3_0\G))
\;\Longrightarrow\; H^{s+t}(B\mathcal{G}_k).
\end{equation}

In this paper we begin a complementary study of the mod~$2$
cohomology of the classifying spaces $B\mathcal{G}_k(G_2)$. Our goal
is to understand the structure of $H^*(B\mathcal{G}_k;\Ftwo)$ as an
unstable module over the mod~$2$ Steenrod algebra $\A$ in a low range
of degrees. Using the evaluation fibration
\[
\Omega_0^3 G_2 \longrightarrow B\mathcal{G}_k \xrightarrow{\;\mathrm{ev}\;} BG_2
\]
together with Serre and Eilenberg--Moore spectral sequences, we
study the Serre spectral sequence
\[
H^s(BG_2;H^t(\Omega^3_0G_2)) \Longrightarrow H^{s+t}(B\mathcal{G}_k)
\]
in total degree $\le 10$. A careful analysis of the homotopy groups
of $G_2$ shows that
\[
H^j(\Omega^3_0G_2;\Ftwo)=0\quad\text{for }1\le j\le 4,
\qquad
H^5(\Omega^3_0G_2;\Ftwo)\neq 0,
\]
so the first positive-degree generator of the fibre cohomology occurs
in degree $5$. As a consequence, there is a distinguished class
\[
u_5\in H^5(\Omega^3_0G_2;\Ftwo)
\]
whose only possible Serre differential in total degree $\le 10$ is a
$d_6$--differential
\[
d_6(u_5) = \varepsilon(k)\,x_6
\]
from $u_5$ to the degree-$6$ generator $x_6\in H^6(BG_2;\Ftwo)$, for a
certain scalar $\varepsilon(k)\in\Ftwo$ depending on the bundle
class~$k$.
\end{abstract}

We work $2$--locally throughout and write $H^*(-)$ for
$H^*(-;\Ftwo)$. Consider the Serre spectral sequence of the
evaluation fibration
\begin{equation}\label{eq:Serre-SS}
E_2^{s,t}(k)\;\cong\; H^s(\BG;H^t(\Omega^3_0\G))
\;\Longrightarrow\; H^{s+t}(B\mathcal{G}_k).
\end{equation}
The $E_2$--term is a module over $H^*(\BG)$ and an unstable module
over the Steenrod algebra. The twisting of this spectral sequence
depends on $k$ through the connecting map
\[
\partial_k\colon \G \longrightarrow \Omega^3_0 \G,
\]
which is the triple adjoint of the Samelson product $\langle
k\cdot i_3,1\rangle$ by work of
Kishimoto--Theriault--Tsutaya~\cite{KishimotoTheriaultTsutayaG2}.
Kameko's calculation of the $2$--primary order of $\langle i_3,1\rangle$
implies that, after localization at $2$, the map $\partial_k$ depends
only on $k\bmod 4$.

Let
\[
M \;=\; H^*(\Omega^3_0\G;\Ftwo)
\]
and write $M^{\le N}=\bigoplus_{j\le N}M^j$ for its truncation in
degrees $\le N$. Our main theorem describes the Serre spectral
sequence \eqref{eq:Serre-SS} in total degree $\le 10$ and singles out
a distinguished scalar $\varepsilon(k)\in\Ftwo$ which already
captures the effect of $k$ on the first nontrivial fibre class.

\item In total degree $j\le 10$ the differentials
 \[
 d_r\colon E_r^{0,t}(k)\longrightarrow E_r^{r,t-r+1}(k)
 \]
 with $2\le r\le 5$ vanish on all classes in bidegree $(0,t)$, and
 the only page on which a differential with source in $E_r^{0,5}(k)$
 can be nonzero is $r=6$. After choosing a generator
 $x_6\in H^6(\BG)\cong E_2^{6,0}(k)$ and $u_5\in M^5\cong E_2^{0,5}(k)$
 there is a uniquely determined scalar $\varepsilon(k)\in\Ftwo$ such that
 \[
 d_6(u_5) = \varepsilon(k)\, x_6
 \]
 in $E_6^{6,0}(k)$.

\item Let $U\subset E_2^{*,*}(k)$ denote the $H^*(\BG)$--submodule
 generated by $u_5$. In total degree $j\le 10$ all differentials
 $d_r$ with $2\le r\le 5$ vanish on $U$, and every differential
 \[
 d_6\colon E_6^{s,5}(k)\longrightarrow E_6^{s+6,0}(k)
 \]
 with $s+5\le 10$ satisfies
 \[
 d_6(x\cdot u_5) = x\cdot d_6(u_5)
 \qquad\text{for }x\in H^s(\BG).
 \]
 In particular, within $U$ the entire effect of the Serre
 differentials in total degree $\le 10$ is determined by the single
 scalar $\varepsilon(k)\in\Ftwo$.

\begin{remark}
Theorem~\ref{thm:main} should be viewed as a structural reduction. It
does not claim an explicit formula for $\varepsilon(k)$, nor does it
assert that $\varepsilon(k)$ is nontrivial. Kameko's result shows
that the $2$--local homotopy type of $\mathcal{G}_k(\G)$ depends on
$k\bmod 4$ through the $2$--primary order of the Samelson product
$\langle i_3,1\rangle$ \cite{KishimotoTheriaultTsutayaG2,
KamekoG2gauge}. It is natural to expect that $\varepsilon(k)$ is a
function of this $2$--primary data; however, determining
$\varepsilon(k)$ explicitly requires detailed knowledge of
$H^*(\Omega^3_0\G)$ and of the connecting map $\partial_k$ on mod~$2$
cohomology, and we leave this as an open problem.
\end{remark}

\item[(ii)] Lemma~\ref{lem:full-bidegree-vanish} only analyzes the differentials
with source in the $H^*(BG_2)$--submodule
\[
U \;=\; H^*(BG_2)\cdot u_5 \;\subset\; E_2^{*,*}(k).
\]
We do not claim that $M^j=H^j(\Omega^3_0G_2;\F_2)$ vanishes for $j\ge 6$,
and additional fibre generators in these degrees could in principle
support nontrivial differentials in the Serre spectral sequence.
Such differentials, however, do not change the description of the
possible $d_6$--differentials on $U$ in total degree $\le 10$, which is
all that is needed for Theorem~\ref{thm:main}. A determination of $M$
in higher degrees would require an explicit computation via
Eilenberg--Moore spectral sequences, and lies beyond the scope of the
present paper.
\end{itemize}
\end{remark}

\begin{note}\label{rem:epsilon-zero}
The case $k=0$ admits a particularly simple description. The class
$0\in\pi_4(BG_2)$ corresponds to the trivial principal
$G_2$--bundle $S^4\times G_2\to S^4$, and by Gottlieb's theorem \cite{GottliebGauge}, we have
\[
B\mathcal{G}_0 \;\simeq\; \mathrm{Map}_0(S^4,BG_2),
\]
where $\mathrm{Map}_0(S^4,BG_2)$ denotes the component containing the
constant map. The evaluation map $\mathrm{ev}\colon \mathrm{Map}_0(S^4,BG_2)\to BG_2$
admits a section $s\colon BG_2\to \mathrm{Map}_0(S^4,BG_2)$ defined by sending
$y\in BG_2$ to the constant map at $y$. The existence of this section
implies that the homomorphism
\[
\mathrm{ev}^*\colon H^*(BG_2)\longrightarrow H^*(B\mathcal{G}_0)
\]
is injective. In the Serre spectral sequence for the evaluation fibration
with $k=0$, the class $x_6\in H^6(BG_2)\cong E_2^{6,0}$ must therefore
survive to $E_\infty$ and cannot lie in the image of any differential.
In particular, $x_6$ cannot be hit by $d_6(u_5)$, forcing $d_6(u_5)=0$.
Thus, in our notation, $\varepsilon(0)=0$.
\end{note}

\begin{remark}
Theorem~\ref{thm:main} should be viewed as a structural reduction.  It
does not claim an explicit formula for $\varepsilon(k)$, nor does it
assert that $\varepsilon(k)$ is nontrivial.  Kameko's result shows
that the $2$--local homotopy type of $\mathcal{G}_k(G_2)$ depends on
$k\bmod 4$ through the $2$--primary order of the Samelson product
$\langle i_3,1\rangle$ \cite{KishimotoTheriaultTsutayaG2,
KamekoG2gauge}.  It is natural to expect that $\varepsilon(k)$ is a
function of this $2$--primary data; however, determining
$\varepsilon(k)$ explicitly requires detailed knowledge of
$H^*(\Omega^3_0G_2)$ and of the connecting map $\partial_k$ on mod~$2$
cohomology, and we leave this as an open problem.
\end{remark}

\medskip

The paper is organised as follows.  In Section~\ref{sec:prelim} we
review the necessary background on gauge groups, Samelson products,
the mod~$2$ cohomology of $B\G$ and the Steenrod algebra.  In
Section~\ref{sec:SS-setup} we describe the evaluation fibration and
its Serre spectral sequence, and we recall the relationship between
the connecting map and the Samelson product. Finally, Section~\ref{sec:low}
contains the proof of Theorem~\ref{thm:main}, including a detailed
analysis of the bidegrees for $j\le 10$ and the vanishing of
$H^j(\Omega^3_0G_2)$ for $1\le j\le 4$.

The post-theorem introduction does not give a step-by-step proof sketch, but it does indicate where the proof is carried out and what it uses: Section~\ref{sec:SS-setup} “describe[s] the evaluation fibration and its Serre spectral sequence,” and “recall[s] the relationship between the connecting map and the Samelson product.” Then Section~\ref{sec:low} “contains the proof of Theorem~\ref{thm:main}, including a detailed analysis of the bidegrees for $j\le 10$ and the vanishing of $H^j(\Omega^3_0G_2)$ for $1\le j\le 4$.”

The post-theorem introduction does not give a step-by-step proof sketch, but it does indicate where the proof is carried out and what it uses: Next the authors describe the evaluation fibration and its Serre spectral sequence, and recall the relationship between the connecting map and the Samelson product. Then later they establish the main theorem, including a detailed analysis of the bidegrees for $j\le 10$ and the vanishing of $H^j(\Omega^3_0G_2)$ for $1\le j\le 4$.

[Low-degree structure of $H^*(B\mathcal{G}_k)$]\label{thm:main}
Work $2$--locally and fix an integer $k$.  Consider the Serre spectral
sequence
\begin{equation}\label{eq:Serre-SS}
E_2^{s,t}(k)\;\cong\; H^s(\BG;H^t(\Omega^3_0\G))
\;\Longrightarrow\; H^{s+t}(B\mathcal{G}_k).
\end{equation}
for the fibration
$\Omega^3_0G_2\to B\mathcal{G}_k\to B\G$, and let
$M=H^*(\Omega^3_0G_2;\mathbb{F}_2)$.
\begin{enumerate}[label=\textup{(\roman*)}]
  \item In total degree $j=s+t\le 10$ the $E_2$--term is naturally
  isomorphic, as a bigraded $H^*(B\G)$--module, to
  \[
  E_2^{*,*}(k)^{\le 10} \;\cong\;
  H^*(B\G)\otimes M^{\le 10},
  \]
  with $E_2^{s,t}=H^s(B\G)\otimes M^t$.

  \item Using the low-degree homotopy groups of $G_2$ and the Hurewicz
  theorem, one has
  \[
  M^j = H^j(\Omega^3_0G_2;\mathbb{F}_2)=0\quad\text{for }1\le j\le 4,
  \qquad
  M^5\neq 0.
  \]
  In particular, there exists a nonzero class
  \[
  u_5 \in M^5\cong E_2^{0,5}(k)
  \]
  and $u_5$ is the first positive-degree generator of
  $H^*(\Omega^3_0G_2)$.

  \item In total degree $j\le 10$ the differentials
  \[
  d_r\colon E_r^{0,t}(k)\longrightarrow E_r^{r,t-r+1}(k)
  \]
  with $2\le r\le 5$ vanish on all classes in bidegree $(0,t)$, and
  the only page on which a differential with source in $E_r^{0,5}(k)$
  can be nonzero is $r=6$.  After choosing a generator
  $x_6\in H^6(B\G)\cong E_2^{6,0}(k)$ and $u_5\in M^5\cong E_2^{0,5}(k)$
  there is a uniquely determined scalar $\varepsilon(k)\in\mathbb{F}_2$ such that
  \[
  d_6(u_5) = \varepsilon(k)\, x_6
  \]
  in $E_6^{6,0}(k)$.

  \item Let $U\subset E_2^{*,*}(k)$ denote the $H^*(B\G)$--submodule
  generated by $u_5$.  In total degree $j\le 10$ all differentials
  $d_r$ with $2\le r\le 5$ vanish on $U$, and every differential
  \[
  d_6\colon E_6^{s,5}(k)\longrightarrow E_6^{s+6,0}(k)
  \]
  with $s+5\le 10$ satisfies
  \[
  d_6(x\cdot u_5) = x\cdot d_6(u_5)
  \qquad\text{for }x\in H^s(B\G).
  \]
  In particular, within $U$ the entire effect of the Serre
  differentials in total degree $\le 10$ is determined by the single
  scalar $\varepsilon(k)\in\mathbb{F}_2$.

  \item For each $j\le 10$ the spectral sequence induces a finite
  filtration of $H^j(B\mathcal{G}_k)$ whose associated graded
  contains, as a direct summand, the image of $U$ in
  $E_\infty^{*,*}(k)^j$.  This summand is explicitly computable from
  $H^*(B\G)\otimes M^{\le 10}$ together with $\varepsilon(k)$, while
  the remaining extension term depends only on $M^{\le j}$ and the
  multiplicative structure of the Serre spectral sequence.
\end{enumerate}
In particular, in total degree $j\le 10$ the dependence of
$H^j(B\mathcal{G}_k;\mathbb{F}_2)$ on the bundle class $k$ that is visible
through the first nontrivial fibre class $u_5$ is completely encoded
in the single Serre differential $d_6(u_5)=\varepsilon(k)x_6$.

Question:
Working 2-locally and with mod-2 cohomology, let \(k\in\mathbb Z\), let \(\mathcal G_k=\mathcal G_k(G_2)\) denote the gauge group of the principal \(G_2\)-bundle over \(S^4\) classified by \(k\in \pi_4(BG_2)\cong \mathbb Z\), and consider the evaluation fibration \(\Omega_0^3G_2\to B\mathcal G_k\to BG_2\). Let
\[
E_2^{s,t}(k)\cong H^s\!\left(BG_2;H^t(\Omega_0^3G_2;\mathbb F_2)\right)\Longrightarrow H^{s+t}(B\mathcal G_k;\mathbb F_2)
\]
be its Serre spectral sequence, write \(M=H^*(\Omega_0^3G_2;\mathbb F_2)\), and for \(N\ge 0\) write \(M^{\le N}=\bigoplus_{j\le N}M^j\). Which statement holds for every such integer \(k\)?

In total degree \(j=s+t\le 10\), the \(E_2\)-term is naturally isomorphic as a bigraded \(H^*(BG_2;\mathbb F_2)\)-module to \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\), with \(E_2^{s,t}(k)=H^s(BG_2;\mathbb F_2)\otimes M^t\). Moreover, \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), and \(u_5\) is the first positive-degree generator of \(H^*(\Omega_0^3G_2;\mathbb F_2)\). In total degree \(\le 10\), all differentials \(d_r:E_r^{0,t}(k)\to E_r^{r,t-r+1}(k)\) with \(2\le r\le 5\) vanish on classes in bidegree \((0,t)\), and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\). After choosing generators \(x_6\in H^6(BG_2;\mathbb F_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there is a unique scalar \(\varepsilon(k)\in \mathbb F_2\) such that \(d_6(u_5)=\varepsilon(k)x_6\) in \(E_6^{6,0}(k)\). If \(U\subset E_2^{*,*}(k)\) is the \(H^*(BG_2;\mathbb F_2)\)-submodule generated by \(u_5\), then in total degree \(\le 10\) all \(d_r\) with \(2\le r\le 5\) vanish on \(U\), and every differential \(d_6:E_6^{s,5}(k)\to E_6^{s+6,0}(k)\) with \(s+5\le 10\) satisfies \(d_6(x\cdot u_5)=x\cdot d_6(u_5)\) for \(x\in H^s(BG_2;\mathbb F_2)\). Hence, within \(U\), the entire effect of the Serre differentials in total degree \(\le 10\) is determined by the single scalar \(\varepsilon(k)\). Finally, for each \(j\le 10\), the spectral sequence induces a finite filtration of \(H^j(B\mathcal G_k;\mathbb F_2)\) whose associated graded contains, as a direct summand, the image of \(U\) in the total-degree-\(j\) part of \(E_\infty\); this summand is explicitly computable from \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\) together with \(\varepsilon(k)\), while the remaining extension term depends only on \(M^{\le j}\) and the multiplicative structure of the Serre spectral sequence. Equivalently, in total degree \(j\le 10\), the dependence of \(H^j(B\mathcal G_k;\mathbb F_2)\) on the bundle class \(k\) that is visible through the first nontrivial fibre class \(u_5\) is completely encoded by the single differential \(d_6(u_5)=\varepsilon(k)x_6\).

Choices:
B. In total degree \(j=s+t\le 10\), the \(E_2\)-term is naturally isomorphic as a bigraded \(H^*(BG_2;\mathbb F_2)\)-module to \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\), with \(E_2^{s,t}(k)=H^s(BG_2;\mathbb F_2)\otimes M^t\). Moreover, \(M^j=0\) for \(1\le j\le 5\) and \(M^6\neq 0\), so there exists a nonzero class \(u_6\in M^6\cong E_2^{0,6}(k)\), and \(u_6\) is the first positive-degree generator of \(H^*(\Omega_0^3G_2;\mathbb F_2)\). In total degree \(\le 10\), all differentials \(d_r:E_r^{0,t}(k)\to E_r^{r,t-r+1}(k)\) with \(2\le r\le 5\) vanish on classes in bidegree \((0,t)\), and the only page on which a differential with source in \(E_r^{0,6}(k)\) can be nonzero is \(r=7\). After choosing generators \(x_7\in H^7(BG_2;\mathbb F_2)\cong E_2^{7,0}(k)\) and \(u_6\in M^6\cong E_2^{0,6}(k)\), there is a unique scalar \(\varepsilon(k)\in \mathbb F_2\) such that \(d_7(u_6)=\varepsilon(k)x_7\) in \(E_7^{7,0}(k)\). If \(U\subset E_2^{*,*}(k)\) is the \(H^*(BG_2;\mathbb F_2)\)-submodule generated by \(u_6\), then in total degree \(\le 10\) all \(d_r\) with \(2\le r\le 6\) vanish on \(U\), and every differential \(d_7:E_7^{s,6}(k)\to E_7^{s+7,0}(k)\) with \(s+6\le 10\) satisfies \(d_7(x\cdot u_6)=x\cdot d_7(u_6)\) for \(x\in H^s(BG_2;\mathbb F_2)\). Hence, within \(U\), the entire effect of the Serre differentials in total degree \(\le 10\) is determined by the single scalar \(\varepsilon(k)\).
C. In total degree \(j=s+t\le 10\), the \(E_2\)-term is naturally isomorphic as a bigraded \(H^*(BG_2;\mathbb F_2)\)-module to \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\), with \(E_2^{s,t}(k)=H^s(BG_2;\mathbb F_2)\otimes M^t\). Moreover, \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\). In total degree \(\le 10\), all differentials \(d_r:E_r^{0,t}(k)\to E_r^{r,t-r+1}(k)\) with \(2\le r\le 5\) vanish on classes in bidegree \((0,t)\), and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\). After choosing generators \(x_6\in H^6(BG_2;\mathbb F_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there is a unique scalar \(\varepsilon(k)\in \mathbb F_2\) such that \(d_6(u_5)=\varepsilon(k)x_6\) in \(E_6^{6,0}(k)\).
D. In total degree \(j=s+t\le 10\), the \(E_2\)-term is naturally isomorphic as a bigraded \(H^*(BG_2;\mathbb F_2)\)-module to \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\), with \(E_2^{s,t}(k)=H^s(BG_2;\mathbb F_2)\otimes M^t\). Moreover, \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), and \(u_5\) is the first positive-degree generator of \(H^*(\Omega_0^3G_2;\mathbb F_2)\). In total degree \(\le 10\), all differentials \(d_r:E_r^{0,t}(k)\to E_r^{r,t-r+1}(k)\) with \(2\le r\le 5\) vanish on classes in bidegree \((0,t)\), and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\). After choosing generators \(x_6\in H^6(BG_2;\mathbb F_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there is a unique scalar \(\varepsilon\in \mathbb F_2\), independent of \(k\), such that \(d_6(u_5)=\varepsilon x_6\) in \(E_6^{6,0}(k)\) for every integer \(k\). If \(U\subset E_2^{*,*}(k)\) is the \(H^*(BG_2;\mathbb F_2)\)-submodule generated by \(u_5\), then in total degree \(\le 10\) all \(d_r\) with \(2\le r\le 5\) vanish on \(U\), and every differential \(d_6:E_6^{s,5}(k)\to E_6^{s+6,0}(k)\) with \(s+5\le 10\) satisfies \(d_6(x\cdot u_5)=x\cdot d_6(u_5)\) for \(x\in H^s(BG_2;\mathbb F_2)\). Hence, within \(U\), the entire effect of the Serre differentials in total degree \(\le 10\) is determined by the single scalar \(\varepsilon\), and therefore is independent of the bundle class \(k\).
E. In total degree \(j=s+t\le 10\), the \(E_2\)-term is naturally isomorphic as a bigraded \(H^*(BG_2;\mathbb F_2)\)-module to \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\), with \(E_2^{s,t}(k)=H^s(BG_2;\mathbb F_2)\otimes M^t\). Moreover, \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), and \(u_5\) is the first positive-degree generator of \(H^*(\Omega_0^3G_2;\mathbb F_2)\). In total degree \(\le 10\), all differentials \(d_r:E_r^{0,t}(k)\to E_r^{r,t-r+1}(k)\) with \(2\le r\le 5\) vanish on classes in bidegree \((0,t)\), and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\). After choosing generators \(x_6\in H^6(BG_2;\mathbb F_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there is a uniquely determined scalar \(\varepsilon(k)\in\mathbb F_2\) such that \(d_6(u_5)=\varepsilon(k)x_6\) in \(E_6^{6,0}(k)\). If \(U\subset E_2^{*,*}(k)\) is the \(H^*(BG_2;\mathbb F_2)\)-submodule generated by \(u_5\), then in total degree \(\le 10\) all \(d_r\) with \(2\le r\le 5\) vanish on \(U\), and every differential \(d_6:E_6^{s,5}(k)\to E_6^{s+6,0}(k)\) with \(s+5\le 10\) satisfies \(d_6(x\cdot u_5)=x\cdot d_6(u_5)\) for \(x\in H^s(BG_2;\mathbb F_2)\). Finally, for each \(j\le 10\), the spectral sequence induces a finite filtration of \(H^j(B\mathcal G_k;\mathbb F_2)\) whose associated graded is completely and explicitly determined by \(H^*(BG_2;\mathbb F_2)\otimes M^{\le 10}\) together with \(\varepsilon(k)\); in particular, there are no additional extension terms depending on higher fibre classes or on the multiplicative structure of the Serre spectral sequence.

drops the claims about the submodule U and the associated-graded direct-summand/extension-term conclusion


Question:
Work 2-locally and fix an integer \(k\). Let \(\mathcal G_k\) be the gauge group of the principal \(G_2\)-bundle over \(S^4\) classified by \(k\), and consider the evaluation fibration
\[
\Omega^3_0 G_2 \longrightarrow B\mathcal G_k \longrightarrow BG_2.
\]
Let
\[
E_2^{s,t}(k)\cong H^s\big(BG_2;H^t(\Omega^3_0G_2;\mathbb F_2)\big) \Longrightarrow H^{s+t}(B\mathcal G_k;\mathbb F_2)
\]
be its Serre spectral sequence, and write \(M=H^*(\Omega^3_0G_2;\mathbb F_2)\). Which statement holds for total degree \(j=s+t\le 10\)?

The low-degree part of the spectral sequence satisfies all of the following: (1) as a bigraded \(H^*(BG_2)\)-module,
\[
E_2^{*,*}(k)^{\le 10}\cong H^*(BG_2)\otimes M^{\le 10},\qquad E_2^{s,t}=H^s(BG_2)\otimes M^t;
\]
(2) \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), which is the first positive-degree generator of \(H^*(\Omega^3_0G_2)\); (3) for classes in bidegree \((0,t)\), all differentials \(d_r\) with \(2\le r\le 5\) vanish, and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\); moreover, after choosing generators \(x_6\in H^6(BG_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there exists a unique scalar \(\varepsilon(k)\in\mathbb F_2\) such that
\[
d_6(u_5)=\varepsilon(k)x_6;
\]
(4) if \(U=H^*(BG_2)\cdot u_5\subset E_2^{*,*}(k)\), then for every \(x\in H^s(BG_2)\) with \(s+5\le 10\), all \(d_r\) with \(2\le r\le 5\) vanish on \(U\), and
\[
d_6(x\cdot u_5)=x\cdot d_6(u_5);
\]
thus within \(U\) the entire effect of the Serre differentials in total degree \(\le 10\) is determined by the single scalar \(\varepsilon(k)\); and (5) for each \(j\le 10\), the spectral sequence gives a finite filtration of \(H^j(B\mathcal G_k)\) whose associated graded contains, as a direct summand, the image of \(U\) in \(E_\infty^{*,*}(k)^j\), this summand being explicitly computable from \(H^*(BG_2)\otimes M^{\le 10}\) together with \(\varepsilon(k)\), while the remaining extension term depends only on \(M^{\le j}\) and the multiplicative structure of the Serre spectral sequence. In particular, in degrees \(j\le 10\), the dependence of \(H^j(B\mathcal G_k;\mathbb F_2)\) on \(k\) that is visible through the first nontrivial fibre class \(u_5\) is completely encoded by the single differential \(d_6(u_5)=\varepsilon(k)x_6\).

Choices:
B.
The low-degree part of the spectral sequence satisfies the same formal description except that the first possible nonzero differential from the first fibre generator occurs one page earlier: (1) as a bigraded \(H^*(BG_2)\)-module,
\[
E_2^{*,*}(k)^{\le 10}\cong H^*(BG_2)\otimes M^{\le 10},\qquad E_2^{s,t}=H^s(BG_2)\otimes M^t;
\]
(2) \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\); (3) for classes in bidegree \((0,t)\), all differentials \(d_r\) with \(2\le r\le 4\) vanish, and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=5\); moreover, after choosing generators \(x_6\in H^6(BG_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there exists a unique scalar \(\varepsilon(k)\in\mathbb F_2\) such that
\[
d_5(u_5)=\varepsilon(k)x_6;
\]
(4) if \(U=H^*(BG_2)\cdot u_5\subset E_2^{*,*}(k)\), then for every \(x\in H^s(BG_2)\) with \(s+5\le 10\), all \(d_r\) with \(2\le r\le 4\) vanish on \(U\), and
\[
d_5(x\cdot u_5)=x\cdot d_5(u_5);
\]
thus within \(U\) the entire effect of the Serre differentials in total degree \(\le 10\) is determined by the single scalar \(\varepsilon(k)\).
C.
In total degree \(j=s+t\le 10\), one has
\[
E_2^{*,*}(k)^{\le 10}\cong H^*(BG_2)\otimes M^{\le 10}
\]
as a bigraded \(H^*(BG_2)\)-module, with \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\); hence there is a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), and for classes in bidegree \((0,t)\) all differentials \(d_r\) with \(2\le r\le 5\) vanish, so the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\).
D.
The low-degree part of the spectral sequence satisfies the same vanishing and generator statements, but the resulting scalar is independent of the bundle class: (1) as a bigraded \(H^*(BG_2)\)-module,
\[
E_2^{*,*}(k)^{\le 10}\cong H^*(BG_2)\otimes M^{\le 10},\qquad E_2^{s,t}=H^s(BG_2)\otimes M^t;
\]
(2) \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), which is the first positive-degree generator of \(H^*(\Omega^3_0G_2)\); (3) for classes in bidegree \((0,t)\), all differentials \(d_r\) with \(2\le r\le 5\) vanish, and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\); moreover, after choosing generators \(x_6\in H^6(BG_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there exists a unique scalar \(\varepsilon\in\mathbb F_2\), independent of \(k\), such that
\[
d_6(u_5)=\varepsilon x_6;
\]
(4) if \(U=H^*(BG_2)\cdot u_5\subset E_2^{*,*}(k)\), then for every \(x\in H^s(BG_2)\) with \(s+5\le 10\), all \(d_r\) with \(2\le r\le 5\) vanish on \(U\), and
\[
d_6(x\cdot u_5)=x\cdot d_6(u_5);
\]
thus within \(U\) the entire effect of the Serre differentials in total degree \(\le 10\) is determined by the single universal scalar \(\varepsilon\).
E.
In total degree \(j=s+t\le 10\), the spectral sequence is completely determined by the first fibre class: (1) as a bigraded \(H^*(BG_2)\)-module,
\[
E_2^{*,*}(k)^{\le 10}\cong H^*(BG_2)\otimes M^{\le 10},\qquad E_2^{s,t}=H^s(BG_2)\otimes M^t;
\]
(2) \(M^j=0\) for \(1\le j\le 4\) and \(M^5\neq 0\), so there exists a nonzero class \(u_5\in M^5\cong E_2^{0,5}(k)\), which is the first positive-degree generator of \(H^*(\Omega^3_0G_2)\); (3) for classes in bidegree \((0,t)\), all differentials \(d_r\) with \(2\le r\le 5\) vanish, and the only page on which a differential with source in \(E_r^{0,5}(k)\) can be nonzero is \(r=6\); moreover, after choosing generators \(x_6\in H^6(BG_2)\cong E_2^{6,0}(k)\) and \(u_5\in M^5\cong E_2^{0,5}(k)\), there exists a unique scalar \(\varepsilon(k)\in\mathbb F_2\) such that
\[
d_6(u_5)=\varepsilon(k)x_6.
\]
(4) If \(U=H^*(BG_2)\cdot u_5\subset E_2^{*,*}(k)\), then in total degree \(\le 10\) every differential with source in filtration \(t>0\) vanishes outside \(U\), so the entire associated graded of \(H^j(B\mathcal G_k)\) for \(j\le 10\) is explicitly computable from \(H^*(BG_2)\otimes M^{\le 10}\) together with \(\varepsilon(k)\) alone, with no additional extension term or dependence on the multiplicative structure.

first possible differential page from (0,5)

dropped the conclusions about the submodule U, the direct-summand statement in E_infty, and the explicit dependence of H^j(B\mathcal G_k) on \varepsilon(k)

scope of control of differentials and extension data beyond the submodule U


\begin{thm}\label{thm:Kitaev.vanishing}
If $U,V\in\mathcal{L}^\times$ satisfy
\begin{equation}\label{eqn:Kitaev.class}
(U-\mathbf{1})(V-\mathbf{1})\in\mathcal{L}^1\quad\mathrm{and}\quad (V-\mathbf{1})(U-\mathbf{1})\quad \in\mathcal{L}^1,
\end{equation}
then $\det(UVU^{-1}V^{-1})=1$.
\end{thm}

\label{sec:introduction}
We write $\mathcal{L}\equiv \mathcal{L}(\mathcal{H})$ for the ring of bounded linear operators on a Hilbert space $\mathcal{H}$, and $\mathcal{L}^1\equiv \mathcal{L}^1(\mathcal{H})$ for the ideal of \emph{trace class} operators. The group of invertibles in $\mathcal{L}$ is denoted $\mathcal{L}^\times$, and $\mathbf{1}$ denotes the identity operator on $\mathcal{H}$.

The following Theorem is our main result.

\section{Introduction}\label{sec:introduction}
We write $\mathcal{L}\equiv \mathcal{L}(\mathcal{H})$ for the ring of bounded linear operators on a Hilbert space $\mathcal{H}$, and $\mathcal{L}^1\equiv \mathcal{L}^1(\mathcal{H})$ for the ideal of \emph{trace class} operators. The group of invertibles in $\mathcal{L}$ is denoted $\mathcal{L}^\times$, and $\mathbf{1}$ denotes the identity operator on $\mathcal{H}$.

The Fredholm determinant is conjugation invariant, and is a group homomorphism from the invertible determinant class operators to the non-zero complex numbers. From these algebraic properties alone, it is straightforward to show that $\det(UVU^{-1}V^{-1})=1$ whenever any of $U,V$ or $UV$ is determinant class. Nevertheless, in general, $\det(UVU^{-1}V^{-1})$ can take \emph{any} non-zero value \cite{AV,EF}, and it is closely related to the rich subject of traces of commutators \cite{HH}, as well as $K$-theory \cite{Brown}.
Now, Kitaev's condition \eqref{eqn:Kitaev.class} implies
\begin{equation}\label{eqn:Kitaev.implies.det.class}
\begin{aligned}
UVU^{-1}V^{-1}-\mathbf{1}&=(UV-VU)U^{-1}V^{-1}\\
&=\big((U-\mathbf{1})(V-\mathbf{1})-(V-\mathbf{1})(U-\mathbf{1})\big)U^{-1}V^{-1}\in\mathcal{L}^1,
\end{aligned}
\end{equation}
so that $\det(UVU^{-1}V^{-1})$ is automatically well-defined, but it is not at all obvious that it must be trivial. Furthermore, because Kitaev's condition occurs naturally in geometric and physics problems (see Section \ref{sec:quantized.traces}), the veracity of Theorem \ref{thm:Kitaev.vanishing} is of considerable general interest for applications.

\section{Proof of determinant trivialization theorem}\label{sec:proof}
For each $n\geq 1$, we write $M_n(-)$ for the $n\times n$ matrix ring with entries in $(-)$, and $\GL_n(\mathcal{L})$ for the group of invertibles in $M_n(\mathcal{L})$. We may identify $M_n(\mathcal{L})$ with $\mathcal{L}(\mathcal{H}^{\oplus n})$, and $M_n(\mathcal{L}^1)$ with the ideal $\mathcal{L}^1(\mathcal{H}^{\oplus n})$. Thus, there are Fredholm determinant homomorphisms
\[
{\det}^{(n)}:\{U\in \GL_n(\mathcal{L})\,:\,U-\mathbf{1}^{\oplus n}\in M_n(\mathcal{L}^1)\}\to\mathbb{C}^\times,\qquad n\geq 1,
\]
and they are compatible with $\det=\det^{(1)}$ in the sense that
\begin{equation}\label{eqn.det.stable}
{\det}^{(n)}\begin{pmatrix} U & 0 \\ 0 & \mathbf{1}^{\oplus (n-1)}\end{pmatrix} = {\det}^{(1)}\, U,\qquad n\geq 1.
\end{equation}

By combining the Steinberg relations with the Kitaev condition \eqref{eqn:Kitaev.class}, and exploiting the redundancy Lemma \ref{lem:redundant.elementary}, we can rewrite $\det(UVU^{-1}V^{-1})$ as the determinant of a \emph{single} commutator of \emph{elementary matrices}:
\begin{prop}\label{prop:algebraic.step}
Let $U,V\in\mathcal{L}^\times$ satisfy the Kitaev condition . Then
\begin{equation}\label{eqn:algebraic.simplification}
\det(UVU^{-1}V^{-1})={\det}^{(2)}\big(e_{12}(U-\mathbf{1})e_{21}(V-\mathbf{1})e_{12}(\mathbf{1}-U)e_{21}(\mathbf{1}-V)\big).
\end{equation}
\end{prop}
\begin{proof}
According to the identities \eqref{eqn:mult.commutator.stabalized}--\eqref{eqn:Whitehead.big.UV},
\begin{equation}\label{eqn:det.alg.calc.first.factorization}
\det(UVU^{-1}V^{-1})={\det}^{(3)} \big(d_{12}(U)d_{13}(V)d_{12}(U)^{-1}d_{13}(V)^{-1}\big),
\end{equation}
where
\begin{equation}\label{eqn:for.shuffling}
\begin{aligned}
d_{12}(U) & = e_{21}(U^{-1})e_{12}(\mathbf{1}-U)e_{21}(-\mathbf{1})e_{12}(\mathbf{1}-U^{-1})\\
d_{13}(V) & = e_{31}(V^{-1})e_{13}(\mathbf{1}-V)e_{31}(-\mathbf{1})e_{13}(\mathbf{1}-V^{-1}).
\end{aligned}
\end{equation}
We would like to swap the positions of $d_{12}(U)$ and $d_{13}(V)$ in \eqref{eqn:det.alg.calc.first.factorization}, in order to cancel $d_{12}(U)^{-1}d_{13}(V)^{-1}$. This entails moving the elementary matrix factors of $d_{12}(U)$ past those of $d_{13}(V)$. Because of \eqref{eqn:Steinberg.commutator.123}, we will pick up some commutator terms of the form $e_{ik}(ST)$ or $e_{ki}(-TS)$ along the way.
In general, a complicated collection of commutator terms will be acquired.

Fortunately, for the purposes of computing the determinant \eqref{eqn:det.alg.calc.first.factorization}, Lemma \ref{lem:redundant.elementary} tells us that we can discard any $e_{ik}(ST)$ or $e_{ki}(-TS)$ term with $ST\in\mathcal{L}^1$ or $TS\in \mathcal{L}^1$, since this kind of elementary matrix has trivial determinant. The Kitaev condition \eqref{eqn:Kitaev.class} enters here: Because $(U-\mathbf{1})(V-\mathbf{1})$ is trace class, for $i\neq k$,
\begin{align*}
&{\det}^{(3)}\big(W e_{ij}(U-\mathbf{1})e_{jk}(V-\mathbf{1}) W^\prime\big)\\
&={\det}^{(3)}\big(W e_{ik}((U-\mathbf{1})(V-\mathbf{1}))e_{jk}(V-\mathbf{1})e_{ij}(U-\mathbf{1}) W^\prime\big) & (\mathrm{Eq.}\,\eqref{eqn:Steinberg.commutator.123})\\
&={\det}^{(3)}\big(W e_{jk}(V-\mathbf{1})e_{ij}(U-\mathbf{1}) W^\prime\big) & (\mathrm{Lemma}\;\ref{lem:redundant.elementary})
\end{align*}
holds whenever $W,W^\prime$ are invertible operators such that the above determinants are defined. Similarly with $U$ and $V$ swapped. The above property is conveniently expressed as an extra ``commutativity'' relation,
\[
e_{ij}(U-\mathbf{1})e_{jk}(V-\mathbf{1})\simeq e_{jk}(V-\mathbf{1})e_{ij}(U-\mathbf{1}),\qquad i\neq k,
\]
\emph{where ``$\simeq$'' indicates equality after taking a determinant}.
Kitaev's condition \eqref{eqn:Kitaev.class} also implies
\[
(U-\mathbf{1})V=U-\mathbf{1}=V(U-\mathbf{1}),\qquad
(V-\mathbf{1})U=V-\mathbf{1}=U(V-\mathbf{1})\qquad \mathrm{mod}\;\mathcal{L}^1,
\]
The extra relations made available by Kitaev's condition \eqref{eqn:Kitaev.class} are summarized below:
\begin{equation}\label{eqn:Steinberg.Kitaev.extra.identity.2}
\begin{aligned}
e_{ij}(U-\mathbf{1})e_{jk}(V-\mathbf{1}) &\simeq e_{jk}(V-\mathbf{1})e_{ij}(U-\mathbf{1}), & i\neq k,\\
e_{ij}(V-\mathbf{1})e_{jk}(U-\mathbf{1}) &\simeq e_{jk}(U-\mathbf{1})e_{ij}(V-\mathbf{1}), & i\neq k;\\
e_{ij}((U-\mathbf{1})V)&\simeq e_{ij}(U-\mathbf{1})\simeq e_{ij}(V(U-\mathbf{1})),\\
e_{ij}((V-\mathbf{1})U)&\simeq e_{ij}(V-\mathbf{1})\simeq e_{ij}(U(V-\mathbf{1})).
\end{aligned}
\end{equation}
As Kitaev's condition \eqref{eqn:Kitaev.class} holds with $U$ replaced by $U^{-1}$ and/or $V$ replaced by $V^{-1}$, the same replacements can be made in \eqref{eqn:Steinberg.Kitaev.extra.identity.2}.

Now specialize to the case $R=\mathcal{L}$ and $I=\mathcal{L}^1$. For $U,V\in \mathcal{L}^\times$ with determinant class $UVU^{-1}V^{-1}$, the above general definitions give
\begin{equation}\label{eqn:Brown.observation}
\begin{aligned}
\check{\det}\circ\partial\{u,v\}
&=\check{\det}\big[d_{12}(U)d_{13}(V)d_{12}(U)^{-1}d_{13}(V)^{-1}\big]_{K_1(\mathcal{L},\mathcal{L}^1)}\\
&=\check{\det}\left[\begin{pmatrix}UVU^{-1}V^{-1} & 0 & 0 \\ 0 & \mathbf{1} & 0 \\ 0 & 0 & \mathbf{1}\end{pmatrix}\right]_{K_1(\mathcal{L},\mathcal{L}^1)}\\
&=\det(UVU^{-1}V^{-1}).
\end{aligned}
\end{equation}
This $K$-theoretic nature of $\det(UVU^{-1}V^{-1})$ was first observed by Brown \cite{Brown}.

\subsection{Application to quantized traces}\label{sec:quantized.traces}\label{sec:application.quantized.traces}
Let $A,B\in\mathcal{L}$, and suppose
\begin{equation}\label{eqn:quantized.trace.hypothesis}
[A,B]\in \mathcal{L}^1\;\;\mathrm{and}\;\;(e^A-\mathbf{1})(e^B-\mathbf{1})\in\mathcal{L}^1.
\end{equation}
Then, as in \eqref{eqn:Kitaev.implies.det.class}, $(e^B-\mathbf{1})(e^A-\mathbf{1})$ is trace class as well. Thus, $e^A,e^B$ satisfy Kitaev's condition \eqref{eqn:Kitaev.class}, so Theorem \ref{thm:Kitaev.vanishing} guarantees that $\det(e^Ae^Be^{-A}e^{-B})=1$. By the Pincus--Helton--Howe formula \eqref{eqn:HH.formula}, we learn that the condition \eqref{eqn:quantized.trace.hypothesis} implies the trace quantization,
\[
\frac{1}{2\pi i}\mathrm{Tr}[A,B]\in\mathbb{Z}.
\]

This result was first suggested by A.~Kitaev in his influential paper \cite[pp.~56]{Kitaev}, for the purpose of demonstrating the integrality of certain trace formulae appearing in modern quantum physics (see Section \ref{sec:quantized.traces}). We shall therefore refer to \eqref{eqn:Kitaev.class} as the \emph{Kitaev condition} on an invertible pair $U,V\in \mathcal{L}^\times$.

\medskip
Recall that an operator $T\in\mathcal{L}$ is \emph{trace class} if for an(y) orthonormal basis $\mathcal{B}$, the sum $\sum_{e\in \mathcal{B}}\big\langle e\big||T|e\big\rangle$ is finite. In this case, its \emph{trace}
\[
\mathrm{Tr}(T)=\sum_{e\in \mathcal{B}}\big\langle e\big|Te\big\rangle \in\mathbb{C}
\]
is well-defined, independently of the choice of $\mathcal{B}$. An operator $U\in\mathcal{L}$ is \emph{determinant class} if $U-\mathbf{1}\in \mathcal{L}^1$, in which case, its \emph{(Fredholm) determinant} $\det(U)\in\mathbb{C}$ is well-defined. See \cite[Chapter 3]{Simon} for a detailed treatment. The identity
\[
\det(\exp(T))=\exp(\mathrm{Tr}(T)),\qquad T\in\mathcal{L}^1,
\]
shows that a trivial determinant is intimately related to integrality of a corresponding trace (up to a $2\pi i$ factor).

The Fredholm determinant is conjugation invariant, and is a group homomorphism from the invertible determinant class operators to the non-zero complex numbers. From these algebraic properties alone, it is straightforward to show that $\det(UVU^{-1}V^{-1})=1$ whenever any of $U,V$ or $UV$ is determinant class. Nevertheless, in general, $\det(UVU^{-1}V^{-1})$ can take \emph{any} non-zero value \cite{AV,EF}, and it is closely related to the rich subject of traces of commutators \cite{HH}, as well as $K$-theory \cite{Brown}.
Now, Kitaev's condition \eqref{eqn:Kitaev.class} implies
\begin{equation}\label{eqn:Kitaev.implies.det.class}
\begin{aligned}
UVU^{-1}V^{-1}-\mathbf{1}&=(UV-VU)U^{-1}V^{-1}\\
&=\big((U-\mathbf{1})(V-\mathbf{1})-(V-\mathbf{1})(U-\mathbf{1})\big)U^{-1}V^{-1}\in\mathcal{L}^1,
\end{aligned}
\end{equation}
so that $\det(UVU^{-1}V^{-1})$ is automatically well-defined, but it is not at all obvious that it must be trivial. Furthermore, because Kitaev's condition occurs naturally in geometric and physics problems (see Section \ref{sec:quantized.traces}), the veracity of Theorem \ref{thm:Kitaev.vanishing} is of considerable general interest for applications.

More than fifteen years after Kitaev's paper \cite{Kitaev} appeared, some progress was made in \cite{EF}, where Theorem \ref{thm:Kitaev.vanishing} was proved under the extra assumption that $(U^*-\mathbf{1})(V-\mathbf{1})$ and $(V-\mathbf{1})(U^*-\mathbf{1})$ are also trace class. Borel functional calculus played a key role in \cite{EF}, and it is unclear whether the techniques there can be improved to address the full \emph{adjoint-free} Theorem \ref{thm:Kitaev.vanishing}. Nevertheless, the result of \cite{EF} shows that there are no easy counterexamples to Theorem \ref{thm:Kitaev.vanishing}.

Our approach to Theorem \ref{thm:Kitaev.vanishing} is completely different, and we believe that it gets to the heart of the problem. Our proof is presented in Section \ref{sec:proof}, and comprises two main steps, both of which use the Kitaev condition \eqref{eqn:Kitaev.class} in crucial ways.
\begin{enumerate}
\item An \emph{algebraic} step, Prop.~\ref{prop:algebraic.step}, simplifies $\det(UVU^{-1}V^{-1})$ to $\det(E_1E_2E_1^{-1}E_2^{-1})$, where $E_1,E_2$ are so-called elementary matrices. This is the most novel part of the proof. Experts in algebraic $K$-theory may notice that the abstract concept of Steinberg symbols in $K_2$-theory is guiding our manipulations. We choose to keep the presentation concrete, and defer the $K$-theoretic interpretation to Section \ref{sec:K.perspective}.
\item An \emph{analytic} step computes $\det(E_1E_2E_1^{-1}E_2^{-1})=1$ using a formula of Pincus--Helton--Howe (Eq.~\eqref{eqn:HH.formula}), and Lidskii's trace theorem \cite{Lidskii}.
\end{enumerate}
Finally, the physical context for Theorem \ref{thm:Kitaev.vanishing} is discussed in Section \ref{sec:quantized.traces}.

The proof of Theorem~\ref{thm:Kitaev.vanishing} (given in Section~\ref{sec:proof}) has two main steps, both using the Kitaev condition \eqref{eqn:Kitaev.class} in crucial ways:
\begin{enumerate}
\item An \emph{algebraic} step (Prop.~\ref{prop:algebraic.step}) that “simplifies $\det(UVU^{-1}V^{-1})$ to $\det(E_1E_2E_1^{-1}E_2^{-1})$, where $E_1,E_2$ are so-called elementary matrices.” The text notes that “Steinberg symbols in $K_2$-theory [are] guiding our manipulations,” though the presentation is kept concrete.
\item An \emph{analytic} step that “computes $\det(E_1E_2E_1^{-1}E_2^{-1})=1$ using a formula of Pincus--Helton--Howe (Eq.~\eqref{eqn:HH.formula}), and Lidskii's trace theorem.”
\end{enumerate}
Additionally, the introduction observes that Kitaev's condition implies the commutator is determinant-class via \eqref{eqn:Kitaev.implies.det.class}, so $\det(UVU^{-1}V^{-1})$ is well-defined, though “it is not at all obvious that it must be trivial.”

The proof of \ref{thm:Kitaev.vanishing} (given later) has two main steps, both using the Kitaev condition
\begin{equation}\label{eqn:Kitaev.class}
(U-\mathbf{1})(V-\mathbf{1})\in\mathcal{L}^1\quad\mathrm{and}\quad (V-\mathbf{1})(U-\mathbf{1})\quad \in\mathcal{L}^1,
\end{equation}
in crucial ways:
\begin{enumerate}
\item An \emph{algebraic} step. We first prove the following proposition.
\begin{prop}\label{prop:algebraic.step}
Let $U,V\in\mathcal{L}^\times$ satisfy the Kitaev condition . Then
\begin{equation}\label{eqn:algebraic.simplification}
\det(UVU^{-1}V^{-1})={\det}^{(2)}\big(e_{12}(U-\mathbf{1})e_{21}(V-\mathbf{1})e_{12}(\mathbf{1}-U)e_{21}(\mathbf{1}-V)\big).
\end{equation}
\end{prop}
This “simplifies $\det(UVU^{-1}V^{-1})$ to $\det(E_1E_2E_1^{-1}E_2^{-1})$, where $E_1,E_2$ are so-called elementary matrices.” The text notes that “Steinberg symbols in $K_2$-theory [are] guiding our manipulations,” though the presentation is kept concrete.
\item An \emph{analytic} step that “computes $\det(E_1E_2E_1^{-1}E_2^{-1})=1$ using a formula of Pincus--Helton--Howe,
\begin{equation}\label{eqn:HH.formula}
\det(e^Ae^Be^{-A}e^{-B})=\exp(\mathrm{Tr}[A,B]).
\end{equation}
and Lidskii's trace theorem.”
\end{enumerate}
Additionally, the introduction observes that Kitaev's condition implies the commutator is determinant-class via \eqref{eqn:Kitaev.implies.det.class}, so $\det(UVU^{-1}V^{-1})$ is well-defined, though “it is not at all obvious that it must be trivial.”

Question:
Let mathcal{H} be a Hilbert space, let \(\mathcal{L}(\mathcal{H})\) be the algebra of bounded linear operators on \(\mathcal{H}\), let \(\mathcal{L}^1(\mathcal{H})\) be the ideal of trace-class operators, let \(\mathcal{L}(\mathcal{H})^\times\) denote the invertible bounded operators, and let \(\mathbf{1}\) be the identity operator. Suppose \(U,V\in \mathcal{L}(\mathcal{H})^\times\) satisfy
\[
(U-\mathbf{1})(V-\mathbf{1})\in \mathcal{L}^1(\mathcal{H})
\quad\text{and}\quad
(V-\mathbf{1})(U-\mathbf{1})\in \mathcal{L}^1(\mathcal{H}).
\]
For an invertible operator of the form \(\mathbf{1}+K\) with \(K\in \mathcal{L}^1(\mathcal{H})\), write \(\det\) for its Fredholm determinant. Under these hypotheses, which statement about the multiplicative commutator \(UVU^{-1}V^{-1}\) holds?

The Fredholm determinant of the commutator is trivial: \(\det(UVU^{-1}V^{-1})=1\).

Choices:
B.
The multiplicative commutator is always determinant class under these hypotheses, and its Fredholm determinant is given by the trace formula
\[
\det(UVU^{-1}V^{-1})=\exp\!\big(\operatorname{Tr}((U-\mathbf{1})(V-\mathbf{1})-(V-\mathbf{1})(U-\mathbf{1}))\big).
\]
C.
The multiplicative commutator is determinant class, i.e.
\[
UVU^{-1}V^{-1}-\mathbf{1}\in \mathcal{L}^1(\mathcal{H}).
\]
D.
If \((U-\mathbf{1})(V-\mathbf{1})\in \mathcal{L}^1(\mathcal{H})\), then automatically
\[
\det(UVU^{-1}V^{-1})=1,
\]
without any need to assume \((V-\mathbf{1})(U-\mathbf{1})\in \mathcal{L}^1(\mathcal{H})\).
E.
Under these hypotheses the commutator itself is trace class relative to the identity in the stronger sense that
\[
UVU^{-1}V^{-1}=\mathbf{1},
\]
so in particular its Fredholm determinant equals \(1\).

analytic-step trace formula applied directly to U,V rather than logarithmic elementary-matrix commutator

drop the final trivialization conclusion and keep only determinant-class/well-definedness

two-sided Kitaev condition reduced to one-sided trace-class product hypothesis

replace determinant triviality of the commutator by actual operator triviality


\begin{thm}\label{Thm-Existence}
	For all $n\geq 2$, $0<\alpha<n$ and $\gamma>0$, there exists a positive constant $m_1$ depending on $n,\alpha$ and $\gamma$ such that for $0<m\leq m_1$, the unique minimizer (up to hyberbolic isometries) to the minimization problem \eqref{Prob-mini} with volume $m$ is given by a geodesic ball.
\end{thm}

\label{Subsec-model} The $n$-dimensional hyperbolic space $\mathbb{H}^n$ can be viewed as $\mathbb{H}^n=(\mathbb{R}^n_{+},g)$, where $\mathbb{R}^n_{+}=\{(x_1,x_2,\dots,x_n)\in\mathbb{R}^n:x_n>0\}$ is

Let $n\geq 2$, $0<\alpha<n$ and $\gamma>0$. For any measurable set $F\subset\mathbb{H}^n$, we consider the following functional:
\begin{equation}\label{defn-Functional}
	\mathcal{E}(F)=P(F)+\gamma NL_{\alpha}(F),
\end{equation}
where
\begin{equation}\label{defn-Perimeter}
 P(F)=\sup\left\{\int_{F}\mathrm{div}_g X\,dV_g:X\in C_c^1(\mathbb{H}^n,\mathrm{T}\mathbb{H}^n),|X|_g\leq 1\right\},
 \end{equation}
 denotes the perimeter of $F$ in the sense of  De Giorgi and the second term $NL_{\alpha}(F)$ is defined as
 \begin{equation}\label{defn-doubleint}
 	NL_{\alpha}(F)=\int_{\mathbb{H}^n}\int_{\mathbb{H}^n}{\frac{\chi_F(x)\chi_F(y)}{{d_g(x,y)}^{\alpha}}}\,dV_g(x)dV_g(y),
 \end{equation}
 where $\chi_F$ denotes the characteristic function of $F$. Furthermore,  in both \eqref{defn-Perimeter} and \eqref{defn-doubleint}, $g$ denotes the metric of $\mathbb{H}^n$ and in this paper, all quantities with subscript $g$ indicate that they are considered in $\mathbb{H}^n$. In the following, we investigate the minimization problem
 \begin{equation}\label{Prob-mini}
 	E(m)=\inf_{|F|_g=m}\mathcal{E}(F).
 \end{equation}
The minimization problem \eqref{Prob-mini} of functional \eqref{defn-Functional} has a physical background in Euclidean space $\mathbb{R}^n$ and has been extensively studied. The case where $\alpha=1$ in $\mathbb{R}^3$ is particularly significant, recalling Gamow's liquid drop model for atomic nuclei. In this model, the nucleus is viewed as a region $\Omega\subset\mathbb{R}^3$ densely packed with nucleons (protons and neutrons) at constant density, implying the nucleon count is proportional to the volume of $\Omega$. The perimeter term in the energy functional corresponds to the surface tension binding the nucleus and the second term represents the Coulomb repulsion between protons (here all physical constants have been normalized to 1 for simplicity). Then the minimization problem is to ask that whether stable atomic nuclei at arbitrarily high atomic numbers can exist in the framework of the classical liquid drop model of nuclear matter.

In the case of $\mathbb{R}^n$, the minimization problem of functional \eqref{defn-Functional} is highly nontrivial since by the isoperimetric inequality, the perimeter  term attains its minimum on balls for fixed volume, while the second term attains its maximum on balls by the Riesz's rearrangement inequality (cf. \cite[Theorem 3.7]{Leib-Loss01}). For set $F\subset\mathbb{R}^n$ measurable with volume $|F|=m$, we scale $F$ by $F\to m^{\frac{1}{n}}U$, then $|U|=1$ and using the scaling properties of $\mathbb{R}^n$, we have
\begin{equation}\label{Ex-scale}
	\mathcal{E}(F)=m^{\frac{n-1}{n}}\left(P(U)+\gamma m^{\frac{n+1-\alpha}{n}}NL_{\alpha}(U)\right),
\end{equation}
which suggests that the perimeter term is dominant if volume $m$ is small and the nonlocal term $NL_{\alpha}(\cdot)$ is dominant if volume $m$ is large. Hence, we expect that there exist minimizers for small $m$ and there exist no minimizers for large $m$. Indeed, we define the critical volume $m_{*}$ to be the volume such that the value of the functional $\mathcal{E}(\cdot)$ of the ball with volume $m_{*}$ equals that of two balls with both volume $m_{*}/2$, spaced infinitely far apart, i.e.,
\begin{equation*}
	\mathcal{E}\left(\left(\frac{m_{*}}{b_n}\right)^{\frac{1}{n}}B\right)=2\mathcal{E}\left(\left(\frac{m_{*}}{2 b_n}\right)^{\frac{1}{n}}B\right),
\end{equation*}
where $B$ denotes the unit ball in $\mathbb{R}^n$ and $b_n=|B|$. In the follwing of paper, we also denote $\omega_{n-1}:=|\partial B|$, which is the area of the unit sphere in $\mathbb{R}^n$. Note that by coarea formula, we have $\omega_{n-1}=n b_n$.  A direct calculation shows
\begin{equation}\label{Eq-Cri}
	m_{*}=\left(\frac{2^{\frac{1}{n}}-1}{1-2^{\frac{\alpha-n}{n}}}\frac{\omega_{n-1}}{\gamma NL_{\alpha}(B)}\right)^{\frac{n}{n+1-\alpha}}b_n.
\end{equation}
The significance of $m_{*}$ is that for $m>m_{*}$, the value of the functional $\mathcal{E}(\cdot)$ of two balls with both volume $m/2$, spaced infinitely far apart is less than that of a ball with volume $m$. Moreover, it is conjectured that
\begin{con}[cf. \cite{Choksi-Peletier11}]\label{Con-Cri}
	Given $n\geq 2$, $0<\alpha<n$ and $\gamma>0$, for $m\leq m_{*}$, the ball of volume $m$ uniquely minimizes $\mathcal{E}(\cdot)$ among measurable sets $F\in\mathbb{R}^n$ with $|F|=m$, and for $m>m_{*}$ no minimizer exists.
	\end{con}
So far, Conjecture \ref{Con-Cri} has not been completely solved. Kn$\ddot{\mathrm{u}}$pfer and Muratov \cites{Knupfer-Muratov13,Knupfer-Muratov14} proved the following results:
\begin{itemize}
	\item[(a)] For every $n\geq 2$, $\alpha\in(0,n)$ and $\gamma>0$, there exists a constant $m_{c_1}>0$ depending on $n,\alpha$ and $\gamma$ such that $E(m)$ has a minimizer for every $m\leq m_{c_1}$.
	\item[(b)] For every $n\geq 2$, $\alpha\in(0,2)$ and $\gamma>0$, there exists a constant $m_{c_2}>0$ depending on $n,\alpha$ and $\gamma$ such that $E(m)$ has no minimizer for every $m>m_{c_2}$.
	\item[(c)] For fixed $\gamma>0$, if $n=2$ and $\alpha>0$ is suffciently small, then $m_{c_1}=m_{c_2}=m_{*}$, and balls are unique minimizers for $E(m)$ when $m\leq m_{*}$.
	\item[(d)] For fixed $\gamma>0$, if $n=2$ and $\alpha<2$, or if $3\leq n\leq 7$ and $\alpha<n-1$, there exists $0<m_{c_0}\leq m_{c_1}$, depending on $n,\alpha$ and $\gamma$ such that balls are unique minimizers for $E(m)$ with $m<m_{c_0}$.
\end{itemize}

Since then, much progress has been made on the basis of these results. Julin \cite{Julin14} extended the result of (d) to all $n\geq 3$ and $\alpha=n-2$. Bonacini and Cristoferi \cite{Bonacini-Cristoferi14} extended the results of (c) and (d) to all $n\geq 2$. Figalli, Fusco, Maggi, Millot and Mirini \cite{Figalli15} extended the result of (d) to to all $n\geq 2$ and $\alpha\in(0,n)$. Frank and Nam \cite{Frank-Nam21} extended the result of (a) to $m_{c_1}=m_{*}$, they also showed that (b) holds for $0<\alpha<n$ and $\alpha\leq 2$. There are also some quantitative estimates for $\alpha$, $m_{c_0}$ and $m_{c_2}$. In the case of $n=2$, Muratov and Zaleski \cite{Muratov15} showed that the result of (c) holds for $\alpha\leq 0.034$.  In the case of $n=3$ and $\alpha=1$, Frank, Killip and Nam \cite{Frank-Killp-Nam16} proved that $m_{c_2}\leq 2.38m_{*}$, while Chodosh and Ruohoniemi \cite{Chodosh-Ruohomiemi25} showed that $m_{c_0}\geq 0.28m_{*}$. We also refer readers to \cites{Alama21,Carazzato23,Carazzato25,Choksi17,Choksi22,Frank19,Frank-Lieb15,Goldman15,Goldman25,Muratov18,Novaga21,Novaga22,Pascale25} for some other interesting results on the nonlocal isoperimetric problem.

Now we turn back to the case of $\mathbb{H}^n$. In this situation, the perimeter term also attains its minimum on geodesic balls with fixed volume by the isoperimetric inequality, and the second term attains its maximum on geodesic balls with fixed volume by the Symmetrization Lemma (cf. \cite[Page 225]{Beckner93}). Hence the minimization problem \eqref{Prob-mini} in $\mathbb{H}^n$ is also highly nontrivial and we expect that there are similar results in $\mathbb{H}^n$ compared with $\mathbb{R}^n$.

Before stating our main results, we make some measure-theoretic notations. We say that a function $u\in L^1(\mathbb{H}^n)$ has bounded variation, i.e. $u\in BV(\mathbb{H}^n)$, if there holds
\begin{equation*}
	\int_{\mathbb{H}^n}{|\nabla u|_g}\,dV_g:=\sup\left\{\int_{\mathbb{H}^n}u\,\mathrm{div}_g X\,dV_g: X\in C_c^1(\mathbb{H}^n,\mathrm{T}\mathbb{H}^n), |X|_g\leq 1\right\}<\infty.
\end{equation*}
For any measurable set $F\subset\mathbb{H}^n$, we say that $F$ has finite perimeter if its characteristic function $\chi_F\in BV(\mathbb{H}^n)$ and then its perimeter can be denoted as $P(F):=\int_{\mathbb{H}^n}{|\nabla\chi_F|}\,dV_g$. The $k$-dimensional Hausdorff measure with $k\in[0,n]$ is denoted as $H^k(F)$.

For any $H^n$-measurable set $F$, its upper and lower densities at a point $x\in\mathbb{H}^n$ are defined respectively by
\begin{equation}\label{Defn-uppden}
	\overline{D}(F,x):=\limsup_{r\to 0}\frac{|F\cap B_r(x)|_g}{|B_r(x)|_g},\quad  \underline{D}(F,x):=\liminf_{r\to 0}\frac{|F\cap B_r(x)|_g}{|B_r(x)|_g},
\end{equation}
where $B_r(x)\subset\mathbb{H}^n$ is the open geodesic ball with center $x\in\mathbb{H}^n$ and radius $r$. If the upper and lower densities are equal, their common value will be called the density of $F$ at $x$, and it will be denoted by $D(F,x)$.

The essential interior $\mathring{F}^M$ of $F$ is then defined as the set of all $x\in\mathbb{H}^n$ for which $D(F,x)=1$, while the essential closure $\bar{F}^{M}$ of $F$ is defined as the sets of all points where $\overline{D}(F,x)>0$. The essential boundary $\partial^M F$ of F is defined as the set of points where $\overline{D}(F,x)>0$ and $\overline{D}(\mathbb{H}^n\setminus F,x)>0$.  For any two sets $A, B\subset\mathbb{H}^n$, we denote $A\Delta B:=(A\setminus B)\cup (B\setminus A)$ as the symmetric difference of $A$ and $B$, then by the Lebesgue–Besicovitch Differentiation Theorem (cf. \cite[Theorem 1.32]{Evans15}), we have
\begin{equation*}
	|F\Delta\mathring{F}^M|_g=|F\Delta\bar{F}^{M}|_g=0.
\end{equation*}
We also have that (cf. \cite[Page 46]{Ambrosio01})
\begin{equation}\label{Eq-Essenb}
	\partial^M F=\mathbb{H}^n\setminus(\mathring{F}^M\cup(\mathring{F}^c)^M).
\end{equation}

The reduced boundary $\partial^{*}F$ of a set of finite perimeter F is defined as a set of points $x\in\partial^M F$ such that the measure-theoretic normal exists at $x$, i.e., if the following limit exists:
\begin{equation*}
	\nu_F(x):=\lim_{r\to 0}\frac{\int_{B_r(x)}{\nabla{\chi_{F}}\,dV_g}}{\int_{B_r(x)}{|\nabla{\chi_{F}}|_g\,dV_g}}\quad \text{and}\quad |\nu_F(x)|_g=1,
\end{equation*}
where $\nabla\chi_F$ is the vector-valued Radon measure associated with the distributional derivative of $\chi_F$ and $|\nabla{\chi_{F}}|_g$ coincides with the $H^{n-1}$ measure restricted to $\partial^M F$. If $F$ is a set of finite perimeter, then by De Giorgi’s structure theorem (cf. \cite[Theorem 3.59]{Ambrosio-Fusco-Pallara00}, \cite[Theorem 15.9]{Maggi-12}), we have $P(F)=H^{n-1}(\partial^{*}F)$. Also, by a result of Federer (cf. \cite[Theorem 3.61]{Ambrosio-Fusco-Pallara00}), we have $\partial^{*} F\subset\partial^M F$ and $H^{n-1}(\partial^M F\setminus\partial^{*} F)$=0.
\begin{rem}
	For any set $A\subset\mathbb{H}^n$, we can define the relative perimeter of $F$ in $A$ as $P(F;A):=|\nabla{\chi_{F}}|(A)=H^{n-1}(\partial^{*}F\cap A)$. We say that $F$ is a set of locally finite perimeter, if for every compact set $K\subset\mathbb{H}^n$, we have $P(F;K)<\infty$.
\end{rem}

We say that a set $F$ of finite perimeter is essentially bounded, if its essential closure $\bar{F}^{M}$ is bounded. Also, we say $F$ is decomposable, if there exists a partition $(A,B)$ of $F$ with both $|A|_g>0$ and $|B|_g>0$, such that $P(F)=P(A)+P(B)$. Otherwise, we say that $F$ is indecomposable.
\begin{rem}
	All the measure-theoretic notations mentioned above suit the case of $\mathbb{R}^n$ if the corresponding quantities are considered in $\mathbb{R}^n$.
\end{rem}

Our first result says that the functional $\mathcal{E}(\cdot)$ admits geodesic ball as the unique minimizer (up to hyperbolic isometries) when the volume $m$ is small.

In the case of $\mathbb{R}^n$, the minimization problem of functional \eqref{defn-Functional} is highly nontrivial since by the isoperimetric inequality, the perimeter term attains its minimum on balls for fixed volume, while the second term attains its maximum on balls by the Riesz's rearrangement inequality (cf. \cite[Theorem 3.7]{Leib-Loss01}). For set $F\subset\mathbb{R}^n$ measurable with volume $|F|=m$, we scale $F$ by $F\to m^{\frac{1}{n}}U$, then $|U|=1$ and using the scaling properties of $\mathbb{R}^n$, we have
\begin{equation}\label{Ex-scale}
 \mathcal{E}(F)=m^{\frac{n-1}{n}}\left(P(U)+\gamma m^{\frac{n+1-\alpha}{n}}NL_{\alpha}(U)\right),
\end{equation}
which suggests that the perimeter term is dominant if volume $m$ is small and the nonlocal term $NL_{\alpha}(\cdot)$ is dominant if volume $m$ is large. Hence, we expect that there exist minimizers for small $m$ and there exist no minimizers for large $m$. Indeed, we define the critical volume $m_{*}$ to be the volume such that the value of the functional $\mathcal{E}(\cdot)$ of the ball with volume $m_{*}$ equals that of two balls with both volume $m_{*}/2$, spaced infinitely far apart, i.e.,
\begin{equation*}
 \mathcal{E}\left(\left(\frac{m_{*}}{b_n}\right)^{\frac{1}{n}}B\right)=2\mathcal{E}\left(\left(\frac{m_{*}}{2 b_n}\right)^{\frac{1}{n}}B\right),
\end{equation*}
where $B$ denotes the unit ball in $\mathbb{R}^n$ and $b_n=|B|$. In the follwing of paper, we also denote $\omega_{n-1}:=|\partial B|$, which is the area of the unit sphere in $\mathbb{R}^n$. Note that by coarea formula, we have $\omega_{n-1}=n b_n$. A direct calculation shows
\begin{equation}\label{Eq-Cri}
 m_{*}=\left(\frac{2^{\frac{1}{n}}-1}{1-2^{\frac{\alpha-n}{n}}}\frac{\omega_{n-1}}{\gamma NL_{\alpha}(B)}\right)^{\frac{n}{n+1-\alpha}}b_n.
\end{equation}
The significance of $m_{*}$ is that for $m>m_{*}$, the value of the functional $\mathcal{E}(\cdot)$ of two balls with both volume $m/2$, spaced infinitely far apart is less than that of a ball with volume $m$. Moreover, it is conjectured that
\begin{con}[cf. \cite{Choksi-Peletier11}]\label{Con-Cri}
 Given $n\geq 2$, $0<\alpha<n$ and $\gamma>0$, for $m\leq m_{*}$, the ball of volume $m$ uniquely minimizes $\mathcal{E}(\cdot)$ among measurable sets $F\in\mathbb{R}^n$ with $|F|=m$, and for $m>m_{*}$ no minimizer exists.
 \end{con}
So far, Conjecture \ref{Con-Cri} has not been completely solved. Kn$\ddot{\mathrm{u}}$pfer and Muratov \cites{Knupfer-Muratov13,Knupfer-Muratov14} proved the following results:
\begin{itemize}
 \item[(a)] For every $n\geq 2$, $\alpha\in(0,n)$ and $\gamma>0$, there exists a constant $m_{c_1}>0$ depending on $n,\alpha$ and $\gamma$ such that $E(m)$ has a minimizer for every $m\leq m_{c_1}$.
 \item[(b)] For every $n\geq 2$, $\alpha\in(0,2)$ and $\gamma>0$, there exists a constant $m_{c_2}>0$ depending on $n,\alpha$ and $\gamma$ such that $E(m)$ has no minimizer for every $m>m_{c_2}$.
 \item[(c)] For fixed $\gamma>0$, if $n=2$ and $\alpha>0$ is suffciently small, then $m_{c_1}=m_{c_2}=m_{*}$, and balls are unique minimizers for $E(m)$ when $m\leq m_{*}$.
 \item[(d)] For fixed $\gamma>0$, if $n=2$ and $\alpha<2$, or if $3\leq n\leq 7$ and $\alpha<n-1$, there exists $0<m_{c_0}\leq m_{c_1}$, depending on $n,\alpha$ and $\gamma$ such that balls are unique minimizers for $E(m)$ with $m<m_{c_0}$.
\end{itemize}

Since then, much progress has been made on the basis of these results. Julin \cite{Julin14} extended the result of (d) to all $n\geq 3$ and $\alpha=n-2$. Bonacini and Cristoferi \cite{Bonacini-Cristoferi14} extended the results of (c) and (d) to all $n\geq 2$. Figalli, Fusco, Maggi, Millot and Mirini \cite{Figalli15} extended the result of (d) to to all $n\geq 2$ and $\alpha\in(0,n)$. Frank and Nam \cite{Frank-Nam21} extended the result of (a) to $m_{c_1}=m_{*}$, they also showed that (b) holds for $0<\alpha<n$ and $\alpha\leq 2$. There are also some quantitative estimates for $\alpha$, $m_{c_0}$ and $m_{c_2}$. In the case of $n=2$, Muratov and Zaleski \cite{Muratov15} showed that the result of (c) holds for $\alpha\leq 0.034$. In the case of $n=3$ and $\alpha=1$, Frank, Killip and Nam \cite{Frank-Killp-Nam16} proved that $m_{c_2}\leq 2.38m_{*}$, while Chodosh and Ruohoniemi \cite{Chodosh-Ruohomiemi25} showed that $m_{c_0}\geq 0.28m_{*}$. We also refer readers to \cites{Alama21,Carazzato23,Carazzato25,Choksi17,Choksi22,Frank19,Frank-Lieb15,Goldman15,Goldman25,Muratov18,Novaga21,Novaga22,Pascale25} for some other interesting results on the nonlocal isoperimetric problem.

Now we turn back to the case of $\mathbb{H}^n$. In this situation, the perimeter term also attains its minimum on geodesic balls with fixed volume by the isoperimetric inequality, and the second term attains its maximum on geodesic balls with fixed volume by the Symmetrization Lemma (cf. \cite[Page 225]{Beckner93}). Hence the minimization problem \eqref{Prob-mini} in $\mathbb{H}^n$ is also highly nontrivial and we expect that there are similar results in $\mathbb{H}^n$ compared with $\mathbb{R}^n$.

We say that a set $F$ of finite perimeter is essentially bounded, if its essential closure $\bar{F}^{M}$ is bounded. Also, we say $F$ is decomposable, if there exists a partition $(A,B)$ of $F$ with both $|A|_g>0$ and $|B|_g>0$, such that $P(F)=P(A)+P(B)$. Otherwise, we say that $F$ is indecomposable.
\begin{rem}
 All the measure-theoretic notations mentioned above suit the case of $\mathbb{R}^n$ if the corresponding quantities are considered in $\mathbb{R}^n$.
\end{rem}

Our first result says that the functional $\mathcal{E}(\cdot)$ admits geodesic ball as the unique minimizer (up to hyperbolic isometries) when the volume $m$ is small.

However, such transformations are lacking in $\mathbb{H}^n$. Fortunately, after comparing several common models of $\mathbb{H}^n$, we have adopted the $\Phi_{\lambda}$-transformation (see \S\ref{Subsec-model} for details) in the upper half-space model of $\mathbb{H}^n$, which has the advantage that by adjusting the value of parameter $\lambda$, the volume of any set in $\mathbb{H}^n$ can be scaled to a given multiple. By means of the $\Phi_{\lambda}$-transformation, we can establish several fundamental properties of the minimizers, including their regularity, essential boundedness and indecomposability, and we can further deduce the existence of minimizers for all sufficiently small volumes $m\leq m_0$ for some positive $m_0$ depending only on $n,\alpha$ and $\gamma$.

On the other hand, by adopting similar ideas from \cite{Knupfer-Muratov14} and leveraging the $\Phi_{\lambda}$-transformation, we can prove that the functional $\mathcal{E}(\cdot)$ does not admit a minimizer for large volume $m$ with an extra assumption that $\alpha<2$.
\begin{thm}\label{Thm-Nonexistence}
 For all $n\geq 2$, $0<\alpha<2$ and $\gamma>0$, there exists a positive constant $m_2$ depending on $n,\alpha$ and $\gamma$ such that for $m\geq m_2$, the minimization problem \eqref{Prob-mini} does not admit a minimizer with volume $m$.
\end{thm}
\begin{rem}
 We find it difficult to extend Theorem \ref{Thm-Nonexistence} to the case of $\alpha=2$, as in the work of Frank and Nam \cite{Frank-Nam21}, the reason being that we are unsure how to obtain the estimate in \cite[Lemma 7]{Frank-Nam21}.
\end{rem}
\begin{rem}
 Motivated by the classical results in $\mathbb{R}^n$ (cf. \cite[Theorem 2.9]{Bonacini-Cristoferi14} and \cite[Theorem 1.5]{Figalli15}), we intend to calculate the first and second variations of the functional \eqref{defn-Functional} in our subsequent work, so as to provide explicit bounds for the volume within which geodesic balls are local minimizers. As an application, we aim to derive similar quantitative results regarding parameter $m_1$ as those presented in \cite{Chodosh-Ruohomiemi25}.
\end{rem}
The paper is organized as follows: In \S\ref{Sec-pre}, we collect some preliminaries which will be used later in this paper, including the upper-half space model of $\mathbb{H}^n$, some isperimetric-type inequalities in $\mathbb{H}^n$, two notions of quasiminimizer of the perimeter, and Fuglede’s type estimates of $P(E)$ and $NL_{\alpha}(E)$ for a nearly spherical set $E\subset\mathbb{H}^n$. In \S\ref{Sec-Basic es}, we prove some basic properties for the minimizer of the functional $\mathcal{E}(\cdot)$, including the regularity result, the boundedness, the indecomposability and then prove a uniform lower density bound. During this process, we also derive a general criterion on a set of finite perimeter being not a minimizer. In \S\ref{Sec-Existence}, we prove that there exists a minimizer if the volume $m$ is small and it is contained in a geodesic ball with small radius. In \S\ref{Sec-Ball}, we prove that the functional $\mathcal{E}(\cdot)$ admits geodesic ball as unique minimizers (up to hyperbolic isometries) when volumes $m$ are small. In \S\ref{Sec-Nonex}, we firstly prove that both the the minimization value \eqref{Prob-mini} and the diameter of the essential closure of a minimizer can be two-sided controlled by the volume $m$, and then complete the proof that there exist no minimizers for large volumes.

Then we are left to prove that for $m\leq m_0$, the minimization problem \eqref{Prob-mini} admits a minimizer $E$ with volume $m$ which is contained in a geodesic ball of radius $(1+C_3 r^{\frac{1}{2n}})r$. Firstly, we can choose a sequence of sets of finite perimeter $\{F_k\}$ with $|F_k|_g=m$ such that
\begin{equation*}
 \lim_{k\to\infty}\mathcal{E}(F_k)=\inf_{|F|_g=m}{\mathcal{E}(F)}.
\end{equation*}
By the argument above, we can further choose a minimizing sequence $\{G_k\}$, such that up to isometries, $G_k\subset B_{(1+C_3 r^{\frac{1}{2n}})r}(p)$ for some fixed $p\in\mathbb{H}^n$. Then by the lower semicontinuity of perimeter (cf. \cite[Proposition 12.15]{Maggi-12}) and the compactness of perimeter (cf. \cite[Theorem 12.26]{Maggi-12}), there exists a set of finite perimeter $E\subset\mathbb{H}^n$, a fixed point $p\in\mathbb{H}^n$ and a subsequence, which we also denote as $\{G_k\}$ such that
\begin{align}
 G_k\to& E,\quad E\subset B_{(1+C_3 r^{\frac{1}{2n}})r}(p),\\
 P(E)&\leq\liminf_{k\to\infty} P(G_k).\label{Eq-peri}
\end{align}
Since $G_k\to E$, we have $m=|G_k|_g\to|E|_g$ and hence $|E|_g=m$. Furthermore, by Lemma \ref{Lem-NL}, we have
\begin{equation}\label{Eq-NLa}
 \lim_{k\to\infty}{NL_{\alpha}(G_k)}=NL_{\alpha}(E).
\end{equation}
Combining \eqref{Eq-peri} with \eqref{Eq-NLa} gives
\begin{align*}
 \inf_{|F|_g=m}{\mathcal{E}(F)}\leq\mathcal{E}(E)&=P(E)+\gamma NL_{\alpha}(E)\\
 &\leq \liminf_{k\to\infty} P(G_k)+\gamma\lim_{k\to\infty}{NL_{\alpha}(G_k)}\\
 &=\liminf_{k\to\infty}\mathcal{E}(G_k)\\
 &\leq \liminf_{k\to\infty}\mathcal{E}(F_k)=\inf_{|F|_g=m}{\mathcal{E}(F)},
\end{align*}
which implies that $E$ is a minimizer of the functional $\mathcal{E}(\cdot)$ with volume $m$.
\end{proof}
\section{Geodesic ball as the minimizer}\label{Sec-Ball}
In this section, we give the proof of Theorem \ref{Thm-Existence}. By Proposition \ref{Prop-Quasi}, we know that for any given $r_0>0$, the minimizer to the minimization problem \eqref{Prob-mini} is an $(\omega,r_0)$-minimizer for the perimeter with some constant $\omega>0$. Using Theorem \ref{Lem-Min}, we can calculate which quantities $\omega$ precisely depends on provided $m\leq m_0$.
\begin{prop}\label{Prop-Qp}
 Let $n\geq 2$, $\alpha\in(0,n)$ and $\gamma>0$. Assume that $E$ is a minimizer with $|E|_g=|B_r|_g=m$ for $m\leq m_0$, then there exist two constants $\Lambda_1$ and $\Lambda_2$ depending only on $n$, $\alpha$ and $\gamma$ such that
\begin{equation}\label{In-Qp}
 P(E)\leq P(F)+\left(\frac{\Lambda_1}{r}+\Lambda_2\right)|E\Delta F|_g
\end{equation}
for every $F\subset\mathbb{H}^n$. Here $\Lambda_1$ and $\Lambda_2$ can be chosen as
\begin{align}
 \Lambda_1&=\frac{6C_4}{b_n},\label{Defin-Lamb1}\\
 \Lambda_2&=\max\left\{\frac{6C_4}{b_n}+14\gamma c_3,\gamma c_3\left(2C_5^{\frac{2(\alpha+1-n)}{n-1}}+C_5+1\right)\right\},\label{Defin-Lamb2}
\end{align}
where $C_4$ is the constant defined in \eqref{Defn-C_4},
\begin{equation}\label{Defn-C5}
 C_5=\left(\frac{2C_4}{nb_n}\right)^{\frac{n}{n-1}},
\end{equation}
and $c_3$ is the constant defined in Lemma \ref{Lem-Bound} depending on $n$, $\alpha$ and $C_5 m_0$ (we choose $\bar{m}=C_5 m_0$).
\end{prop}
\begin{proof}
 In the following of proof, we always assume that $P(F)\leq P(E)$, since otherwise \eqref{In-Qp} holds trivially. We claim that we can reduce to prove \eqref{In-Qp} in the case that
 \begin{equation}\label{In-twos}
 \frac{1}{2}\leq\frac{|F|_g}{m}\leq C_5.
 \end{equation}
 Indeed, if we compare $E$ with $B_r$, by \eqref{Eq-Min0}, we have
 \begin{equation}\label{Qp1}
 \mathcal{E}(E)=P(E)+\gamma NL_{\alpha}(E)\leq C_4(r^{n-1}+r^n).
 \end{equation}
 Then if $|F|_g<\frac{1}{2}m$, we have
 \begin{equation*}
 |E\Delta F|_g\geq |E|_g-|F|_g>\frac{1}{2}m\geq\frac{b_n}{2} r^n.
 \end{equation*}
 Combined with the definition of $\Lambda_1$ and $\Lambda_2$ gives \eqref{In-Qp}. Also, if $|F|_g>C_5 m\geq C_5 b_n r^n$, then by \eqref{Eq-HEin} and $m\leq m_0$ (which implies $r\leq 1$ by \eqref{Defn-m0}), we have
 \begin{equation*}
 P(F)\geq nb_n^{\frac{1}{n}}|F|_g^{\frac{n-1}{n}}\geq nb_n C_5^{\frac{n-1}{n}}r^{n-1}=2C_4 r^{n-1}\geq P(E),
 \end{equation*}
 which also gives \eqref{In-Qp}.

\begin{rem}
	It remains an open problem to provide  explicit lower bounds on $m_1$.
\end{rem}
\begin{rem}
	Through a more refined analysis of the constants appearing in the estimates, we can further show that if $0<\alpha\leq\bar{\alpha}$ and $0<\gamma\leq\bar{\gamma}$ for two fixed $\bar{\alpha}\in(0,n)$ and $\bar{\gamma}>0$, then the positive constant $m_1$ in Theorem \ref{Thm-Existence} can be chosen to depend only on $n$, $\bar{\alpha}$ and $\bar{\gamma}$.
\end{rem}
The proof of this theorem in $\mathbb{H}^n$ differs significantly from that in $\mathbb{R}^n$. In $\mathbb{R}^n$, scaling transformations possess particularly convenient geometric properties. In particular, they allow one to rescale the volume to an arbitrary prescribed value while preserving the shape of the underlying set, and geodesic balls are mapped to geodesic balls under such transformations. Consequently, the analysis of the shape of minimizers in the small-volume regime can be reduced to the study of the corresponding minimization problem for a fixed reference volume $b_n$, as presented in \cites{Bonacini-Cristoferi14,Knupfer-Muratov13,Knupfer-Muratov14,Figalli15}.

However, such transformations are lacking in $\mathbb{H}^n$. Fortunately, after comparing several common models of $\mathbb{H}^n$, we have adopted the $\Phi_{\lambda}$-transformation (see \S\ref{Subsec-model} for details) in the upper half-space model of $\mathbb{H}^n$, which has the advantage that by adjusting the value of parameter $\lambda$, the volume of any set in $\mathbb{H}^n$ can be scaled to a given multiple. By means of the $\Phi_{\lambda}$-transformation, we can establish several fundamental properties of the minimizers, including their regularity, essential boundedness and indecomposability, and we can further deduce the existence of minimizers for all sufficiently small volumes $m\leq m_0$ for some positive $m_0$ depending only on $n,\alpha$ and $\gamma$.

When proving that geodesic balls are the unique minimizers for small volumes, we encounter a new difficulty. Since our $\Phi_{\lambda}$-transformation cannot map geodesic balls to geodesic balls, the argument for a fixed volume of $b_n$ in $\mathbb{R}^n$ is inapplicable. A key observation is that if the minimizer $E$ satisfies $|E|_g=|B_r|_g=m$ for $m\leq m_0$ and some positive $r$, then $E$ is contained in a geodesic ball of radius $(1+C_3 r^{\frac{1}{2n}})r$ for some uniform constant $C_3$. Combining this observation with a notion of quasiminimizer of the perimeter, we are able to show that $\partial E$ can be viewed as a radial graph over $\mathbb{S}^{n-1}$ with the radial function close to $r$ provided the volume $m$ is small enough. Then by means of proof by contradiction and Fuglede's type estiamtes for the perimeter and the nonlocal term, we are able to prove that for small volumes $m$, geodesic balls are unique minimizers up to hyperbolic isometries.

On the other hand, by adopting similar ideas from \cite{Knupfer-Muratov14} and leveraging the $\Phi_{\lambda}$-transformation, we can prove that the functional $\mathcal{E}(\cdot)$ does not admit a minimizer for large volume $m$ with an extra assumption that $\alpha<2$.
\begin{thm}\label{Thm-Nonexistence}
	For all $n\geq 2$, $0<\alpha<2$ and $\gamma>0$, there exists a positive constant $m_2$ depending on $n,\alpha$ and $\gamma$ such that for $m\geq m_2$, the minimization problem \eqref{Prob-mini} does not admit a minimizer with volume $m$.
\end{thm}
\begin{rem}
	We find it difficult to extend Theorem \ref{Thm-Nonexistence} to the case of $\alpha=2$, as in the work of Frank and Nam \cite{Frank-Nam21}, the reason being that we are unsure how to obtain the estimate in \cite[Lemma 7]{Frank-Nam21}.
\end{rem}
\begin{rem}
	Motivated by the classical results in $\mathbb{R}^n$ (cf. \cite[Theorem 2.9]{Bonacini-Cristoferi14} and \cite[Theorem 1.5]{Figalli15}), we intend to calculate the first and second variations of the functional \eqref{defn-Functional} in our subsequent work, so as to provide explicit bounds for the volume within which geodesic balls are local minimizers. As an application, we aim to derive similar quantitative results regarding parameter $m_1$ as those presented in \cite{Chodosh-Ruohomiemi25}.
\end{rem}
The paper is organized as follows: In \S\ref{Sec-pre}, we collect some preliminaries which will be used later in this paper, including the upper-half space model of $\mathbb{H}^n$, some isperimetric-type inequalities in $\mathbb{H}^n$, two notions of quasiminimizer of the perimeter, and Fuglede’s type estimates of $P(E)$ and $NL_{\alpha}(E)$ for a nearly spherical set $E\subset\mathbb{H}^n$. In \S\ref{Sec-Basic es}, we prove some basic properties for the minimizer of the functional $\mathcal{E}(\cdot)$, including the regularity result, the boundedness, the indecomposability and then prove a uniform lower density bound. During this process, we also derive a general criterion on a set of finite perimeter being not a minimizer. In \S\ref{Sec-Existence}, we prove that there exists a minimizer if the volume $m$ is small and it is contained in a geodesic ball with small radius. In \S\ref{Sec-Ball}, we prove that the functional $\mathcal{E}(\cdot)$ admits geodesic ball as unique minimizers (up to hyperbolic isometries) when volumes $m$ are small. In \S\ref{Sec-Nonex}, we firstly prove that both the the minimization value \eqref{Prob-mini} and the diameter of the essential closure of a minimizer can be two-sided controlled by the volume $m$, and then complete the proof that there exist no minimizers for large volumes.

\begin{ack}
	The research was supported by NSFC Grant No.12471047 and  China Postdoctoral Science Foundation No.2024M751605.
\end{ack}

Compared to $\mathbb{R}^n$, the proof of Theorem~\ref{Thm-Existence} in $\mathbb{H}^n$ “differs significantly” because “such transformations are lacking in $\mathbb{H}^n$.” The authors instead “adopted the $\Phi_{\lambda}$-transformation … in the upper half-space model,” which lets one “by adjusting the value of parameter $\lambda$, [scale] the volume of any set in $\mathbb{H}^n$ … to a given multiple.”

Using the $\Phi_{\lambda}$-transformation, they “establish several fundamental properties of the minimizers, including their regularity, essential boundedness and indecomposability,” and “further deduce the existence of minimizers for all sufficiently small volumes $m\leq m_0$.”

For uniqueness of geodesic balls at small volume, they note a key obstruction: “our $\Phi_{\lambda}$-transformation cannot map geodesic balls to geodesic balls,” so the Euclidean fixed-volume reduction is “inapplicable.” A “key observation” is that if a minimizer $E$ has $|E|_g=|B_r|_g=m\leq m_0$, then “$E$ is contained in a geodesic ball of radius $(1+C_3 r^{\frac{1}{2n}})r$.” Combining this with “a notion of quasiminimizer of the perimeter,” they show “$\partial E$ can be viewed as a radial graph over $\mathbb{S}^{n-1}$ with the radial function close to $r$ provided the volume $m$ is small enough.” Finally, “by means of proof by contradiction and Fuglede's type estiamtes for the perimeter and the nonlocal term,” they conclude that “for small volumes $m$, geodesic balls are unique minimizers up to hyperbolic isometries.”

Compared to $\mathbb{R}^n$, the proof in $\mathbb{H}^n$ “differs significantly” because “such transformations are lacking in $\mathbb{H}^n$.” The authors instead “adopted the $\Phi_{\lambda}$-transformation … in the upper half-space model,” which lets one “by adjusting the value of parameter $\lambda$, [scale] the volume of any set in $\mathbb{H}^n$ … to a given multiple.”

Using the $\Phi_{\lambda}$-transformation, they “establish several fundamental properties of the minimizers, including their regularity, essential boundedness and indecomposability,” and “further deduce the existence of minimizers for all sufficiently small volumes $m\leq m_0$.”

For the uniqueness of geodesic balls at small volume (which is the conclusion in establishing the main theorem), they note a key obstruction: “our $\Phi_{\lambda}$-transformation cannot map geodesic balls to geodesic balls,” so the Euclidean fixed-volume reduction is “inapplicable.” A “key observation” is that if a minimizer $E$ has $|E|_g=|B_r|_g=m\leq m_0$, then “$E$ is contained in a geodesic ball of radius $(1+C_3 r^{\frac{1}{2n}})r$.” Combining this with “a notion of quasiminimizer of the perimeter,” they show “$\partial E$ can be viewed as a radial graph over $\mathbb{S}^{n-1}$ with the radial function close to $r$ provided the volume $m$ is small enough.” Finally, “by means of proof by contradiction and Fuglede's type estiamtes for the perimeter and the nonlocal term,” they conclude that “for small volumes $m$, geodesic balls are unique minimizers up to hyperbolic isometries.”

\label{Thm-Existence}
	For all $n\geq 2$, $0<\alpha<n$ and $\gamma>0$, there exists a positive constant $m_1$ depending on $n,\alpha$ and $\gamma$ such that for $0<m\leq m_1$, the unique minimizer (up to hyberbolic isometries) to the minimization problem
\begin{equation}\label{Prob-mini}
 	E(m)=\inf_{|F|_g=m}\mathcal{E}(F).
 \end{equation}
 with volume $m$ is given by a geodesic ball.,

Question:
Let \(\mathbb H^n\) be hyperbolic space, let \(|F|_g\) denote the hyperbolic volume of a measurable set \(F\subset \mathbb H^n\), and let \(\mathcal E(F)=P(F)+\gamma NL_{\alpha}(F)\), where \(P(F)\) is the hyperbolic perimeter and \(NL_{\alpha}(F)\) is the nonlocal term. For \(m>0\), define
\[
E(m)=\inf\{\mathcal E(F): F\subset \mathbb H^n,\ |F|_g=m\}.
\]
Fix \(n\ge 2\), \(0<\alpha<n\), and \(\gamma>0\). Which statement holds about minimizers of \(E(m)\) for small prescribed volume?

There exists a constant \(m_1>0\), depending only on \(n\), \(\alpha\), and \(\gamma\), such that for every \(m\) with \(0<m\le m_1\), the minimization problem \(E(m)\) has a unique minimizer among measurable subsets of \(\mathbb H^n\) with \(|F|_g=m\), up to hyperbolic isometries, and this minimizer is a geodesic ball.

Choices:
B. There exists a constant \(m_1>0\), depending only on \(n\), \(\alpha\), and \(\gamma\), such that for every \(m\) with \(0<m\le m_1\), the minimization problem \(E(m)\) has a unique minimizer among measurable subsets of \(\mathbb H^n\) with \(|F|_g=m\), up to hyperbolic isometries, and this minimizer is a horoball.
C. There exists a constant \(m_1>0\), depending only on \(n\), \(\alpha\), and \(\gamma\), such that for every \(m\) with \(0<m\le m_1\), the minimization problem \(E(m)\) has a minimizer among measurable subsets of \(\mathbb H^n\) with \(|F|_g=m\).
D. For every prescribed volume \(m>0\), the minimization problem \(E(m)\) has a unique minimizer among measurable subsets of \(\mathbb H^n\) with \(|F|_g=m\), up to hyperbolic isometries, and this minimizer is a geodesic ball.
E. There exists a constant \(m_1>0\), depending only on \(n\) and \(\alpha\), such that for every \(m\) with \(0<m\le m_1\), the minimization problem \(E(m)\) has a unique minimizer among measurable subsets of \(\mathbb H^n\) with \(|F|_g=m\), up to hyperbolic isometries, and this minimizer is a geodesic ball.

radial-graph-over-sphere implies geodesic-ball-type conclusion, not horoball

dropped uniqueness and geodesic-ball identification

dependence of threshold constant on \(\gamma\)