Benchmark conventions

In brief. Ketboard compares energies only between calculations of the same explicitly defined problem. An entry is a directly evaluated variational energy of a neural quantum state, with its uncertainty, its paper version and its location in the paper. Extrapolations, projections and numbers quoted from other papers are not entries.

Notation follows the conventions page.

An instance is a complete specification

A benchmark instance fixes everything that changes the number:

Hamiltonian and operator convention. The operators ( $\mathbf S$ versus $\boldsymbol\sigma$ , fermionic sign conventions), every coupling with its value and sign, and any constant offset. For spin- $\tfrac12$ bilinear terms the Pauli convention gives energies four times larger than the spin convention.
Lattice, cluster and boundaries. Primitive vectors or an explicit bond list, the number of sites, and periodic, open, cylindrical or twisted boundaries. A $10\times10$ torus and a $10\times10$ cylinder are different instances. Bond counting must be stated: each undirected bond once.
Sector. Total $S^z$ , particle numbers, momentum and point-group representation when the calculation is restricted to one. A projected state is reported as the energy of the projected state, with the group named.
Normalization and units. Energy per site or total, and the coupling that sets the unit.

Each instance has a key such as j1j2-square-10x10-pbc-j2-0p5-sz0 that appears on the board and in the data files. Instances of the same model with different sizes or couplings share the system name and are listed separately.

What is reported

The variational energy $E_\theta=\langle\Psi_\theta|H|\Psi_\theta\rangle/\langle\Psi_\theta|\Psi_\theta\rangle$ , evaluated on the final fixed state with its own Monte Carlo run. Not the last value of the optimization trace.
The statistical uncertainty, with its meaning: one standard error of the mean from a binning analysis, a standard deviation, a confidence interval, or an unspecified reported uncertainty. Every entry must have one; an energy printed without an error bar is not accepted. A bound stated in the text, such as a caption saying that statistical errors are below the last printed digit, counts and is recorded as an upper bound.
The energy variance $\operatorname{Var}_\theta(H)$ , optional, recorded in the same normalization as the energy ( $\operatorname{Var}_\theta(H)/N$ for a per-site energy) when the paper prints it. It is the only intrinsic accuracy measure when $E_0$ is unknown, and with a reference energy $E_\infty$ it gives the V-score $N\operatorname{Var}_\theta(H)/(E-E_\infty)^2$ of Wu et al. (arXiv:2302.04919).
The printed digits exactly as they appear, as a decimal string, with the parenthesized uncertainty preserved in the original token. Only the conversion from total to per-site energy is done by the site.
The number of parameters, the architecture class, the symmetry treatment and the sampling budget, when available.

Eligibility

An entry represents a state $|\Psi_\theta\rangle$ whose amplitude is given by a neural network, possibly multiplied by a determinant, Pfaffian, Jastrow factor or sign prior, and whose energy is the expectation value of $H$ in that state. Lanczos-improved states $(1+\alpha_1H+\dots)|\Psi_\theta\rangle$ with explicitly evaluated energies qualify.

Not eligible: energies extrapolated in the variance, the system size, the bond dimension or the number of Lanczos steps; energies from projection methods (Green's function, diffusion or auxiliary-field Monte Carlo) even when a neural state is the guiding function; non-neural variational states; and comparison values credited to earlier work. Such numbers belong in the benchmark pages as context, clearly marked.

Comparing entries

The variational energy is an upper bound, so a lower energy is a better bound. It is not evidence that one method is better in general: results differ in parameter count, sampling budget, symmetry projection and optimization time, and differences within one or two standard errors are not resolved. Ketboard orders by energy and shows the uncertainty next to it; the reader draws the conclusion.

Two caveats specific to this field. The sample mean of the local energy can lie below $E_0$ by a statistical fluctuation; this is not a violation of the variational principle. And an energy quoted to more digits than its uncertainty supports is recorded as printed, with the uncertainty displayed alongside.

Provenance and review

Every entry links to a specific arXiv version and to a table, figure or equation in it. Bot-proposed entries carry a checksum of the source file; human entries carry the exact versioned source URL. A curator checks eligibility, instance matching and transcription before merging. Review confirms that the number is what the paper says for the problem the instance defines; it is not an independent reproduction.

Adding an instance

Propose the definition in data/benchmarks.json with the result that needs it: the symbolic Hamiltonian in LaTeX, the parameter values, the cluster, the boundaries, the sector, the unit, and a plain-text statement of every convention. Keep the system name free of sizes and couplings; put those in the instance label. A benchmark page in the wiki, linked from the instance, gives the physics context and the non-variational reference values.