Chen et al. (Nature Communications 13, 7499, 2022) demonstrated that a deep reinforcement learning agent trained directly on real scanning tunneling microscope data can laterally manipulate Ag adatoms on Ag(111) with atomic precision (0.089 nm mean placement error after roughly 2000 training episodes), manipulate Co adatoms on Ag(111) in two distinct parameter regimes, and — combined with path planning — autonomously assemble a 42-atom kagome lattice. Five interlinked open issues define the gap between this single-system prototype and a general autonomous atomic-fabrication capability: (i) systematic transferability of the training procedure across substrate/adsorbate combinations with qualitatively different adsorption-site energetics and manipulation mechanisms, and the retraining budgets required per new system; (ii) multi-atom assembly path planning with quantified efficiency gains over non-optimal rapidly-exploring-random-tree planning; (iii) reliable detection of adatom movement from tunneling-current traces beyond the roughly 80% accuracy baseline, and the formal effect of scan reduction on training bias and ultimate placement precision; (iv) fast adaptation to spontaneous tip-apex changes without full retraining; and (v) deriving the learned policy structure — the preference for high bias and conductance and the approximately linear tip-end overshoot — from a microscopic tip–adatom interaction model.

Background

Scanning tunneling microscopy (STM) images and moves individual atoms by bringing an atomically sharp metal tip within a few Ångström of a surface and measuring the quantum-mechanical tunneling current between the two. Since Eigler and Schweizer (1990) positioned single xenon atoms one by one, lateral manipulation — dragging an adsorbed atom (adatom) across a surface in the tip's force field while setting the sample bias voltage and the tip–surface tunneling conductance — has been the standard route to artificial atomic structures such as quantum corrals, designer spin lattices, and single-atom circuitry. The obstacle is control: the tip–adatom interaction is a combination of van der Waals, electrostatic, and chemical forces whose strength depends on the atomic structure of the tip apex, which is usually unknown and changes spontaneously (for example after an accidental crash into the surface). As a result, usable manipulation parameters must be found empirically for every tip–adsorbate–substrate combination, and they drift with the tip state. On the close-packed Ag(111) surface, adatoms rest preferentially on the hollow fcc and hcp adsorption sites (the two threefold-coordinated hollow site types of a (111) metal surface, named after the face-centered-cubic and hexagonal-close-packed stacking of the layers beneath), and the surface lattice constant is a=0.288a = 0.288 nm.

Automation preceded machine learning: an autonomous assembler (Celotta et al., 2014) built atomically perfect structures from Co adatoms on Cu(111) using fixed, human-tuned manipulation parameters, which fail when the tip state drifts; reinforcement learning (RL) — training an agent, by trial and error, to choose actions that maximize accumulated reward — was first applied to STM tip preparation and discrete-action nanofabrication (Leinen et al., 2020). The source paper of this candidate (Chen et al., 2022) took the decisive step to continuous-control RL on real hardware. A soft actor–critic agent (SAC, an off-policy deep-RL algorithm that also maximizes the policy's entropy for robustness), augmented with hindsight experience replay (HER, which relabels each failed episode with the goal actually achieved so that every attempt yields a learning signal) and emphasizing-recent-experience sampling (ERE, which oversamples recent interactions to track a changing environment), learned to laterally manipulate Ag adatoms on Ag(111). The agent's state is the two-dimensional target and adatom positions read from STM images; its action is a six-dimensional manipulation command (bias V=5V = 5–1515 mV, tunneling conductance G=3G = 3–66 µA/V, and the tip's start and end positions). After roughly 2000 training episodes (about 6000 physical manipulations, comparable to large manual assembly campaigns) the agent placed atoms with a mean error of 0.089 nm — well below the lattice constant — at a 100% success rate over 100 consecutive episodes. Separate agents were trained for Co adatoms on Ag(111) in a close-proximity regime and in a high-bias regime (1.5–3 V, at much lower conductance), where a different manipulation mechanism operates. Whether the adatom moved is inferred from the tunneling-current trace by a convolutional neural network (CNN, a deep neural network for sequence and image classification) combined with an empirical derivative rule; this classifier was trained on about 10,000 traces from an earlier experiment and reaches roughly 80% test accuracy, and when it reports 'no movement' the confirming STM scan is skipped to save time. Combined with the Hungarian assignment algorithm (which pairs adatoms to target positions at minimal total movement) and a rapidly-exploring random tree path planner (RRT, a sampling-based any-angle motion planner that carries no optimality guarantee), the system autonomously built a 42-atom kagome lattice — a two-dimensional lattice of corner-sharing triangles that hosts flat electronic bands — in about an hour. Spontaneous tip changes degraded performance only temporarily: continued training recovered within a few hundred episodes, outperforming every fixed-parameter baseline.

The paper closes by inspecting what the trained agent actually does: with training it drifts toward higher bias and conductance, and the distance by which the tip path overshoots the target grows linearly with target distance (xend≈1.02xtarget+0.08x_{end} \approx 1.02 x_{target} + 0.08 nm and yend≈1.04ytarget+0.03y_{end} \approx 1.04 y_{target} + 0.03 nm), in contrast to the constant 0.1 nm overshoot of the human-tuned baseline. These policy-level regularities are empirical observations, not consequences of any microscopic model. Recent work on further systems — reinforcement-learning manipulation of molecules on Ag(111), CO-molecule lattice construction on Cu(111), room-temperature AI manipulation on Si(111), and deep-reinforcement-learning-driven on-surface chemistry on Au(111) (refs 5–11) — reports partial progress on these issues but leaves each open. Together with the engineering limitations — one demonstrated substrate/adsorbate family, non-optimal assembly planning, imperfect movement detection, and recovery from tip changes that still requires hundreds of episodes — this separates the demonstrated prototype from a general autonomous atomic-fabrication capability, and defines the problem stated below.

Problem Statement

The problem is to close the gap between the demonstrated single-system prototype of Chen et al. (2022) and a general, physically understood autonomous capability for atomic-precision STM manipulation, by resolving five interlinked open issues. The common objective uniting them is an autonomous manipulation framework that works beyond one known substrate/adsorbate combination, assembles multi-atom structures efficiently, senses adatom motion reliably under reduced scan overhead, survives spontaneous tip-apex changes without full retraining, and has its learned control policy explained by the underlying tip–adatom physics. These five issues are facets of this single objective rather than five independent tasks: a partial result on any element counts as a legitimate partial result toward closing the gap, while the objective itself is met only when the gap is closed. Specifically:

(1) Transferability: establish, systematically and quantitatively, whether and how the reinforcement-learning training procedure transfers across substrate/adsorbate combinations whose adsorption-site energetics and manipulation mechanisms differ qualitatively (for example, metal-on-metal lateral manipulation at close proximity versus high-bias regimes versus molecular adsorbates), including the retraining budgets — the number of real-hardware training episodes — required per new system, with or without transfer from a previously trained policy.

(2) Assembly planning: determine path-planning schemes for multi-atom assembly that improve on planning based on rapidly-exploring random trees (RRT, a sampling-based planner that carries no optimality guarantee) in overall assembly time and total number of manipulations at equal placement precision, and quantify those gains.

(3) Sensing: improve the reliability of adatom-movement detection from tunneling-current traces beyond the ~80% accuracy baseline, and characterize formally how scan reduction (skipping confirmation scans when no movement is detected) biases the training data and affects the ultimately achievable placement precision.

(4) Robustness: achieve fast adaptation to spontaneous tip-apex changes — via meta-learning (training across many tasks so that adaptation to a new task requires few samples), rapid online adaptation, or explicit inference of the tip state — restoring atomic-precision performance without the several hundred continued-training episodes currently required, with the sample-efficiency tradeoffs characterized.

(5) Physical interpretation: derive the structure of the learned manipulation policy from a microscopic model of the tip–adatom interaction — in particular, why the trained agent prefers large bias and conductance, and why the tip-end overshoot scales approximately linearly with target distance — and assess whether such microscopic models can predict, or even replace, learned policies on new systems.

References

  1. I-Ju Chen, Markus Aapro, Abraham Kipnis, Alexander Ilin, Peter Liljeroth, Adam S. Foster, 'Precise atom manipulation through deep reinforcement learning', Nature Communications 13, 7499 (2022), DOI: 10.1038/s41467-022-35149-w, arXiv:2203.06975, https://doi.org/10.1038/s41467-022-35149-w
  2. D. M. Eigler, E. K. Schweizer, 'Positioning single atoms with a scanning tunnelling microscope', Nature 344, 524–526 (1990), DOI: 10.1038/344524a0, https://doi.org/10.1038/344524a0
  3. Robert J. Celotta, Stephen B. Balakirsky, Aaron P. Fein, Frank M. Hess, Gregory M. Rutter, Joseph A. Stroscio, 'Invited Article: Autonomous assembly of atomically perfect nanostructures using a scanning tunneling microscope', Review of Scientific Instruments 85, 121301 (2014), DOI: 10.1063/1.4902536, https://doi.org/10.1063/1.4902536
  4. Philipp Leinen, Malte Esders, Kristof T. Schütt, Christian Wagner, Klaus-Robert Müller, F. Stefan Tautz, 'Autonomous robotic nanofabrication with reinforcement learning', Science Advances 6, eabb6987 (2020), DOI: 10.1126/sciadv.abb6987, https://doi.org/10.1126/sciadv.abb6987
  5. Bernhard Ramsauer, Grant J. Simpson, Johannes J. Cartus, Andreas Jeindl, Victor García-López, James M. Tour, Leonhard Grill, Oliver T. Hofmann, 'Autonomous Single-Molecule Manipulation Based on Reinforcement Learning', The Journal of Physical Chemistry A 127, 2041–2050 (2023), DOI: 10.1021/acs.jpca.2c08696, https://doi.org/10.1021/acs.jpca.2c08696
  6. Bernhard Ramsauer, Johannes J. Cartus, Oliver T. Hofmann, 'MAM-STM: A software for autonomous control of single moieties towards specific surface positions', Computer Physics Communications 303, 109264 (2024), DOI: 10.1016/j.cpc.2024.109264, arXiv:2404.09694, https://doi.org/10.1016/j.cpc.2024.109264
  7. Nian Wu, Markus Aapro, Joakim S. Jestilä, R. Drost, Miguel Martínez García, Tomás Torres, Feifei Xiang, Nan Cao, Zhijie He, G. Bottari, P. Liljeroth, Adam S. Foster, 'Precise Large-Scale Chemical Transformations on Surfaces: Deep Learning Meets Scanning Probe Microscopy with Interpretability', Journal of the American Chemical Society 147, 1240–1250 (2025), DOI: 10.1021/jacs.4c14757, arXiv:2409.20014, https://doi.org/10.1021/jacs.4c14757
  8. Jie Su, Jiali Li, Na Guo, Xinnan Peng, Jun Yin, Jiahao Wang, Pin Lyu, Zhiyao Luo, Koen Mouthaan, Jishan Wu, Chun Zhang, Xiaonan Wang, Jiong Lu, 'Intelligent synthesis of magnetic nanographenes via chemist-intuited atomic robotic probe', Nature Synthesis 3, 466–476 (2024), DOI: 10.1038/s44160-024-00488-7, https://doi.org/10.1038/s44160-024-00488-7
  9. Junya Okuyama, Zhuo Diao, Hayato Yamashita, Masayuki Abe, 'Integrated AI Framework for Room-Temperature Atom Manipulation in Scanning Probe Microscopy', Nano Letters 25, 17771–17777 (2025), DOI: 10.1021/acs.nanolett.5c04982, https://doi.org/10.1021/acs.nanolett.5c04982
  10. Ganesh Narasimha, Mykola Telychko, Wooin Yang, Arthur P. Baddorf, P. Ganesh, An-Ping Li, Rama Vasudevan, 'Automated Construction of Artificial Lattice Structures with Designer Electronic States', ACS Nano 20, 3393–3401 (2026), DOI: 10.1021/acsnano.5c12469, arXiv:2508.02581, https://doi.org/10.1021/acsnano.5c12469
  11. G. Narasimha, R. Bulanadi, J. Chowdhury, R. K. Vasudevan, Y. Liu, 'Self-Driving Scanning Probe Microscopy: From Acceleration to Discovery and Manipulation', Accounts of Chemical Research 59, 2635–2646 (2026), DOI: 10.1021/acs.accounts.6c00273, https://doi.org/10.1021/acs.accounts.6c00273