Skip to main content
IBM Quantum Platform

Manage a quantum-centric supercomputing workflow with QCSC-Prefect

Usage estimate: 30 seconds of QPU time on an IBM Quantum® Heron r2 processor. This estimate excludes queue time and classical processing. Your runtime might vary.


Learning outcomes

  • Trace a quantum-centric supercomputing (QCSC) workflow in Prefect Cloud, with one task per Qiskit pattern stage.
  • Configure the workflow by role, with run parameters in the Prefect Cloud run form and the high-performance computing (HPC) and quantum processing unit (QPU) settings in Prefect blocks.
  • Resume a failed run without repeating its QPU job, and reuse the samples of one run in a new run.

Prerequisites


Background

A QCSC workflow runs a QPU and an HPC system as one machine. QCSC-Prefect runs QCSC workflows on Prefect and keeps them traceable, configurable, and resumable.

This tutorial demonstrates these features with sample-based quantum diagonalization (SQD) of the N2_2 molecule. The Quantum Resource Management Interface (QRMI) samples the circuit inside a Slurm job. The Fulqrum eigensolver diagonalizes the sampled subspaces in more Slurm jobs. One Prefect Cloud deployment runs both.

QCSC-Prefect also runs large QCSC workloads, as in the following projects.

  • A study of protein-ligand complexes with more than 12,000 atoms used QCSC-Prefect. It sampled quantum-embedding fragments on two 156-qubit IBM Quantum processors, with up to 94 qubits, and diagonalized the fragment subspaces on the Fugaku, Miyabi-G, and ROQUO supercomputers [1].
  • A study of iron-sulfur clusters ran an extended QCSC-Prefect SQD workflow with 36 orbitals (72 qubits) and subspace dimensions up to 3×10103 \times 10^{10} on the Fugaku, Miyabi, and ROQUO supercomputers [2].

The challenge of QCSC workflows

The SQD workflow is typical of QCSC workloads. QCSC-Prefect answers each of its challenges with one feature, as in the following table.

Challenge
QCSC-Prefect feature
It spans several resources. A Slurm job holds the QPU, then many Slurm jobs diagonalize subspaces, round after round.Traceable workflow
It has many parameters at different levels (science, circuit, and cluster), and credentials must stay out of all of them.Configurable parameters
It is long and can fail anywhere, from a walltime limit, a node failure, or a long QPU queue. Without care, a late failure means sampling on the QPU again.Resumable execution

Each step of Qiskit Pattern in QCSC-Prefect step by step ends with a box that shows how QCSC-Prefect traces, configures, and resumes that stage.

Traceable workflow

A Prefect flow is a Python function made of tasks. QCSC-Prefect makes each stage of a Qiskit pattern (Map, Optimize, Execute, and Postprocess) one task. For every run, Prefect Cloud shows the following information:

  • The task graph, with an arrow wherever one task passes its result to another.
  • The state, inputs, logs, and duration of each task, filterable by the stage tags map, optimize, execute, and post-process.
  • The artifacts, which are tables and links that tasks publish, such as the source of the samples or the energy of each subspace.

The workflow host only submits and monitors work. The compute-intensive work runs as Slurm jobs, as in the following image.

Prefect Cloud holds the runs, logs, and artifacts. The workflow host polls Prefect Cloud for runs and reports states and logs. The served deployment on the workflow host submits and monitors the Slurm jobs. The workflow host submits a Slurm job (sbatch --qpu) that runs the QRMI worker, which transpiles the circuit and samples it on the IBM Quantum QPU. It also submits one Fulqrum Slurm job (sbatch) per subspace.
What runs where

Configurable parameters

QCSC-Prefect keeps each kind of setting in its own place, as in the following table.

Kind
Examples
Changes
Where it lives
Run parametersMolecule, shots, subspace dimensionEvery runThe Prefect Cloud run form, grouped by stage
BlocksPartition, walltime, threads, executable paths, QPU resourcePer clusterPrefect blocks, created one time
CredentialsPrefect API key, IBM Quantum API keyRarelyThe Prefect Cloud login on the workflow host, and QRMI (qcsc-qrmi login) on the cluster, both set up in the setup guide

The blocks describe each job in the three layers of the following table.

Block
Specifies
Contents
CommandBlockWhat runsThe executable and its default arguments
ExecutionProfileBlockHow it runsNodes, threads, walltime, launcher, modules, and environment
HPCProfileBlockWhere it runsScheduler, partition, account, and the path of each executable on that system

The flow loads blocks by name when it runs, so a change to a block leaves the flow and its run parameters unchanged.

Resumable execution

Prefect stores the results of the first three stages with the flow run. If a later task fails, Retry on the flow run reuses the stored results and runs only what failed, as in the following image.

When a run fails and you raise the walltime in the block, Retry repeats only the last stage and reuses the results of the first three stages.
Retry a failed run

A new run can also load the samples of an earlier run instead of sampling, because the Execute stage records where its samples are.

SQD at a glance

SQD samples bitstrings from a local unitary cluster Jastrow (LUCJ) circuit built from coupled cluster singles and doubles (CCSD) amplitudes, fixes the bitstrings that noise has corrupted (configuration recovery), and diagonalizes the Hamiltonian in subspaces spanned by the sampled configurations [3]. The SQD tutorial explains the algorithm. In QCSC-Prefect, SQD is the flow sqd, which has the following task graph.

Stage
Task
Input from another task
Mapcompute_molecular_integralsNone
Optimizeget_lucj_paramsIntegrals from Map
Executequantum_sampling, with build_lucj_circuit inside itLUCJ parameters from Optimize
Postprocesssqd_loopIntegrals from Map, and samples from Execute

When a run reuses samples, load_qrmi_result (or load_runtime_job for samples from IBM Quantum Platform) replaces both get_lucj_params and quantum_sampling, so the run has no Optimize task. In random mode (Quantum Source set to random), there is no Optimize or Execute task, and Map feeds sqd_loop directly.

The example is N2_2 at a bond length of 1.1 A˚\AA in the 6-31G basis, with the two 1s core orbitals frozen. This leaves 16 spatial orbitals with five α\alpha and five β\beta electrons, which is a 32-qubit circuit.


Requirements

To run the code cells of this notebook, install the following packages in the notebook's kernel.

  • Qiskit SDK v2.0 or later, with visualization support.
  • Qiskit Runtime v0.40 or later (pip install qiskit-ibm-runtime).
  • SQD Qiskit addon v0.14 or later (pip install qiskit-addon-sqd).
  • ffsim v0.0.75 or later (pip install ffsim).
  • Fulqrum v1.0 or later (see the Fulqrum installation guide).

Step 3 of this notebook samples with Qiskit Runtime, so it needs an IBM Quantum account saved on the computer that runs the notebook, as described in Save your login credentials.

Note

The QCSC-Prefect workflow runs on a Slurm-managed cluster, not in this notebook. Before you run it, complete Set up QCSC-Prefect on a Slurm cluster, which provides the following items.

  • The QCSC-Prefect checkout at $QCSC_PREFECT (~/qcsc-prefect) and a work directory at $QCSC_WORK_DIR (/shared/work/qcsc-prefect), both on storage that the compute nodes share.
  • On the Slurm login node, which is the workflow host, the environment ~/venv/qcsc-host with the QCSC-Prefect packages, and a Prefect Cloud login.
  • The IBM Quantum credentials, stored with qcsc-qrmi login, and the qrmi-runtime block, on a cluster with QRMI and its SPANK plugin.
  • On the compute nodes, the environment ~/venv/qcsc-node with QRMI, its task_runner, and qcsc-prefect-qiskit.

Run the following steps one time. Each step names the node that it runs on.

Install the SQD workflow on the workflow host

Set the variables of the setup guide, activate ~/venv/qcsc-host, and install the SQD workflow and the Fulqrum integration.

# Run in a terminal on the workflow host.
# Optional if your ~/.bashrc sets the two variables
export QCSC_PREFECT="$HOME/qcsc-prefect"
export QCSC_WORK_DIR="/shared/work/qcsc-prefect"

source ~/venv/qcsc-host/bin/activate
cd "$QCSC_PREFECT"
uv pip install --no-deps -e packages/qcsc-prefect-fulqrum
uv pip install -e algorithms/qcsc_workflow_utility
uv pip install -e algorithms/open_loop/sqd
which sqd-create-blocks sqd-deploy

Install the Fulqrum solver on a compute node

Open a shell on a compute node. srun passes the variables of the workflow host shell to the compute node.

# Run in a terminal on the workflow host.
srun -N1 --pty bash

The QRMI worker transpiles the circuit on the compute node with the SQD transpilation hook, which imports only Qiskit. Add the hook to ~/venv/qcsc-node and check the imports. The python command prints nothing when the imports succeed.

# Run in the shell on the compute node.
source ~/venv/qcsc-node/bin/activate
cd "$QCSC_PREFECT"
uv pip install --no-deps -e algorithms/open_loop/sqd
python -c "import qrmi, qcsc_prefect.integrations.qiskit.qrmi.worker, sqd_common.transpile"

Build Fulqrum and qcsc-fulqrum-solve into their own environment, packages/qcsc-prefect-fulqrum/native/fulqrum. The build needs a C++ compiler with OpenMP. FULQRUM_WITH_PRIMME=1 also builds the PRIMME eigensolver, which needs a C compiler, Python headers, pkg-config, and a BLAS and LAPACK library with a .pc file.

# Run in the shell on the compute node.
cd "$QCSC_PREFECT"
FULQRUM_WITH_PRIMME=1 bash packages/qcsc-prefect-fulqrum/native/build_fulqrum.sh
realpath packages/qcsc-prefect-fulqrum/native/fulqrum/bin/qcsc-fulqrum-solve
exit

Record the printed path, because the solver blocks point to it. exit returns to the workflow host. For the build options, see Fulqrum integration.

Configure and create the Prefect blocks on the workflow host

Copy the example configuration.

# Run in a terminal on the workflow host.
cd "$QCSC_PREFECT"
cp algorithms/open_loop/sqd/sqd_blocks.example.toml \
   algorithms/open_loop/sqd/sqd_blocks.toml

Edit the settings at the top of sqd_blocks.toml as in the following table.

Parameter
Example
Description
python_executable"~/venv/qcsc-node/bin/python"The Python of ~/venv/qcsc-node, with the hook from Install the Fulqrum solver on a compute node
profile"slurm"System profile, which is slurm, slurm-gpu, miyabi, miyabi-gpu, fugaku, or a path to your own file
solver"fulqrum"Subspace diagonalizer, which is fulqrum, sbd, or dice
queue"normal"Slurm partition of the solver jobs
project"my-account"Optional Slurm account (#SBATCH --account)
work_dir"/shared/work/qcsc-prefect"The value of QCSC_WORK_DIR, which holds the job directories
fulqrum_executable"~/qcsc-prefect/packages/qcsc-prefect-fulqrum/native/fulqrum/bin/qcsc-fulqrum-solve"The path that you recorded in Install the Fulqrum solver on a compute node
eigensolver"primme"The Fulqrum sparse solver, primme or scipy. The example file sets scipy. Set primme after a FULQRUM_WITH_PRIMME=1 build

Keep the following points in mind when you edit the file.

  • Write the paths out, because the file does not expand environment variables. The ~ character works. Keep every top-level key, such as python_executable, before the first table header.
  • The [qrmi] table needs no change, because the qrmi-runtime block supplies the instance and the QPU. Set resource only to override the backend. The credentials stay in qrmi_config.json, not in this file.
  • The slurm profile holds the scheduler and resource settings of a generic Slurm cluster, and any key in your file overrides it. The commented [sqd_advanced] table sets the sqd-advanced-settings block, the commented [advanced] table sets block names and timeouts, and the table of each solver holds its tuning. To choose the nodes, ranks, and threads of the solver jobs, see Size the SQD diagonalization for your HPC resources.
  • The QRMI job and each solver job have a walltime of 12 hours (#SBATCH --time), and the flow waits up to 13 hours (timeout_seconds = 46800) for each job. To change the walltime, set walltime at the top of the file for the solver jobs, or in the [qrmi] table for the QRMI job, for example walltime = "02:00:00". A shorter walltime can start sooner on a busy cluster.

Create the blocks.

# Run in a terminal on the workflow host.
sqd-create-blocks --config algorithms/open_loop/sqd/sqd_blocks.toml

The command saves the blocks in the following table.

Block
Role
Solver set (cmd-sqd-fulqrum-solver, exec-sqd-fulqrum-slurm, hpc-slurm-sqd-fulqrum)The three layers of the solver jobs
QRMI set (cmd-sqd-qrmi, exec-sqd-qrmi, hpc-slurm-sqd-qrmi)The three layers of the QRMI sampler job, which requests the QPU
qrmi-runtimeA QrmiRuntimeBlock (scheduler = "slurm") with the Slurm settings of the QRMI set and the instance and backend from qcsc-qrmi login. It is the default QRMI Runtime Block of the run form
sqd-fulqrum-solverA FulqrumSolverJob that wraps the solver set and holds the eigensolver settings
sqd-advanced-settingsThe Solver Block of every run (fulqrum_solver_job/sqd-fulqrum-solver), the number of CCSD iterations (CCSD Restarts, 10 by default), reset mitigation, and the transpiler settings

Add --dry-run to print the resolved settings without saving anything. prefect block ls and the Blocks page of Prefect Cloud list the blocks, as in the following image.

The Blocks page of Prefect Cloud lists the nine blocks of the SQD workflow. The page of the exec-sqd-fulqrum-slurm block shows its execution profile, with the nodes, threads, and walltime of the solver jobs.

Deploy the workflow on the workflow host

sqd-deploy serves the flow from the workflow host and picks up the runs that you start in Prefect Cloud, so keep the terminal open while you run the workflow. The following command also writes the output to a new log file under $QCSC_WORK_DIR/logs/deploy at each start, named after the start time in UTC, for example sqd-deploy-20261109T051530Z.log.

# Run in a terminal on the workflow host.
mkdir -p "$QCSC_WORK_DIR/logs/deploy"
sqd-deploy 2>&1 | tee "$QCSC_WORK_DIR/logs/deploy/sqd-deploy-$(date -u +%Y%m%dT%H%M%SZ).log"

When the output shows that the flow sqd is served, the deployment appears in Prefect Cloud under Deployments. Press Ctrl+C to stop serving.


Setup

Import the dependencies.

import ffsim
import matplotlib.pyplot as plt
import numpy as np
import pyscf
from fulqrum.addon_sqd_support import sci_solver_from_integrals
from pyscf import ao2mo, cc, mcscf, scf
from qiskit import ClassicalRegister, QuantumCircuit, QuantumRegister
from qiskit.transpiler import generate_preset_pass_manager
from qiskit_addon_sqd.counts import generate_bit_array_uniform
from qiskit_addon_sqd.fermion import diagonalize_fermionic_hamiltonian
from qiskit_addon_sqd.trim import TrimPolicy
from qiskit_ibm_runtime import QiskitRuntimeService, Sampler

Qiskit Pattern in QCSC-Prefect step by step

The following steps compute the four stages in this notebook with PySCF, ffsim, Qiskit, Qiskit Runtime, and qiskit-addon-sqd, without Prefect. See the SQD tutorial for the details of each computation.

Step 1: Map classical inputs to a quantum problem

The Map stage, the task compute_molecular_integrals in the workflow, turns the molecule into the inputs of all later stages. Restricted Hartree-Fock (RHF) gives the molecular orbitals, and the active-space integrals leave out the two frozen 1s cores. CCSD gives the t2t_2 amplitudes of the ansatz, and its density matrix gives the orbital occupancies that start configuration recovery. CCSD iterates CCSD_MAXITER times, each time from the previous amplitudes, so the result does not depend on the DIIS history of the first iteration. This notebook uses 4 iterations, and the workflow uses 10.

ATOM = "N 0 0 0; N 0 0 1.1"
BASIS = "6-31g"
N_FROZEN = 2
CCSD_MAXITER = 4

mol = pyscf.gto.M(atom=ATOM, basis=BASIS, symmetry=False, verbose=0)
mf = scf.RHF(mol).run()
norb = mol.nao_nr() - N_FROZEN
nelec = (mol.nelec[0] - N_FROZEN, mol.nelec[1] - N_FROZEN)

# Active-space Hamiltonian with the 1s cores frozen
cas = mcscf.CASCI(mf, ncas=norb, nelecas=nelec)
one_body_tensor, core_energy = cas.get_h1eff()
two_body_tensor = ao2mo.restore(1, cas.get_h2eff(), norb)

# CCSD, iterated from the previous amplitudes
ccsd = cc.CCSD(mf, frozen=N_FROZEN)
eris = ccsd.ao2mo()
ccsd.kernel(eris=eris)
for _ in range(CCSD_MAXITER - 1):
    ccsd.kernel(ccsd.t1, ccsd.t2, eris=eris)
t2 = ccsd.t2

occupancies = np.diag(ccsd.make_rdm1())[N_FROZEN:] / 2
initial_occupancies = (occupancies, occupancies)

print(f"norb = {norb}, nelec = {nelec}")
print(f"E(RHF)  = {mf.e_tot:.8f} Ha")
print(f"E(CCSD) = {ccsd.e_tot:.8f} Ha")

Output:

norb = 16, nelec = (5, 5)
E(RHF)  = -108.86761837 Ha
E(CCSD) = -109.09392115 Ha
How QCSC-Prefect manages this step

Trace: The task compute_molecular_integrals has the tag map. Its result goes to get_lucj_params and sqd_loop, so two arrows leave it in the graph. The task logs show the PySCF output.

Configure: The Molecule group of the run form selects the problem, as an FCIDUMP file or as a geometry and basis. This workflow reads algorithms/open_loop/sqd/data/n2_631g_fc.fcidump, which has the cores frozen already. The number of CCSD iterations (CCSD Restarts, 10 by default) is in the sqd-advanced-settings block.

Resume: The result is persisted (compressed/json), so Retry loads it instead of running PySCF again.

The flow run graph in Prefect Cloud. Two arrows leave the compute_molecular_integrals task, one to get_lucj_params and one to sqd_loop. The task logs show the PySCF output of CCSD.

Step 2: Optimize problem for quantum hardware execution

The Optimize stage, the task get_lucj_params in the workflow, fits the ansatz to the QPU before any circuit exists. ffsim builds the LUCJ operator from t2t_2 with only the interactions that a heavy-hex lattice runs without SWAP gates. These are the same-spin pairs of neighboring orbitals and the opposite-spin pairs on every fourth orbital, at most five on a heavy-hex (Heron) processor. ffsim then compresses the operator (optimize=True, on by default in the workflow) so that one layer reproduces t2t_2 more closely.

N_REPS = 1

pairs_aa = [(p, p + 1) for p in range(norb - 1)]
pairs_ab = [(p, p) for p in range(0, norb, 4)][:5]

lucj_op = ffsim.UCJOpSpinBalanced.from_t_amplitudes(
    t2,
    n_reps=N_REPS,
    interaction_pairs=(pairs_aa, pairs_ab),
    optimize=True,
    options={"maxiter": 50},
)
n_params = ffsim.UCJOpSpinBalanced.n_params(
    norb, N_REPS, interaction_pairs=(pairs_aa, pairs_ab)
)

print(f"Opposite-spin pairs: {pairs_ab}")
print(f"Ansatz parameters: {n_params}")

Output:

Opposite-spin pairs: [(0, 0), (4, 4), (8, 8), (12, 12)]
Ansatz parameters: 275
How QCSC-Prefect manages this step

Trace: The task get_lucj_params has the tag optimize. Its result carries everything that the circuit needs, so one arrow leads from it to quantum_sampling.

Configure: The Circuit group of the run form sets the LUCJ layers, the compression and its iteration limit, and the QPU Topology (heavy-hex or square), which sets the interaction pairs. The sqd-advanced-settings block holds reset mitigation and the transpiler settings (the optimization level and the SABRE layout search) that the Execute stage uses.

Resume: The result is persisted (compressed/pickle), so a long compression runs only one time, even if you retry the run.

The Circuit group of the custom run form, with the fields LUCJ Layers, Compress LUCJ Parameters, Compression Max Iteration, and QPU Topology, each with its description.

Step 3: Execute using Qiskit primitives

The Execute stage, the task quantum_sampling in the workflow, builds the circuit (Hartree-Fock state, LUCJ operator, and measurement), transpiles it for the QPU, and samples it. The ffsim PRE_INIT passes run first in the transpiler and turn the ffsim gates into efficient native-gate sequences. This notebook samples with Qiskit Runtime on ibm_fez. Replace ibm_fez with a backend that your instance can use. The workflow samples with QRMI instead, in a Slurm job that holds the QPU.

service = QiskitRuntimeService()
backend = service.backend("ibm_fez")

qreg = QuantumRegister(2 * norb, name="q")
circuit = QuantumCircuit(qreg, ClassicalRegister(2 * norb, name="meas"))
circuit.append(ffsim.qiskit.PrepareHartreeFockJW(norb, nelec), qreg)
circuit.append(ffsim.qiskit.UCJOpSpinBalancedJW(lucj_op), qreg)
circuit.measure(qreg, circuit.cregs[0])

pass_manager = generate_preset_pass_manager(
    optimization_level=3, target=backend.target, seed_transpiler=24
)
pass_manager.pre_init = ffsim.qiskit.PRE_INIT
isa_circuit = pass_manager.run(circuit)
print(f"Backend: {backend.name}, ISA circuit depth: {isa_circuit.depth()}")

Output:

Backend: ibm_fez, ISA circuit depth: 640
SHOTS = 100_000

sampler = Sampler(mode=backend)
job = sampler.run([isa_circuit], shots=SHOTS)
quantum_bool_array = job.result()[0].data.meas.to_bool_array()
print(f"Job {job.job_id()}: {len(quantum_bool_array)} shots")

Output:

Job dbpl975t7fqc7389koog: 100000 shots

As a baseline, draw the same number of uniform random bitstrings, as the workflow does with Quantum Source set to random. The following cell also prints the fraction of each sample set with the correct electron numbers. Step 4 compares SQD on the two sample sets.

random_bool_array = generate_bit_array_uniform(
    SHOTS, 2 * norb, rand_seed=24
).to_bool_array()


def correct_electron_fraction(bool_array):
    """Fraction of the bitstrings with the correct numbers of alpha and beta electrons."""
    # The right half of each bitstring is alpha, the left half is beta.
    n_alpha, n_beta = nelec
    valid = (bool_array[:, norb:].sum(axis=1) == n_alpha) & (
        bool_array[:, :norb].sum(axis=1) == n_beta
    )
    return valid.mean()


print("Correct electron numbers:")
print(
    f"  Quantum samples: {correct_electron_fraction(quantum_bool_array):.4f}"
)
print(
    f"  Random samples:  {correct_electron_fraction(random_bool_array):.4f}"
)

Output:

Correct electron numbers:
  Quantum samples: 0.0336
  Random samples:  0.0040
How QCSC-Prefect manages this step

Trace: The task quantum_sampling has the tag execute, and the task build_lucj_circuit runs inside it. The workflow host prepares the circuit and submits a Slurm job that requests the QPU. On the compute node, the SPANK plugin opens the QPU session, and the QRMI worker transpiles and samples the circuit. The task logs show the walltime of the Slurm job, the two-qubit depth of the transpiled circuit, and, when sampling completes, the IBM Quantum job ID and the QPU execution time.

Configure: The Sampling group of the run form sets the shots and the source of the samples, which is the qrmi-runtime block, the job of an earlier run, or random bitstrings. The QRMI blocks hold where and how the sampler job runs, and the credentials stay on the cluster.

Resume: The samples are persisted (compressed/pickle), so Retry submits no new QPU job. The sqd-sampling-source artifact records where the samples are, so a new run can reuse them, as in Optional: resume and reuse the workflow.

The quantum_sampling task in the flow run graph, with the nested build_lucj_circuit task and the task of the QRMI Slurm job. The task logs report the Slurm job, the IBM Quantum job, the shots, and the QPU execution time.
Why Execute builds and transpiles the circuit

The QRMI worker transpiles the circuit on the compute node against the current Target of the QPU, so the QRMI Runtime owns transpilation. This split prepares for fault-tolerant QCSC systems, where the transpilation of quantum low-density parity-check (LDPC) error-correction circuits is a heavy workload. The workflow caches the circuit parameters, not the circuits. Map and Optimize design the circuit. Map computes the CCSD amplitudes, and Optimize compresses the LUCJ operator with ffsim.UCJOpSpinBalanced.from_t_amplitudes(optimize=True), which can take longer than the transpilation for a current QPU. build_lucj_circuit and the transpiler only encode these parameters into an ISA circuit. This split differs from the conventional Qiskit pattern, and it suggests what a Qiskit pattern can look like in a QCSC system.

Step 4: Postprocess and return result in desired classical format

The Postprocess stage, the task sqd_loop in the workflow, runs self-consistent configuration recovery. Each round corrects the sampled bitstrings with the current orbital occupancies, diagonalizes the Hamiltonian in several batches of configurations, and updates the occupancies from the best result. The following cell runs diagonalize_fermionic_hamiltonian on both sample sets with the same settings. The subspace dimension is the workflow's 1,000,000 determinants (1,000 strings per spin). The notebook uses two batches per round and ten rounds, and the workflow uses four batches and five rounds. As in the workflow, Fulqrum diagonalizes each subspace. sci_solver_from_integrals builds the Fulqrum Hamiltonian one time and returns the sci_solver that the function calls for each batch. A TrimPolicy, as in the SQD tutorial, merges the heaviest strings of the batches into one subspace and diagonalizes it. Of the strings above carryover_threshold, the policy keeps the heaviest half (carryover_ratio=0.5) for the next round, like the workflow's Carryover Fraction.

SUBSPACE_DIM = 1_000_000
NUM_BATCHES = 2
MAX_ITERATIONS = 10

policy = TrimPolicy(
    trim_ratio=0.8,
    carryover_threshold=1e-4,
    carryover_ratio=0.5,
)
sci_solver = sci_solver_from_integrals(one_body_tensor, two_body_tensor)


def run_sqd(bool_array, label):
    """Run SQD on the samples, and return the final energy and the energy of each round."""
    round_energies = []

    def callback(results):
        round_energies.append(
            min(result.energy for result in results) + core_energy
        )
        print(
            f"{label}, round {len(round_energies)}: {round_energies[-1]:.8f} Ha"
        )

    result = diagonalize_fermionic_hamiltonian(
        one_body_tensor,
        two_body_tensor,
        bool_array,
        samples_per_batch=1000,
        norb=norb,
        nelec=nelec,
        num_batches=NUM_BATCHES,
        max_iterations=MAX_ITERATIONS,
        policy=policy,
        sci_solver=sci_solver,
        symmetrize_spin=True,
        max_dim=int(np.sqrt(SUBSPACE_DIM)),
        initial_occupancies=initial_occupancies,
        callback=callback,
        seed=np.random.default_rng(24),
    )
    return result.energy + core_energy, round_energies


quantum_energy, quantum_round_energies = run_sqd(
    quantum_bool_array, "Quantum samples"
)
random_energy, random_round_energies = run_sqd(
    random_bool_array, "Random samples"
)
print(f"SQD energy with quantum samples: {quantum_energy:.8f} Ha")
print(f"SQD energy with random samples:  {random_energy:.8f} Ha")

Output:

Quantum samples, round 1: -109.09698391 Ha
Quantum samples, round 2: -109.10046517 Ha
Quantum samples, round 3: -109.10193633 Ha
Quantum samples, round 4: -109.10186225 Ha
Quantum samples, round 5: -109.10196116 Ha
Quantum samples, round 6: -109.10190268 Ha
Quantum samples, round 7: -109.10199765 Ha
Quantum samples, round 8: -109.10196037 Ha
Quantum samples, round 9: -109.10199167 Ha
Quantum samples, round 10: -109.10205287 Ha
Random samples, round 1: -109.09582359 Ha
Random samples, round 2: -109.09962562 Ha
Random samples, round 3: -109.10059383 Ha
Random samples, round 4: -109.10115380 Ha
Random samples, round 5: -109.10131183 Ha
Random samples, round 6: -109.10148002 Ha
Random samples, round 7: -109.10154865 Ha
Random samples, round 8: -109.10159227 Ha
Random samples, round 9: -109.10161699 Ha
Random samples, round 10: -109.10158185 Ha
SQD energy with quantum samples: -109.10205287 Ha
SQD energy with random samples:  -109.10161699 Ha

The plot compares both sample sets, round by round, with the full configuration interaction (FCI) energy of this active space, −109.10336546-109.10336546 Ha, which PySCF computes in about a minute (cas.run() on the preceding CASCI object). The dashed line is chemical accuracy, 1.6 mHa.

E_FCI = -109.10336546

# SQD stops early if it converges, so each run can have fewer rounds.
for label, round_energies in [
    ("Quantum samples", quantum_round_energies),
    ("Random samples", random_round_energies),
]:
    rounds = range(1, len(round_energies) + 1)
    plt.semilogy(
        rounds, np.abs(np.array(round_energies) - E_FCI), "o-", label=label
    )
plt.axhline(
    abs(ccsd.e_tot - E_FCI), color="tab:gray", linestyle=":", label="CCSD"
)
plt.axhline(
    1.6e-3, color="tab:red", linestyle="--", label="Chemical accuracy"
)
plt.xlabel("Recovery round")
plt.ylabel("|E - E(FCI)| (Ha)")
plt.xticks(range(1, MAX_ITERATIONS + 1))
plt.legend()
plt.show()

Output:

Output of the previous code cell
How QCSC-Prefect manages this step

Trace: The task sqd_loop has the tag post-process. It receives the samples and the integrals, so the graph shows two arrows into it. Each batch is one FulqrumSolverJob call, a solve_eigenstate task that submits one Slurm job running Fulqrum. The batches of a round run concurrently on the compute nodes. The sqd-configuration-recovery artifact lists the energy and dimension of every subspace in every round.

Configure: The SQD group of the run form sets the subspace dimension, the number of batches, and the number of rounds. Solver Block Override (solver_block_ref) in the run form replaces the Solver Block of sqd-advanced-settings for one run, for example with a DICE or SBD block. The eigensolver (PRIMME), threads, and walltime are in the sqd-fulqrum-solver block and its execution profile. The flow loads the solver block before any stage, so a wrong block name fails the run before it uses the QPU.

Resume: sqd_loop is not cached, but it saves a checkpoint after every round, in sqd_checkpoints/<flow run id>/ckpt_t00_w0.npz under the solver's work_dir. The checkpoint holds the occupancies that seed the next round and the best energy so far. On Retry, sqd_loop continues after the last completed round, with the current blocks.

The timeline of the sqd_loop task, where the solve_eigenstate tasks of each round run concurrently. The task logs show the energy of each SQD iteration and the checkpoint that each round writes.

Summary of the stages

The following image summarizes the four stages.

The map stage runs the compute_molecular_integrals task. Its parameters are the molecule group, and integrals, amplitudes, and occupancies are stored in case they're needed for a retry. The optimize stage runs the get_lucj_params task. Its parameters are the circuit group, and LUCJ parameters are stored in case they're needed for a retry. The execute stage runs the quantum_sampling task. Its parameters are the sampling group and QRMI blocks, and samples (and their source as an artifact) are stored in case they're needed for a retry. The postprocess stage runs the sqd_loop task. Its parameters are the SQD group and solver blocks, and by default, a checkpoint is stored after every round in case it is needed for a retry.
Task stages

Run the full workflow

The blocks from Configure and create the Prefect blocks on the workflow host already describe the cluster and the QPU, so a run needs only the run parameters.

Submit the run

In Prefect Cloud, go to Deployments, select sqd, then select Run -> Custom run. Set FCIDump File, the only field without a default, as in the following table. The other rows show the defaults. The form marks the three SQD fields as required, although they have defaults. Select Submit.

Group
Field
Value
MoleculeFCIDump File~/qcsc-prefect/algorithms/open_loop/sqd/data/n2_631g_fc.fcidump
CircuitLUCJ Layers1
CircuitCompress LUCJ ParametersOn
CircuitCompression Max Iteration50
CircuitQPU Topologyheavy-hex
SamplingQuantum Sourcereal-device
SamplingShots100000
SQDSubspace Dimension1000000
SQDBatch Number4
SQDMax Iteration5
SQDCarryover Fraction0.5
SQDCheckpoint RecoveryOn
—Advanced Settings Blocksqd-advanced-settings

A path relative to the ~/qcsc-prefect checkout also works. algorithms/open_loop/sqd/default_params/n2.json holds the same parameters for scripted runs. Checkpoint Recovery adds a folder of less than 1 MB per run under work_dir/sqd_checkpoints/, even for 1000 orbitals, and nothing removes it.

The custom run form of the sqd deployment, with FCIDump File set to the N2 FCIDUMP file.
Note

For a first run without the QPU, set Quantum Source to random, which runs the solver jobs on uniform random bitstrings. To sample through IBM Quantum Platform (IQP) from the workflow host instead of a Slurm job, save a QrmiRuntimeBlock with scheduler = "iqp" (the [runtime] table of sqd_blocks.toml, in place of [qrmi]), and name it in QRMI Runtime Block. This block reads the API key from the qrmi_config.json file of the workflow host.

Read the results

The flow run page shows the task graph with the state of each task, the logs, and, on the Artifacts tab, the five artifacts in the following table.

Artifact
Published by
Contents
sqd-sampling-sourceExecuteWhere the samples are (QRMI work directory, or IBM Quantum job ID for IQP), resource, and shots
qrmi-slurm-jobExecuteThe ID, state, and work directory of the QRMI sampler's Slurm job
sqd-configuration-recoveryPostprocessEnergy and dimension of every subspace in every round
sqd-checkpointPostprocessCheckpoint path, completed rounds, and best energy, when Checkpoint Recovery is on
slurm-sqd-fulqrum-metricsPostprocessThe ID, state, exit code, elapsed time, and nodes of the latest solver job
The Artifacts tab of a completed run lists five artifacts, with the sqd-configuration-recovery table open.

The energy that the flow reports is the lowest energy in the sqd-configuration-recovery table. Each solver job leaves a job_<timestamp>_<id> directory under work_dir. The directory holds the rendered Slurm script, input.npz, solver.json, and the Slurm output. It also holds output.npz and result.json on success, or error.json on failure. Use sacct to see the jobs of the run.

# Run in a terminal on the workflow host.
sacct --format=JobID,JobName,State,ExitCode,Elapsed,WorkDir%60

Optional: resume and reuse the workflow

Retry a failed run, and reuse its samples

Retry: Suppose that, in a run with Max Iteration set to 10, a solver job hits its walltime in round 5 and sqd_loop fails. Raise the walltime in the exec-sqd-fulqrum-slurm block under Blocks, then select Retry on the failed flow run. The retried run loads the stored results of the first three stages and submits no QPU job. It runs only sqd_loop again, with the new walltime, and starts at round 5 from the checkpoint of round 4. Its log shows [ckpt] RESUME from …: completed 4 step(s), then SQD iteration 5/10, as in the following image.

Reuse: A new run computes everything again unless you point it to earlier samples. To try another subspace dimension with the same samples, start a custom run with the same parameters, set Load from QRMI Job Directory (qrmi_result_dir) to the work directory in the sqd-sampling-source artifact of the first run, and change Subspace Dimension. The new run replaces get_lucj_params and quantum_sampling with load_qrmi_result, which reads the samples and submits nothing to the QPU. Reset Mitigation in sqd-advanced-settings must match the first run, because the reload applies it. For samples taken through IQP, set Load from Qiskit Runtime Job ID (qiskit_runtime_job_id) instead, and the stage is load_runtime_job. It reads the job with the credentials of the prefect-qiskit QuantumRuntime block named in Quantum Runtime Block (reload) (ibm-runner by default). Create that block one time with prefect block create quantum-runtime.

Continue a recovery with more rounds: Start a custom run with the same parameters, set Resume from Checkpoint (resume_checkpoint) to the path in the sqd-checkpoint artifact of the first run, and raise Max Iteration. The new run continues after the completed rounds, from the stored occupancies and best energy. For example, a checkpoint with 32 of 100 rounds done and Max Iteration set to 200 continues at round 33/200. The new run reads that checkpoint and writes its own, and the following rules apply.

  • The samples, the molecule, Subspace Dimension, Batch Number, and the solver type (the fulqrum_solver_job part of the solver block) must match the first run, so reuse its samples as in Reuse.
  • The name and the job settings of the solver block, such as its walltime, can change.
  • If a setting that must match differs, the run fails before any solver job and names the difference.
A retried run in Prefect Cloud. The first three tasks complete from stored results, and the new sqd_loop task resumes from the checkpoint, as its first log line shows.

Next steps

Recommendations

If you found this work interesting, you might be interested in the following material:


References

[1] Merz Jr., K. M., Shajan, A., Kaliakin, D., Liang, F., Otsuka, Y., Shirakawa, T., et al. (2026). Crossing the 12,000-atom barrier with heterogeneous quantum-classical supercomputing: quantum chemistry of protein-ligand complexes. arXiv:2605.01138

[2] Kumar, K., Shirakawa, T., Kim, B., Wakizaka, R., Hamamura, I., Sato, M., et al. (2026). Broken-symmetry reference states for sample-based quantum diagonalization with near-DMRG accuracy. arXiv:2611.04496

[3] Robledo-Moreno, J., Motta, M., Haas, H., Javadi-Abhari, A., Jurcevic, P., Kirby, W., et al. (2025). Chemistry beyond the scale of exact diagonalization on a quantum-centric supercomputer. Science Advances, 11(25), eadu9991. DOI, arXiv:2405.05068

Was this page helpful?
Report a bug, typo, or request content on GitHub.