{ "cells": [ { "cell_type": "markdown", "id": "c52e7bba-1230-4974-8e86-2dbe8f6f219b", "metadata": {}, "source": [ "---\n", "title: Debug jobs with NEAT\n", "description: Use the Neat class from the qiskit-ibm-runtime debugging tools module to debug and analyze jobs.\n", "---\n", "\n", "# Debug jobs with NEAT\n", "\n", "{/* cspell:ignore ZIIIII, IZIIII,IIZIII, IIIZII, IIIIZI, IIIIIZ, rdiff */}\n", "\n" ] }, { "cell_type": "markdown", "id": "d0599f3e", "metadata": { "tags": [ "version-info" ] }, "source": [ "{/*\n", " DO NOT EDIT THIS CELL!!!\n", " This cell's content is generated automatically by a script. Anything you add\n", " here will be removed next time the notebook is run. To add new content, create\n", " a new cell before or after this one.\n", " */}\n", "\n", "\n", " \n", " The code on this page was developed using the following requirements.\n", " We recommend using these versions or newer.\n", "\n", " ```\n", " qiskit[all]~=2.5.1\n", " qiskit-ibm-runtime~=0.47.0\n", " qiskit-aer~=0.17\n", " ```\n", " \n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "80d0a1da-8d98-49e1-9cb0-0006093bf44c", "metadata": {}, "source": [ "You can use the `Neat` class to analyze the noise impact on an Estimator workload. For syntax verification, use [Local testing mode](/docs/guides/local-testing-mode).\n", "\n" ] }, { "cell_type": "markdown", "id": "b08f0589", "metadata": {}, "source": [ "## `Neat` class usage\n", "\n", "Before submitting a resource-intensive workload to execute on hardware, you can use the IBM Quantum Compute [NEAT (Noisy Estimator Analyzer Tool)](/docs/api/qiskit-ibm-runtime/debug-tools-neat#neat) class to verify that your Estimator workload is set up correctly, is likely to return accurate results, uses the most appropriate options for the specified problem, and more.\n", "\n", "`Neat` Cliffordizes the input circuits for efficient simulation, while retaining its structure and depth. Clifford circuits suffer similar levels of noise and are a good proxy for studying the original circuit of interest.\n", "\n" ] }, { "cell_type": "markdown", "id": "0dc5bf2a-e536-4141-a77c-0ee407cbd9b2", "metadata": {}, "source": [ "First, import the relevant packages and [authenticate to IBM Quantum Compute Service](/docs/guides/cloud-setup).\n", "\n" ] }, { "cell_type": "markdown", "id": "d653e186-7ec3-4f1b-b0e9-b322055dd6c8", "metadata": {}, "source": [ "### Prepare the environment\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "2f28c824-3158-43e6-ab3c-fd96c31859f0", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import random\n", "\n", "from qiskit.circuit import QuantumCircuit\n", "from qiskit.transpiler import generate_preset_pass_manager\n", "from qiskit.quantum_info import SparsePauliOp\n", "\n", "from qiskit_ibm_runtime import QiskitRuntimeService, EstimatorV2 as Estimator\n", "from qiskit_ibm_runtime.debug_tools import Neat\n", "\n", "from qiskit_aer.noise import NoiseModel, depolarizing_error" ] }, { "cell_type": "code", "execution_count": 2, "id": "a45a6d9e-de39-4586-8395-a7f580f0e0dc", "metadata": {}, "outputs": [], "source": [ "# Choose the least busy backend\n", "service = QiskitRuntimeService()\n", "backend = service.least_busy(operational=True, simulator=False)\n", "\n", "# Generate a preset pass manager\n", "# This will be used to convert the abstract circuit to an equivalent\n", "# Instruction Set Architecture (ISA) circuit.\n", "\n", "pm = generate_preset_pass_manager(backend=backend, optimization_level=0)\n", "\n", "# Set the random seed\n", "random.seed(10)" ] }, { "cell_type": "markdown", "id": "67572a70-da01-40fe-b299-b5599561164a", "metadata": {}, "source": [ "### Initialize a target circuit\n", "\n", "Consider a six-qubit circuit that has the following properties:\n", "\n", "* Alternates between random `RZ` rotations and layers of `CNOT` gates.\n", "* Has a mirror structure, that is, it applies a unitary `U` followed by its inverse.\n", "\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "df19af55-897d-4b1f-baf8-fac2641ae87d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"Output" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def generate_circuit(n_qubits, n_layers):\n", " r\"\"\"\n", " A function to generate a pseudo-random a circuit with ``n_qubits`` qubits\n", " and ``2*n_layers`` entangling layers of the type used in this notebook.\n", " \"\"\"\n", " # An array of random angles\n", " angles = [\n", " [random.random() for q in range(n_qubits)] for s in range(n_layers)\n", " ]\n", "\n", " qc = QuantumCircuit(n_qubits)\n", " qubits = list(range(n_qubits))\n", "\n", " # do random circuit\n", " for layer in range(n_layers):\n", " # rotations\n", " for q_idx, qubit in enumerate(qubits):\n", " qc.rz(angles[layer][q_idx], qubit)\n", "\n", " # cx gates\n", " control_qubits = (\n", " qubits[::2] if layer % 2 == 0 else qubits[1 : n_qubits - 1 : 2]\n", " )\n", " for qubit in control_qubits:\n", " qc.cx(qubit, qubit + 1)\n", "\n", " # undo random circuit\n", " for layer in range(n_layers)[::-1]:\n", " # cx gates\n", " control_qubits = (\n", " qubits[::2] if layer % 2 == 0 else qubits[1 : n_qubits - 1 : 2]\n", " )\n", " for qubit in control_qubits:\n", " qc.cx(qubit, qubit + 1)\n", "\n", " # rotations\n", " for q_idx, qubit in enumerate(qubits):\n", " qc.rz(-angles[layer][q_idx], qubit)\n", "\n", " return qc\n", "\n", "\n", "# Generate a random circuit\n", "qc = generate_circuit(6, 3)\n", "# Convert the abstract circuit to an equivalent ISA circuit.\n", "isa_qc = pm.run(qc)\n", "\n", "qc.draw(\"mpl\", idle_wires=0)" ] }, { "cell_type": "markdown", "id": "0167329b-c6a6-4b2c-98fc-bf9aba9b7ee6", "metadata": {}, "source": [ "Choose single-Pauli `Z` operators as observables and use them to initialize the primitive unified blocs (PUBs).\n", "\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "830b1dcc-2669-46cc-bff8-01a96a05c6ab", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Observables: ['ZIIIII', 'IZIIII', 'IIZIII', 'IIIZII', 'IIIIZI', 'IIIIIZ']\n" ] } ], "source": [ "# Initialize the observables\n", "obs = [\"ZIIIII\", \"IZIIII\", \"IIZIII\", \"IIIZII\", \"IIIIZI\", \"IIIIIZ\"]\n", "print(f\"Observables: {obs}\")\n", "\n", "# Map the observables to the backend's layout\n", "isa_obs = [SparsePauliOp(o).apply_layout(isa_qc.layout) for o in obs]\n", "\n", "# Initialize the PUBs, which consist of six-qubit circuits\n", "# with `n_layers` 1, ..., 6\n", "all_n_layers = [1, 2, 3, 4, 5, 6]\n", "\n", "pubs = [(pm.run(generate_circuit(6, n)), isa_obs) for n in all_n_layers]" ] }, { "cell_type": "markdown", "id": "2a49fc84-0c82-4cbb-a557-6e676e57c9fa", "metadata": {}, "source": [ "### Cliffordize the circuits\n", "\n", "The previously defined PUB circuits are not Clifford, which makes them difficult to simulate classically. However, you can use the `Neat` [`to_clifford`](/docs/api/qiskit-ibm-runtime/debug-tools-neat#to_clifford) method to map them to Clifford circuits for more efficient simulation. The [`to_clifford`](/docs/api/qiskit-ibm-runtime/debug-tools-neat#to_clifford) method is a wrapper around the [`ConvertISAToClifford`](/docs/api/qiskit-ibm-runtime/transpiler-passes-convert-isa-to-clifford) transpiler pass, which can also be used independently. In particular, it replaces non-Clifford single-qubit gates in the original circuit with Clifford single-qubit gates, but it does not mutate the two-qubit gates, number of qubits, or circuit depth.\n", "\n", "See [Efficient simulation of stabilizer circuits with Qiskit Aer primitives](/docs/guides/simulate-stabilizer-circuits) for more information about Clifford circuit simulation.\n", "\n" ] }, { "cell_type": "markdown", "id": "7a86d99e-4431-4d62-8227-c49d17856369", "metadata": {}, "source": [ "First, initialize `Neat`.\n", "\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "4b5bbd4c-bd7f-4679-9348-d41da74d26eb", "metadata": {}, "outputs": [], "source": [ "# You could specify a custom `NoiseModel` here. If `None`, `Neat`\n", "# pulls the noise model from the given backend\n", "noise_model = None\n", "\n", "# Initialize `Neat`\n", "analyzer = Neat(backend, noise_model)" ] }, { "cell_type": "markdown", "id": "b740dcdf-660e-41e2-b5e6-e8cc288af38b", "metadata": {}, "source": [ "Next, Cliffordize the PUBs.\n", "\n" ] }, { "cell_type": "code", "execution_count": 6, "id": "3ad78f41-a2f8-4381-826a-ae728e081ad6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"Output" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "clifford_pubs = analyzer.to_clifford(pubs)\n", "\n", "clifford_pubs[0].circuit.draw(\"mpl\", idle_wires=0)" ] }, { "cell_type": "markdown", "id": "83c3ff81-9f18-43eb-ba6e-57c5ef3d118f", "metadata": {}, "source": [ "## Application 1: Analyze the impact of noise on the circuit outputs\n", "\n", "This example shows how to use `Neat` to study the impact of different noise models on PUBs as a function of circuit depth by running simulations in both ideal ([`ideal_sim`](/docs/api/qiskit-ibm-runtime/debug-tools-neat#ideal_sim)) and noisy ([`noisy_sim`](/docs/api/qiskit-ibm-runtime/debug-tools-neat#noisy_sim)) conditions. This can be useful to set up expectations on the quality of the experimental results before running a job on a QPU. To learn more about noise models, see [Exact and noisy simulation with Qiskit Aer primitives](/docs/guides/simulate-with-qiskit-aer#exact-and-noisy-simulation-with-qiskit-aer-primitives).\n", "\n", "The simulated results support mathematical operations, and can therefore be compared with each other (or with experimental results) to calculate figures of merit.\n", "\n", "\n", " A QPU can be affected by different kinds of noise. The Qiskit Aer noise model used here only simulates some of them and therefore is likely to be less severe than the noise on a real QPU.\n", "\n", " For details on what errors are included when initializing a noise model from a QPU, see the Aer [`NoiseModel`](https://qiskit.github.io/qiskit-aer/stubs/qiskit_aer.noise.NoiseModel.html#qiskit_aer.noise.NoiseModel.from_backend) API reference.\n", "\n", "\n", "Begin by performing ideal and noisy classical simulations.\n", "\n" ] }, { "cell_type": "code", "execution_count": 7, "id": "23859a99-2455-460e-98ea-17b36ea59c36", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ideal results:\n", " NeatResult([NeatPubResult(vals=array([1., 1., 1., 1., 1., 1.])), NeatPubResult(vals=array([1., 1., 1., 1., 1., 1.])), NeatPubResult(vals=array([1., 1., 1., 1., 1., 1.])), NeatPubResult(vals=array([1., 1., 1., 1., 1., 1.])), NeatPubResult(vals=array([1., 1., 1., 1., 1., 1.])), NeatPubResult(vals=array([1., 1., 1., 1., 1., 1.]))])\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Noisy results:\n", " NeatResult([NeatPubResult(vals=array([0.9921875 , 0.99023438, 0.99023438, 0.99023438, 0.97265625,\n", " 0.97070312])), NeatPubResult(vals=array([0.98046875, 0.98632812, 0.98828125, 0.9921875 , 0.96289062,\n", " 0.9765625 ])), NeatPubResult(vals=array([0.96289062, 0.96875 , 0.953125 , 0.953125 , 0.95507812,\n", " 0.9609375 ])), NeatPubResult(vals=array([0.94726562, 0.95507812, 0.93945312, 0.94921875, 0.94921875,\n", " 0.95898438])), NeatPubResult(vals=array([0.91992188, 0.92382812, 0.91015625, 0.921875 , 0.92382812,\n", " 0.9375 ])), NeatPubResult(vals=array([0.90039062, 0.9140625 , 0.90234375, 0.92578125, 0.91601562,\n", " 0.94335938]))])\n", "\n" ] } ], "source": [ "# Perform a noiseless simulation\n", "ideal_results = analyzer.ideal_sim(clifford_pubs)\n", "print(f\"Ideal results:\\n {ideal_results}\\n\")\n", "\n", "# Perform a noisy simulation with the backend's noise model\n", "noisy_results = analyzer.noisy_sim(clifford_pubs)\n", "print(f\"Noisy results:\\n {noisy_results}\\n\")" ] }, { "cell_type": "markdown", "id": "a000a77a-0285-4b72-a69f-8f144f2c2a80", "metadata": {}, "source": [ "Next, apply mathematical operations to compute the absolute difference. The remainder of the guide uses the absolute difference as a figure of merit to compare ideal results with noisy or experimental results, but similar figures of merit can be set up.\n", "\n", "The absolute difference shows that the impact of noise grows with the circuits' sizes.\n", "\n" ] }, { "cell_type": "code", "execution_count": 8, "id": "cd61e437-bd2f-4349-a667-7edab51c4a6e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean absolute difference between ideal and noisy results for circuits with 1 layers:\n", " 1.56%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 2 layers:\n", " 1.89%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 3 layers:\n", " 4.1%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 4 layers:\n", " 5.01%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 5 layers:\n", " 7.72%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 6 layers:\n", " 8.3%\n", "\n" ] } ], "source": [ "# Figure of merit: Absolute difference\n", "def rdiff(res1, re2):\n", " r\"\"\"The absolute difference between `res1` and re2`.\n", "\n", " --> The closer to `0`, the better.\n", " \"\"\"\n", " d = abs(res1 - re2)\n", " return np.round(d.vals * 100, 2)\n", "\n", "\n", "for idx, (ideal_res, noisy_res) in enumerate(\n", " zip(ideal_results, noisy_results)\n", "):\n", " vals = rdiff(ideal_res, noisy_res)\n", "\n", " # Print the mean absolute difference for the observables\n", " mean_vals = np.round(np.mean(vals), 2)\n", " print(\n", " f\"Mean absolute difference between ideal and noisy results \"\n", " f\"for circuits with {all_n_layers[idx]} layers:\\n {mean_vals}%\\n\"\n", " )" ] }, { "cell_type": "markdown", "id": "7abcd001-9eac-4015-97a3-d6250ea4b667", "metadata": {}, "source": [ "You can follow these rough and simplified guidelines to improve circuits of this type:\n", "\n", "* If the mean absolute difference is greater than 90%, mitigation will likely not help.\n", "* If the mean absolute difference is less than 90%, [Probabilistic Error Amplification (PEA)](/docs/guides/error-mitigation-and-suppression-techniques#probabilistic-error-amplification-pea) will likely be able to improve the results.\n", "* If the mean absolute difference is less than 80%, [ZNE with gate folding](/docs/guides/error-mitigation-and-suppression-techniques#zero-noise-extrapolation-zne) will also likely be able to improve the results.\n", "\n", "Because all of the absolute differences above are less than 90%, applying PEA to the original circuit will hopefully improve the quality of its results.\n", "\n" ] }, { "cell_type": "markdown", "id": "c64c936b-5b8f-4fd2-861d-8b1ded2a0ad4", "metadata": {}, "source": [ "You can specify different noise models in the analyzer. The following example performs the same test but adds a custom noise model.\n", "\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "0835c562-55c9-4dbe-879e-7271f8bed280", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean absolute difference between ideal and noisy results for circuits with 1 layers:\n", " 0.0%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 2 layers:\n", " 0.0%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 3 layers:\n", " 0.0%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 4 layers:\n", " 0.0%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 5 layers:\n", " 0.0%\n", "\n", "Mean absolute difference between ideal and noisy results for circuits with 6 layers:\n", " 0.0%\n", "\n" ] } ], "source": [ "# Set up a noise model with strength 0.02 on every two-qubit gate\n", "noise_model = NoiseModel()\n", "for qubits in backend.coupling_map:\n", " noise_model.add_quantum_error(\n", " depolarizing_error(0.02, 2), [\"ecr\", \"cx\"], qubits\n", " )\n", "\n", "# Update the analyzer's noise model\n", "analyzer.noise_model = noise_model\n", "\n", "# Perform a noiseless simulation\n", "ideal_results = analyzer.ideal_sim(clifford_pubs)\n", "\n", "# Perform a noisy simulation with the backend's noise model\n", "noisy_results = analyzer.noisy_sim(clifford_pubs)\n", "\n", "# Compare the results\n", "for idx, (ideal_res, noisy_res) in enumerate(\n", " zip(ideal_results, noisy_results)\n", "):\n", " values = rdiff(ideal_res, noisy_res)\n", "\n", " # Print the mean absolute difference for the observables\n", " mean_values = np.round(np.mean(values), 2)\n", " print(\n", " f\"Mean absolute difference between ideal and noisy results \"\n", " f\"for circuits with {all_n_layers[idx]} layers:\\n {mean_values}%\\n\"\n", " )" ] }, { "cell_type": "markdown", "id": "f2408ca9-3e3c-4a2f-a99a-ce413d5d470f", "metadata": {}, "source": [ "As shown, given a noise model, you can try to quantify the impact of noise on the (Cliffordized version of the) PUBs of interest before running them on a QPU.\n", "\n" ] }, { "cell_type": "markdown", "id": "ddd6da5f-4e84-4bf4-aaeb-0403f21275db", "metadata": {}, "source": [ "## Application 2: Benchmark different strategies\n", "\n", "This example uses `Neat` to help identify the best options for your PUBs. To do so, consider running an estimation problem with PEA, which cannot be simulated with `qiskit_aer`. You can use `Neat` to help determine which noise amplification factors will work best, then use those factors when running the original experiment on a QPU.\n", "\n" ] }, { "cell_type": "code", "execution_count": 10, "id": "358bb82a-4bc9-46c2-98a0-e745ffc6788f", "metadata": {}, "outputs": [], "source": [ "# Generate a circuit with six qubits and six layers\n", "isa_qc = pm.run(generate_circuit(6, 3))\n", "\n", "# Use the same observables as previously\n", "pubs = [(isa_qc, isa_obs)]\n", "clifford_pubs = analyzer.to_clifford(pubs)" ] }, { "cell_type": "code", "execution_count": 11, "id": "5774cb3f-c999-4242-a83a-7dcc0c57510b", "metadata": {}, "outputs": [], "source": [ "noise_factors = [\n", " [1, 1.1],\n", " [1, 1.1, 1.2],\n", " [1, 1.5, 2],\n", " [1, 1.5, 2, 2.5, 3],\n", " [1, 4],\n", "]" ] }, { "cell_type": "code", "execution_count": 12, "id": "0b9900e6-84fe-4776-9bb5-08c6c729be29", "metadata": {}, "outputs": [], "source": [ "# Run the PUBs on a QPU\n", "estimator = Estimator(backend)\n", "estimator.options.default_shots = 100000\n", "estimator.options.twirling.enable_gates = True\n", "estimator.options.twirling.enable_measure = True\n", "estimator.options.twirling.shots_per_randomization = 100\n", "estimator.options.resilience.measure_mitigation = True\n", "estimator.options.resilience.zne_mitigation = True\n", "estimator.options.resilience.zne.amplifier = \"pea\"\n", "\n", "jobs = []\n", "for factors in noise_factors:\n", " estimator.options.resilience.zne.noise_factors = factors\n", " jobs.append(estimator.run(clifford_pubs))\n", "\n", "results = [job.result() for job in jobs]" ] }, { "cell_type": "code", "execution_count": 13, "id": "16c18377-059a-4751-9ab1-afee0ed5b089", "metadata": {}, "outputs": [], "source": [ "# Perform a noiseless simulation\n", "ideal_results = analyzer.ideal_sim(clifford_pubs)" ] }, { "cell_type": "code", "execution_count": 14, "id": "7db531a1-c417-4d5b-bdc3-7a4ad3385fd4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean absolute difference for factors [1, 1.1]:\n", " 12.04%\n", "\n", "Mean absolute difference for factors [1, 1.1, 1.2]:\n", " 3.79%\n", "\n", "Mean absolute difference for factors [1, 1.5, 2]:\n", " 4.73%\n", "\n", "Mean absolute difference for factors [1, 1.5, 2, 2.5, 3]:\n", " 3.65%\n", "\n", "Mean absolute difference for factors [1, 4]:\n", " 2.72%\n", "\n" ] } ], "source": [ "# Look at the mean absolute difference to quickly determine\n", "# the best choice for your options\n", "for factors, res in zip(noise_factors, results):\n", " d = rdiff(ideal_results[0], res[0])\n", " print(\n", " f\"Mean absolute difference for factors \"\n", " f\"{factors}:\\n {np.round(np.mean(d), 2)}%\\n\"\n", " )" ] }, { "cell_type": "markdown", "id": "0c37ef7b-df56-4f5f-9e11-10f209f105f9", "metadata": {}, "source": [ "The result with the smallest difference suggests which options to choose.\n", "\n" ] }, { "cell_type": "markdown", "id": "2530a3e9-21a6-4841-9449-fe181c54aca4", "metadata": {}, "source": [ "## Next steps\n", "\n", "\n", " * Read an overview of [Qiskit debugging tools](/docs/guides/debugging-tools).\n", " * Learn about [Exact and noisy simulation with Qiskit Aer primitives](/docs/guides/simulate-with-qiskit-aer).\n", " * Learn about [available IBM Quantum Compute options](/docs/guides/runtime-options-overview).\n", " * Learn about [Error mitigation and suppression techniques](/docs/guides/error-mitigation-and-suppression-techniques).\n", " * Visit the [Transpile with pass managers](transpile-with-pass-managers) topic.\n", " * Learn [how to transpile circuits](/docs/guides/circuit-transpilation-settings#compare-transpiler-settings) as part of Qiskit patterns workflows.\n", " * Review the [Debugging tools API documentation](/docs/api/qiskit-ibm-runtime/debug-tools).\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "id": "a1b8767d", "source": "© IBM Corp., 2017-2026" } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3" } }, "nbformat": 4, "nbformat_minor": 4 }