{
"cells": [
{
"cell_type": "markdown",
"id": "6b7abc7b-b435-43d1-9fd8-c349ee8710f3",
"metadata": {},
"source": [
"---\n",
"title: Manage Qiskit Serverless compute and data resources\n",
"description: Manage compute and data across your Qiskit pattern with Qiskit Serverless.\n",
"---\n",
"\n",
"# Manage Qiskit Serverless compute and data resources\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "3b0771d6-95c9-46dc-955a-f8702f6a2632",
"metadata": {
"tags": [
"version-info"
]
},
"source": [
"\n",
" \n",
" The code on this page was developed using the following requirements.\n",
" We recommend using these versions or newer.\n",
"\n",
" ```\n",
" qiskit[all]~=2.0.0\n",
" qiskit-ibm-runtime~=0.37.0\n",
" qiskit-serverless~=0.22.0\n",
" ```\n",
" \n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "95b2f280-f685-455f-83e9-b172445d7c6a",
"metadata": {},
"source": [
"\n",
" **Qiskit Serverless is getting an upgrade, and its features are changing fast.** During this development phase, find release notes and the most recent documentation at the [Qiskit Serverless GitHub](https://qiskit.github.io/qiskit-serverless/index.html) page.\n",
"\n",
"\n",
"With Qiskit Serverless, you can manage compute and data across your [Qiskit pattern](/docs/guides/intro-to-patterns), including CPUs, QPUs, and other compute accelerators.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "380354c0-5cab-464d-b10f-c94055de3605",
"metadata": {},
"source": [
"## Set detailed statuses\n",
"\n",
"Serverless workloads have several stages across a workflow. By default, the following statuses are viewable with `job.status()`:\n",
"\n",
"* **`QUEUED`**: the workload is queued for classical resources\n",
"* **`INITIALIZING`**: the workload is set up\n",
"* **`RUNNING`**: the workload is currently running on classical resources\n",
"* **`DONE`**: the workload has successfully completed\n",
"\n",
"You can also set custom statuses that further describe the specific workflow stage, as follows.\n",
"\n",
"\n",
" If you're reading the code cells locally in a notebook, you will see the `%%writefile` [magic command](https://ipython.readthedocs.io/en/stable/interactive/magics.html#cellmagic-writefile). Executing cells with this magic command saves them to disk rather than executing them.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a69df8bc-5033-45bf-a837-cffa9d29b844",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing ./source_files/status_example.py\n"
]
}
],
"source": [
"%%writefile ./source_files/status_example.py\n",
"\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"from qiskit_serverless import update_status, Job\n",
"\n",
"## If your function has a mapping stage, particularly application functions, you can set the status to \"RUNNING: MAPPING\" as follows:\n",
"update_status(Job.MAPPING)\n",
"\n",
"## While handling transpilation, error suppression, and so forth, you can set the status to \"RUNNING: OPTIMIZING_FOR_HARDWARE\":\n",
"update_status(Job.OPTIMIZING_HARDWARE)\n",
"\n",
"## After you submit jobs to Qiskit Runtime, the underlying quantum job will be queued. You can set status to \"RUNNING: WAITING_FOR_QPU\":\n",
"update_status(Job.WAITING_QPU)\n",
"\n",
"## When the Qiskit Runtime job starts running on the QPU, set the following status \"RUNNING: EXECUTING_QPU\":\n",
"update_status(Job.EXECUTING_QPU)\n",
"\n",
"## Once QPU is completed and post-processing has begun, set the status \"RUNNING: POST_PROCESSING\":\n",
"update_status(Job.POST_PROCESSING)"
]
},
{
"cell_type": "markdown",
"id": "a8746eae-6f15-4faf-8771-0f3062efc723",
"metadata": {},
"source": [
"After successful completion of this workload (with `save_result()`), this status will be updated to `DONE` automatically.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "b2d40a63-3359-46e9-8f1b-4746b449b407",
"metadata": {},
"source": [
"## Parallel workflows\n",
"\n",
"For classical tasks that can be parallelized, use the `@distribute_task` decorator to define compute requirements needed to perform a task. Start by recalling the `transpile_remote.py` example from the [Write your first Qiskit Serverless program](./serverless-first-program) topic with the following code.\n",
"\n",
"The following code requires that you have already [saved your credentials](/docs/guides/cloud-setup).\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "475d82f0-15cc-4db3-b3b0-54b07822b2a0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing ./source_files/transpile_remote.py\n"
]
}
],
"source": [
"%%writefile ./source_files/transpile_remote.py\n",
"\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"from qiskit.transpiler import generate_preset_pass_manager\n",
"from qiskit_ibm_runtime import QiskitRuntimeService\n",
"from qiskit_serverless import distribute_task\n",
"\n",
"service = QiskitRuntimeService()\n",
"\n",
"@distribute_task(target={\"cpu\": 1})\n",
"def transpile_remote(circuit, optimization_level, backend):\n",
" \"\"\"Transpiles an abstract circuit (or list of circuits) into an ISA circuit for a given backend.\"\"\"\n",
" pass_manager = generate_preset_pass_manager(\n",
" optimization_level=optimization_level,\n",
" backend=service.backend(backend)\n",
" )\n",
" isa_circuit = pass_manager.run(circuit)\n",
" return isa_circuit"
]
},
{
"cell_type": "markdown",
"id": "a5914f1d-f898-4db4-8d1e-ccc8081883b9",
"metadata": {},
"source": [
"In this example, you decorated the `transpile_remote()` function with `@distribute_task(target={\"cpu\": 1})`. When run, this creates an asynchronous parallel worker task with a single CPU core, and returns with a reference to track the worker. To fetch the result, pass the reference to the `get()` function. We can use this to run multiple parallel tasks:\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e8fd31e6-9ab9-4d75-9ef9-a2b9ff9ad37a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Appending to ./source_files/transpile_remote.py\n"
]
}
],
"source": [
"%%writefile --append ./source_files/transpile_remote.py\n",
"\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"from time import time\n",
"from qiskit_serverless import get, get_arguments, save_result, update_status, Job\n",
"\n",
"# Get arguments\n",
"arguments = get_arguments()\n",
"circuit = arguments.get(\"circuit\")\n",
"optimization_level = arguments.get(\"optimization_level\")\n",
"backend = arguments.get(\"backend\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "74fdcd4a-01cd-46ca-aa24-2a8a3605346f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Appending to ./source_files/transpile_remote.py\n"
]
}
],
"source": [
"%%writefile --append ./source_files/transpile_remote.py\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"# Start distributed transpilation\n",
"update_status(Job.OPTIMIZING_HARDWARE)\n",
"\n",
"start_time = time()\n",
"transpile_worker_references = [\n",
" transpile_remote(circuit, optimization_level, backend)\n",
" for circuit in arguments.get(\"circuit_list\")\n",
"]\n",
"\n",
"transpiled_circuits = get(transpile_worker_references)\n",
"end_time = time()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81696ede-3aa5-4e8c-9d35-fdd70c1bf4db",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Appending to ./source_files/transpile_remote.py\n"
]
}
],
"source": [
"%%writefile --append ./source_files/transpile_remote.py\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"# Save result, with metadata\n",
"result = {\n",
" \"circuits\": transpiled_circuits,\n",
" \"metadata\": {\n",
" \"resource_usage\": {\n",
" \"RUNNING: OPTIMIZING_FOR_HARDWARE\": {\n",
" \"CPU_TIME\": end_time - start_time,\n",
" \"QPU_TIME\": 0,\n",
" },\n",
" }\n",
" },\n",
"}\n",
"\n",
"save_result(result)"
]
},
{
"cell_type": "markdown",
"id": "611fe030-4494-46b5-9ea1-9678ac513210",
"metadata": {},
"source": [
"### Explore different task configurations\n",
"\n",
"You can flexibly allocate CPU, GPU, and memory for your tasks via `@distribute_task()`. For Qiskit Serverless on IBM Quantum® Platform, each program is equipped with 16 CPU cores and 32 GB RAM, which can be allocated dynamically as needed.\n",
"\n",
"CPU cores can be allocated as full CPU cores, or even fractional allocations, as shown in the following.\n",
"\n",
"Memory is allocated in number of bytes. Recall that there are 1024 bytes in a kilobyte, 1024 kilobytes in a megabyte, and 1024 megabytes in a gigabyte. To allocate 2 GB of memory for your worker, you need to allocate `\"mem\": 2 * 1024 * 1024 * 1024`.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cea90969-cfbf-4181-9ffa-524f3709dc69",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Appending to ./source_files/transpile_remote.py\n"
]
}
],
"source": [
"%%writefile --append ./source_files/transpile_remote.py\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"@distribute_task(target={\n",
" \"cpu\": 16,\n",
" \"mem\": 2 * 1024 * 1024 * 1024\n",
"})\n",
"def transpile_remote(circuit, optimization_level, backend):\n",
" return None"
]
},
{
"cell_type": "markdown",
"id": "6bc45489-56d0-4f46-8659-9df4d1555516",
"metadata": {},
"source": [
"## Manage data across your program\n",
"\n",
"Qiskit Serverless allows you to manage files in the `/data` directory across all your programs. This includes several limitations:\n",
"\n",
"* Only `tar` and `h5` files are supported today\n",
"* This is only a flat `/data` storage, and cannot have `/data/folder/` subdirectories\n",
"\n",
"The following shows how to upload files. Be sure you have authenticated to Qiskit Serverless with your IBM Quantum account (see [Upload to Qiskit Serverless](/docs/guides/serverless-first-program#upload-to-qiskit-serverless) for instructions).\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "0183278f-8ce3-4466-9255-097b2d211052",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'{\"message\":\"/usr/src/app/media/5e1f442128cdf60018496a04/transpile_demo.tar\"}'"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import tarfile\n",
"from qiskit_serverless import IBMServerlessClient\n",
"\n",
"# Create a tar\n",
"filename = \"transpile_demo.tar\"\n",
"file = tarfile.open(filename, \"w\")\n",
"file.add(\"./source_files/transpile_remote.py\")\n",
"file.close()\n",
"\n",
"# Get a reference to a QiskitFunction\n",
"serverless = IBMServerlessClient()\n",
"transpile_remote_demo = next(\n",
" program\n",
" for program in serverless.list()\n",
" if program.title == \"transpile_remote_serverless\"\n",
")\n",
"\n",
"# Upload the tar to Serverless data directory\n",
"serverless.file_upload(file=filename, function=transpile_remote_demo)"
]
},
{
"cell_type": "markdown",
"id": "4f762470-945f-48d5-a65b-c60d3b2dae3f",
"metadata": {},
"source": [
"Next, you can list all the files in your `data` directory. This data is accessible to all programs.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "14241fc4-d0cb-4803-8752-a460e1f48708",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['classifier_name.pkl.tar', 'output.json.tar', 'transpile_demo.tar']"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"serverless.files(function=transpile_remote_demo)"
]
},
{
"cell_type": "markdown",
"id": "a97bd83e-8250-43bb-b1c4-d40d822c7ba2",
"metadata": {},
"source": [
"This can be done from a program by using `file_download()` to download the file to the program environment, and uncompressing the `tar`.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef649b2a-ed95-4dd2-89d9-61438faa7c1e",
"metadata": {},
"outputs": [],
"source": [
"%%writefile ./source_files/extract_tarfile.py\n",
"# If you include the preceding `%%writefile` command (visible only when you read this locally in a notebook), running this cell saves to disk rather than executing the code.\n",
"\n",
"import tarfile\n",
"from qiskit_serverless import IBMServerlessClient\n",
"\n",
"serverless = IBMServerlessClient(token=\"\") # Use the 44-character API_KEY you created and saved from the IBM Quantum Platform Home dashboard\n",
"files = serverless.files()\n",
"demo_file = files[0]\n",
"downloaded_tar = serverless.file_download(demo_file)\n",
"\n",
"\n",
"with tarfile.open(downloaded_tar, 'r') as tar:\n",
" tar.extractall()"
]
},
{
"cell_type": "markdown",
"id": "5b93dbdb-2060-468b-8496-ba98142a780b",
"metadata": {},
"source": [
"At this point, your program can interact with the files, as you would a local experiment. `file_upload()` , `file_download()`, and `file_delete()` can be called from your local experiment, or your uploaded program, for consistent and flexible data management.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "a004dd78-0e0a-4a3b-83cb-333469533ef6",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"\n",
" * See a full example that [ports existing code to Qiskit Serverless](./serverless-port-code).\n",
" * Read a paper in which researchers used Qiskit Serverless and quantum-centric supercomputing to [explore quantum chemistry](https://arxiv.org/abs/2405.05068v1).\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"id": "a1b8767d",
"source": "© IBM Corp., 2017-2026"
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}