JobGroup

The JobGroup class is designed to help manage jobs client-side by storing them in named groups. Large experiments can be easily cut in chunks and even ran during multiple days, over multiple Python sessions, the job group will make sure all data can be retrieved from a single location.

Warning

JobGroups store their job data in the persistent data directory.

As these files can grow quite large, you will have to explicitely erase the ones you don’t want to keep.

JobGroup provides the following commands:

  • JobGroup.delete_job_group(name)

  • JobGroup.delete_job_groups_date(del_before_date: datetime)

  • JobGroup.delete_all_job_groups()

Usage example

Here’s an example creating a job group with two jobs, running the same acquisition on a post-processed vs an heralded CNOT gate:

>>> import perceval as pcvl
>>> from perceval.algorithm import Sampler
>>>
>>> p_ralph = pcvl.RemoteProcessor("sim:altair")
>>> p_ralph.add(0, pcvl.catalog["postprocessed cnot"].build_processor())
>>> p_ralph.min_detected_photons_filter(2)
>>> p_ralph.with_input(pcvl.BasicState([0, 1, 0, 1]))
>>> sampler_ralph = Sampler(p_ralph, max_shots_per_call=1_000_000)
>>>
>>> p_knill = pcvl.RemoteProcessor("sim:altair")
>>> p_knill.add(0, pcvl.catalog["heralded cnot"].build_processor())
>>> p_knill.min_detected_photons_filter(2)
>>> p_knill.with_input(pcvl.BasicState([0, 1, 0, 1]))
>>> sampler_knill = Sampler(p_knill, max_shots_per_call=1_000_000)
>>>
>>> jg = pcvl.JobGroup("compare_knill_and_ralph_cnot")
>>> jg.add(sampler_ralph.sample_count, max_samples=10_000)
>>> jg.add(sampler_knill.sample_count, max_samples=10_000)

This first script only prepared the experiment, nothing was executed remotely. Before going on, it’s important for a user to know the details of their plan on the Cloud, for this will establish the number of job they can run concurrently.

The job group supports executing jobs sequentially or in parallel and includes the ability to rerun failed jobs, if needed.

The second script may be used exclusively to run jobs. It includes a built-in tqdm progress bar to provide real-time updates on job execution. To run jobs sequentially with a given delay:

>>> import perceval as pcvl
>>>
>>> jg = pcvl.JobGroup("compare_knill_and_ralph_cnot")  # Loads prepared experiment data
>>> jg.run_sequential(0)  # Will send the 2nd job to the Cloud as soon as the first one is complete

Other methods - jg.run_parallel(), jg.rerun_failed_parallel(), and jg.rerun_failed_sequential(delay).

Note

The jg.run_parallel() method tries to start all jobs in the group on Cloud. An error will occur if it exceeds the limitations defined by the pricing plan (see Quandela Cloud).

A third script can then prepared to analyze results:

>>> import perceval as pcvl
>>>
>>> jg = pcvl.JobGroup("compare_knill_and_ralph_cnot")
>>> results = jg.get_results()
>>> ralph_res = results[0]
>>> knill_res = results[1]
>>> perf_ratio = (ralph_res['physical_perf'] * ralph_res['logical_perf']) / (knill_res['physical_perf'] * knill_res['logical_perf'])
>>> print(f"Ralph CNOT is {perf_ratio} times better than Knill CNOT, but needs a measurement to work")
Ralph CNOT is 490.01059 times better than Knill CNOT, but needs a measurement to work

Note

If the connection token you use in a JobGroup expires or gets revoked, said JobGroup will not be usable anymore. Stay tuned for further improvements on this feature, fixing this issue.

Class reference

class perceval.runtime.job_group.JobGroup(name)

JobGroup handles a collection of remote jobs. A job group is named and persistent (job metadata will be written on disk). Job results will never be stored and will be retrieved every time from the Cloud.

The JobGroup class can perform various tasks such as:

  • Saving information for a collection of jobs, whether they have been sent to the cloud or not.

  • Running jobs within the group either in parallel or sequentially.

  • Rerunning failed jobs within the group.

  • Retrieving all results at once.

Parameters:

name (str) – A name uniquely identifying the group (also, the filename used to save data on disk). If the same name is used more than once, jobs can be appended to the same group.

add(job_to_add, **kwargs)

Adds information of the new RemoteJob to an existing Group. Saves the data in a chronological order in the group (each entry is a dictionary of necessary information - status, id, body, metadata)

Parameters:

  • job_to_add (Job) – a remote job to add to the list of existing job group

  • kwargs – parameters to pass to the remote job, at execution

Return type:

None

static delete_all_job_groups()

Delete all existing groups on disk

Return type:

None

static delete_job_group(name)

Delete a single group by name

Parameters:

name (str) – name of the JobGroup to delete

Return type:

None

static delete_job_groups_date(del_before_date)

Delete all saved groups created before a date.

Parameters:

del_before_date (datetime) – datetime of the oldest job group to keep. Anterior groups will be deleted.

Return type:

None

get_results()

Retrieve results for all jobs in the group. It aggregates results by calling the get_results() method of each job object that have completed successfully.

Return type:

list[dict]

list_active_jobs()

Returns a list of all RemoteJobs in the group that are currently active on the cloud - those with a Running or Waiting status.

Return type:

list[RemoteJob]

static list_existing()

Returns a list of filenames of all JobGroups saved to disk

Return type:

list[str]

list_successful_jobs()

Returns a list of all RemoteJobs in the group that have run successfully on the cloud.

Return type:

list[RemoteJob]

list_unsent_jobs()

Returns a list of all RemoteJobs in the group that have not been sent to the cloud

Return type:

list[RemoteJob]

list_unsuccessful_jobs()

Returns a list of all RemoteJobs in the group that have run unsuccessfully on the cloud - errored or canceled

Return type:

list[RemoteJob]

property name: str

Name of the job group

progress()

Iterates over all jobs in the group to create a dictionary of the current status of jobs. Jobs in the group are categorized as follows (depending on their RunningStatus on the Cloud)

  • Finished

    • successful {‘SUCCESS’}

    • unsuccessful {‘CANCELED’, ‘ERROR’, ‘UNKNOWN’, ‘SUSPENDED’}

  • Unfinished

    • sent {‘WAITING’, ‘RUNNING’, ‘CANCEL_REQUESTED’}

    • not sent {None}

Return type:

dict

Returns:

dictionary of the current status of jobs

property remote_jobs: list[perceval.runtime.remote_job.RemoteJob]

Returns a chronologically ordered list of RemoteJobs in the group.

rerun_failed_parallel(replace_failed_jobs=True)

Restart all failed jobs in the group on the Cloud, running them in parallel.

If the user lacks authorization to send multiple jobs at once or exceeds the maximum allowed limit, an exception is raised, terminating the launch process. Any remaining jobs in the group will not be sent. :rtype: None

Parameters:

replace_failed_jobs – Indicates whether a new job created from a rerun should

replace the previously failed job (defaults to True).

rerun_failed_sequential(delay, replace_failed_jobs=True)

Reruns Failed jobs in the group on the Cloud in a sequential manner with a user-specified delay between the completion of one job and the start of the next.

Return type:

None

Parameters:

  • delay (int) – number of seconds to wait between re-launching jobs on cloud

  • replace_failed_jobs – Indicates whether a new job created from a rerun should

replace the previously failed job (defaults to True).

run_parallel()

Launches all the unsent jobs in the group on Cloud, running them in parallel.

If the user lacks authorization to send multiple jobs to the cloud or exceeds the maximum allowed limit, an exception is raised, terminating the launch process. Any remaining jobs in the group will not be sent.

Return type:

None

run_sequential(delay)

Launches the unsent jobs in the group on Cloud in a sequential manner with a user-specified delay between the completion of one job and the start of the next.

Parameters:

delay (float) – number of seconds to wait between launching jobs on cloud

Return type:

None

track_progress()

Displays the status and progress of each job in the group using tqdm progress bars. Jobs are categorized into “Successful,” “Running/Active on Cloud,” and “Inactive/Unsuccessful.” The method iterates over the list of jobs, continuously refreshing their statuses and updating the progress bars to provide real-time feedback until no “Running/Waiting” jobs remain on the Cloud.

Return type:

None