tidy3d.web.api.container.Batch

class Batch

Bases: WebContainer

Interface for submitting several Simulation objects to sever.

Parameters:

• attrs (dict = {}) – Dictionary storing arbitrary metadata for a Tidy3D object. This dictionary can be freely used by the user for storing data without affecting the operation of Tidy3D as it is not used internally. Note that, unlike regular Tidy3D fields, attrs are mutable. For example, the following is allowed for setting an attr obj.attrs['foo'] = bar. Also note that Tidy3D` will raise a TypeError if attrs contain objects that can not be serialized. One can check if attrs are serializable by calling obj.json().

• simulations (Mapping[str, Annotated[Union[tidy3d.components.simulation.Simulation, tidy3d.components.heat.simulation.HeatSimulation, tidy3d.components.eme.simulation.EMESimulation], FieldInfo(default=PydanticUndefined, discriminator='type', extra={})]]) – Mapping of task names to Simulations to run as a batch.

• folder_name (str = default) – Name of folder to store member of each batch on web UI.

• verbose (bool = True) – Whether to print info messages and progressbars.

• solver_version (Optional[str] = None) – Custom solver version to use, otherwise uses default for the current front end version.

• callback_url (Optional[str] = None) – Http PUT url to receive simulation finish event. The body content is a json file with fields {'id', 'status', 'name', 'workUnit', 'solverVersion'}.

• simulation_type (str = tidy3d) – Type of each simulation in the batch, used internally only.

• parent_tasks (Optional[Mapping[str, Tuple[str, ...]]] = None) – Collection of parent task ids for each job in batch, used internally only.

• num_workers (Optional[PositiveInt] = 10) – Number of workers for multi-threading upload and download of batch. Corresponds to max_workers argument passed to concurrent.futures.ThreadPoolExecutor. When left None, will pass the maximum number of threads available on the system.

• jobs_cached (Optional[Mapping[str, Job]] = None) – Optional field to specify jobs. Only used as a workaround internally so that jobs is written when Batch.to_file() and then the proper task is loaded from Batch.from_file(). We recommend leaving unset as setting this field along with fields that were not used to create the task will cause errors.

Notes

Commonly one needs to submit a batch of Simulation. The built-in Batch object is the best way to upload, start, monitor, and load a series of tasks. The batch object is like a Job, but stores task metadata for a series of simulations.

See also

tidy3d.web.api.webapi.run_async()

Submits a set of Simulation objects to server, starts running, monitors progress, downloads, and loads results as a BatchData object.

Job:

Interface for managing the running of a Simulation on server.

Notebooks

• Running simulations through the cloud

• Performing parallel / batch processing of simulations

• Inverse taper edge coupler

Attributes

jobs	Create a series of tasks in the Batch and upload them to server.
num_jobs	Number of jobs in the batch.
attrs

Methods

delete()	Delete server-side data associated with each task in the batch.
download([path_dir])	Download results of each task.
estimate_cost([verbose])	Compute the maximum FlexCredit charge for a given Batch.
get_info()	Get information about each task in the Batch.
get_run_info()	get information about a each of the tasks in the Batch.
load([path_dir])	Download results and load them into BatchData object.
monitor()	Monitor progress of each of the running tasks.
real_cost([verbose])	Get the sum of billed costs for each task associated with this batch.
run([path_dir])	Upload and run each simulation in Batch.
start()	Start running all tasks in the Batch.
to_file(fname)	Exports Tidy3dBaseModel instance to .yaml, .json, or .hdf5 file
upload()	Upload a series of tasks associated with this Batch using multi-threading.

Inherited Common Usage

simulations

folder_name

verbose

solver_version

callback_url

simulation_type

parent_tasks

num_workers

jobs_cached

run(path_dir='.')

Upload and run each simulation in Batch.

Parameters:

path_dir (str) – Base directory where data will be downloaded, by default current working directory.

Returns:

Contains Union[SimulationData, HeatSimulationData, EMESimulationData] for each Union[:class:.Simulation`, HeatSimulation, EMESimulation] in Batch.

Return type:

BatchData

Note

A typical usage might look like:

>>> from tidy3d.web.api.container import Batch
>>> custom_batch = Batch()
>>> batch_data = custom_batch.run() 
>>> for task_name, sim_data in batch_data.items(): 
...     # do something with data. 

bach_data does not store all of the data objects in memory, rather it iterates over the task names and loads the corresponding data from file one by one. If no file exists for that task, it downloads it.

property jobs

Create a series of tasks in the Batch and upload them to server.

Note

To start the simulations running, must call Batch.start() after uploaded.

to_file(fname)

Exports Tidy3dBaseModel instance to .yaml, .json, or .hdf5 file

Parameters:

fname (str) – Full path to the .yaml or .json file to save the Tidy3dBaseModel to.

Example

>>> simulation.to_file(fname='folder/sim.json') 

property num_jobs

Number of jobs in the batch.

upload()

Upload a series of tasks associated with this Batch using multi-threading.

get_info()

Get information about each task in the Batch.

Returns:

Mapping of task name to data about task associated with each task.

Return type:

Dict[str, TaskInfo]

start()

Start running all tasks in the Batch.

Note

To monitor the running simulations, can call Batch.monitor().

get_run_info()

get information about a each of the tasks in the Batch.

Returns:

Dict[str – Maps task names to run info for each task in the Batch.

Return type:

RunInfo]

monitor()

Monitor progress of each of the running tasks.

Note

To loop through the data of completed simulations, can call Batch.items().

download(path_dir='.')

Download results of each task.

Parameters:

path_dir (str = './') – Base directory where data will be downloaded, by default the current working directory.

Note

To load and iterate through the data, use Batch.items().

The data for each task will be named as {path_dir}/{task_id}.hdf5. The Batch hdf5 file will be automatically saved as {path_dir}/batch.hdf5, allowing one to load this Batch later using batch = Batch.from_file().

load(path_dir='.')

Download results and load them into BatchData object.

Parameters:

path_dir (str = './') – Base directory where data will be downloaded, by default current working directory.

Returns:

• BatchData – Contains Union[SimulationData, HeatSimulationData, EMESimulationData] for each Union[Simulation, HeatSimulation, EMESimulation] in Batch.

• The Batch hdf5 file will be automatically saved as {path_dir}/batch.hdf5,

• allowing one to load this Batch later using batch = Batch.from_file().

delete()

Delete server-side data associated with each task in the batch.

real_cost(verbose=True)

Get the sum of billed costs for each task associated with this batch.

Parameters:

verbose (bool = True) – Whether to log the cost and helpful messages.

Returns:

Billed cost for the entire Batch.

Return type:

float

estimate_cost(verbose=True)

Compute the maximum FlexCredit charge for a given Batch.

Parameters:

verbose (bool = True) – Whether to log the cost and helpful messages.

Note

Cost is calculated assuming the simulation runs for the full run_time. If early shut-off is triggered, the cost is adjusted proportionately.

Returns:

Estimated total cost of the tasks in FlexCredits.

Return type:

float

__hash__()

Hash method.