Skip to content

Extend TimeSeriesExecutor to num_chunks_per_job #4831

Open

Description

@alejoe91

Related to shards and zarr v3 #4260 , it would be cool for individual jobs to loop through multiple chunks.

The change can take place only in the TimeSeriesExecutor and global job kwargs

Activity

  1. added this to the 0.106.0 milestone on Oct 6, 2026
  2. samuelgarcia commented on Oct 6, 2026

    @samuelgarcia
    Member

    If did a proof of concpet here : #4833
    I think I prefer: num_chunk_per_batch for the variable.
    The current is absolutly not tested. Just to demonstrated how easy it can be done and usefull for zarr v3 shards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

concurrencyRelated to parallel processing

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions