glob.py
Glob
dataclass
Bases: Input[Generator[PathLike, None, None]]
IO class that loads all paths provided a pattern.
Although this class can be used as dataset in your nodes,
for most cases it would be more suitable to inherit from
this class and extend the load
method, for example:
>>> class LoadPartitions(Glob):
... def load(self):
... paths = super().load()
... for path in paths:
... yield my_load_func(path)