datalad_next.itertools.route_in

datalad_next.itertools.route_in(iterable: Iterable, data_store: list, joiner: Callable[[Any, Any], Any]) → Generator

Yield previously rerouted data to the consumer

This function is the counter-part to route_out(). It takes the iterable iterable and a data store given in data_store and yields items in the same order in which route_out() received them from its underlying iterable (using the same data store). This includes items that were not yielded by route_out(), but only stored.

route_in() uses joiner()-function to determine how stored and optionally processed data should be joined into a single item, which is then yielded by route_in(). route_in() calls joiner() with a 2-tuple. The first element of the tuple is either datalad_next.itertools.StoreOnly or the next item from the underlying iterator. The second element is the data that was stored in the data store. The result of joiner() which will be yielded by route_in().

This module provides a standard joiner-function: join_with_list() that works with splitter-functions that return a list as second element of the result tuple.

The cardinality of iterable must match the number of processed data elements in the data store. The output cardinality of route_in() will be the cardinality of the input iterable of the corresponding route_out()-call. Given the following code:

store_1 = list()
route_in(
    some_generator(
        route_out(input_iterable, store_1, splitter_1)
    ),
    store_1,
    joiner_1
)

route_in() will yield the same number of elements as input_iterable. But, the number of elements processed by some_generator is determined by the splitter_1() in route_out(), i.e. by the number of splitter_1()-results that have don't have datalad_next.itertools.don_process as first element.

Parameters:

• iterable (Iterable) -- The iterable that yields the input data.

• data_store (list) -- The list from which the data that is to be "routed in" is read.

• joiner (Callable[[Any, Any], Any]) -- A function that determines how the items that are yielded by iterable should be combined with the corresponding data from data_store, in order to yield the final result. The first argument to joiner is the item that is yielded by iterable, or datalad_next.itertools.StoreOnly if no data was processed in the corresponding step. The second argument is the data that was stored in data_store in the corresponding step.