Can this library be used with Persistent channel

[askids]

hi,

I am trying to figure out, if this extension library be used with persistent channel from dotnext libraries?
https://dotnet.github.io/dotNext/features/threading/channel.html

My primary usage is for unbounded channel with never ending stream of incoming messages, which will need to go through couple of transformations using PipeAsync feature of this extension library to apply required transformations. My main problem is to ensure memory growth of the app is capped, while continuing to use unbounded channel and second is message recovery, if app crashes.

So wanted to check if I use this library will the interim channels used also leverage persistent channel and not normal in-memory channel?

Thanks!

[electricessence]

Interesting question, and I'll admit right away I'm not very confident what the reality is.

This library was designed around in-memory channels. If your persistent channel is just a channel, that when initialized and configured might already have items, then I don't think there's any issue here.

The extensions are just patterns that you would often do yourself but easily mess up.
So if you wanted to read from that channel, there are numerous options to simplify that.

Filtering and transforming use a ChannelReader and avoid creating new channels, they simply "pull" items from the channel in a similar way to LINQ.

Batching, and other more complex operations like .PipeAsync use an additional in-memory channel or buffer and without question, if you app died in the middle of a batch, you'd probably lose that batch.

IMO, this is not what channels are really for and something like an Azure Queue or its equivalent where there are poison queues, or some sort of ownership or transaction, is where you'll get resiliency and guarantee that no messages are lost.