Mailkit: Add APIs that return itemized data incrementally

[jstedfast]

For APIs like Pop3Client.GetMessages() or ImapFolder.Fetch() that return an array of items, it might be better if, instead of returning the items as an array, they called a callback with each item as we finish reading the item from the network.

The reason for this is that if, for example, you have a very large IMAP folder and you are fetching the summary for each message in the folder, it's possible that the connection might drop in the middle of the query. If you can only get the items as a return value, then if that connection drops, you get nothing (due to an exception being thrown).

If the Fetch() method calls a callback each time it finishes parsing a message summary, then the client can keep anything it manages to get before the connection is dropped, thus allowing it to continue where it left off once the connection is re-established.

[xavier-rigau]

Not only that. Some servers/connections are super slow and that would help provide a progress bar.

[Sicos1977]

Why not make use of yield?

[jstedfast]

I've implemented the callback approach as yielding IEnumerables isn't really feasible in the IMAP Fetch() case unless I buffer the entire server response and then parse it afterward due to cases like this:

foreach (var item in folder.Fetch (.....) {
    var message = folder.GetMessage (item.UniqueId);
}

If ImapFolder.Fetch() yielded results as it got them, then it's possible for the caller to invoke another IMAP command while the previous command hasn't yet completed, causing a sort of deadlock waiting for the message to be downloaded while more Fetch data arrives but goes un-yielded.

I suppose it would be possible to solve this by sending many individual FETCH requests to the server (1 per message rather than batching), but that would be expensive bandwidth-wise.

For a while now, I've had:

protected virtual void OnMessageSummaryFetched(IMessageSummary summary);

The default implementation emits a MessageSummaryFetched event (which is public).

I think that's good enough. I haven't been able to come up with a better solution :-\

[mwpowellhtx]

@jstedfast Not to add to the library dependencies, but have you looked into Observable Collections? They are designed to look like an enumerable of futures, essentially, and can even support wiring and unwiring of event handlers. Not right for every situation, but when you do need some parallelism they can be useful.