Language-ext: Question: using TryAsync together with async operations on collections of data

[Malgefor]

Hi!
I have a question regarding how to work with possibly failing tasks (TryAsync) and collections of data .
Let's say I have the situation I have an IEnumerable<Image> containing x images which consist of a FileName and the content as a Stream. I want to check with a remote API whether or not the image already exists by filename and, if it doesn't exist, add it via a second API call.

Since the API could be unavailable, I wrapped the API calls with TryAsync, but that leaves me with, say, an IEnumerable<TryAsync<Unit>>, with each TryAsync containing Unit when the operation was successful and an Exception when the operation failed for whatever reason.

I could use .Sequence() to create a single TryAsync<IEnumerable<Unit>> and Match() to determine that one of the operations failed along the way, but I'm not sure if that is the way to go about these kinds of situations.
Can anyone give some pointers on dealing with collections of Try / TryAsync?

[Malgefor]

To elaborate with some code:

private Task<Unit> Start(IReadOnlyList<ImageReference> imagesReferences, Insight article)
{
    return TryAddArticleToDatabase(article) // returns TryAsync<Unit>
        .Bind(_ => DownloadImagesToStorage(imagesReferences))
        .IfFail(exception =>
        {
            Log.Error(exception, "Error when handling article!");
            return Unit.Default;
        });
}

private TryAsync<Unit> DownloadImagesToStorage(IReadOnlyList<ImageReference> imageReferences)
{
    return imageReferences.Map(TryDownloadImageToStorage)
        .Sequence()
        .Map(_ => Unit.Default);
}

private TryAsync<Unit> TryDownloadImageToStorage(ImageReference reference)
{
    return TryAsync(() => this.fileRepository.DoesFileExist(reference.FileName, FolderPath))
        .Bind(doesFileExist =>
        {
            if (!doesFileExist)
            {
                return
                    from contentStream in TryAsync(() => this.imageWebClient.DownloadImage(reference.Url))
                    from unit in this.fileRepository.Add(contentStream, reference.FileName, FolderPath) // Returns TryAsync<Unit>
                    select unit;
            }

            return TryAsync(Unit.Default);
        });
}

When I run this and no exceptions are thrown the code does what I expect it to do. Suppose that this.imageWebClient.DownloadImage(reference.Url) throws an exception, I see a BottomException when I log the error in the IfFail(). Why does that happen?

[louthy]

Sorry for the delay. Yes, .Sequence() and .Traverse() is the way to fire off many Trys and get the _first error_. I note you're on an older version of lang-ext, the beta (which will become a full release any day now) has SequenceParallel and SequenceSerial for TryAsync.

I would encourage you to upgrade if you can, and consider switching from TryAsync<A> to Aff<A>. Aff<A> is a much more opinionated way to handle side-effects and has a significantly better story for managing errors, doing retries with back-off, defining time-outs, etc.

There's a synchronous variant called Eff<A> which is notionally equivalent to Try<A>. It's been designed to play nice with Aff<A>, to the point where you can use them together in the same monadic expression.

On top of that, there's Aff<RT, A> and Eff<RT, A> which are designed to take a '_runtime_' that allows for injectable behaviours, and carrying an config environment through your computations. These are also designed to play nice with Aff<A> and Eff<A>.

And finally, there's LanguageExt.Pipes which is a streaming composition system. I'm still finessing that, but it's a monad transformer based on the Haskell Pipes library. It supports lifting of Aff and Eff computations into streams.

Try and TryAsync will continue to be supported, but I won't be focussing on any new features for them, because they're superfluous to requirements now Eff and Aff are eating their lunch.

I've been writing up some documentation on them, if you're interested in seeing whether they'd work for you. And there's a new sample project to go along with it.

[Malgefor]

Thanks for the response, really appreciate it!

I'm currently replacing TryAsync<A> with Aff<A> throughout my code. The change doesn't have much impact on the overall structure, which is nice. :) I did notice that there is not (yet) a .MapT() or .BindT() function for Aff<IEnumerable<A>>/Eff<IEnumerable<A>> which made me nest some Map() functions further.

I will have a look at the LanguageExt.Pipes package aswell! I will probably need a bit more time to dive into that one though haha.

[louthy]

Re: Pipes, I’ve had some breakthroughs with that over the past few days that massively reduces the generics burden.

I’m still working through it - so there’s a few examples here.

From readLine2 down is the heavy generics approach, above that is the newer approach. There’s examples that work with IEnumerable, IObservable, and IAsynEnumerable