Rx back off and retry
c#, reactive-programming, system.reactive
Solution
It's because you are putting a `Delay` on the result stream. (The value for n passed to `ExponentialBackoff` on the second iteration is 1, giving a delay of 1 second.)
`Delay` operates on source, but the source proceeds as normal. `Delay` schedules the results it receives to be emitted after the specified duration. So the subscriber is getting the results after the logic of Generate has run to completion.
If you think about it this is how `Delay` must be - otherwise `Delay` would be able to somehow interfere with upstream operators!
It is possible to interfere with upstream operators (without throwing exceptions), by being a slow consumer. But that would certainly be a very bad way for a simple `Delay` to behave.
I don't think the `Delay` is what you intend here - because `Delay` doesn't delay it's subscription. If you use `DelaySubscription` instead, you'll get what you're after I think. This is what's used in the linked question too.
Your question provides a great illustration of the difference between `Delay` and `DelaySubscription`! It's worth thinking about `Defer` in here too.
The distinction between these three is subtle but significant, so let's summarize all three:
`Delay` - Calls target operator immediately to get an `IObservable`, on its `Subscribe` calls `Subscribe` on target immediately, schedules events for delivery after specified delay on the specified `Scheduler`.
`DelaySubscription` - Calls target operator immediately to get an `IObservable`. On its `Subscribe` schedules `Subscribe` on target for execution after specified delay on the specified `Scheduler`.
`Defer` - Has no target operator. On `Subscribe` runs provided factory function to get target `IObservable` and immediately calls `Subscribe`. There's no delay added, hence no `Scheduler` to specify.
Problem
This is based on the code presented in this SO : Write an Rx "RetryAfter" extension method I am using the code by Markus Olsson (evaluation only at the moment), and before anyone asks I have tried to get hold of Markus on Github, but that is blocked where I work, so I felt the only thing I could do was ask here at SO. Sorry about that, if this sits badly with any one. So I am using the following code, in a small demo which is this: ``` class Attempt1 { private static bool shouldThrow = true; static void Main(string[] args) { Generate().RetryWithBackoffStrategy(3, MyRxExtensions.ExponentialBackoff, ex => { return ex is NullReferenceException; }, Scheduler.TaskPool) .Subscribe( OnNext, OnError ); Console.ReadLine(); } private static void OnNext(int val) { Console.WriteLine("subscriber value is {0} which was seen on threadId:{1}", val, Thread.CurrentThread.ManagedThreadId); } private static void OnError(Exception ex) { Console.WriteLine("subscriber bad {0}, which was seen on threadId:{1}", ex.GetType(), Thread.CurrentThread.ManagedThreadId); } static IObservable<int> Generate() { return Observable.Create<int>( o => { Scheduler.TaskPool.Schedule(() => { if (shouldThrow) { shouldThrow = false; Console.WriteLine("ON ERROR NullReferenceException"); o.OnError(new NullReferenceException("Throwing")); } Console.WriteLine("Invoked on threadId:{0}", Thread.CurrentThread.ManagedThreadId); Console.WriteLine("On nexting 1"); o.OnNext(1); Console.WriteLine("On nexting 2"); o.OnNext(2); Console.WriteLine("On nexting 3"); o.OnNext(3); o.OnCompleted(); Console.WriteLine("On complete"); Console.WriteLine("Finished on threadId:{0}", Thread.CurrentThread.ManagedThreadId); }); return () => { }; }); } } public static class MyRxExtensions { /// <summary> /// An exponential back off strategy which starts with 1 second and then 4, 9, 16... /// </summary> public static readonly Func<int, TimeSpan> ExponentialBackoff = n => TimeSpan.FromSeconds(Math.Pow(n, 2)); public static IObservable<T> RetryWithBackoffStrategy<T>( this IObservable<T> source, int retryCount = 3, Func<int, TimeSpan> strategy = null, Func<Exception, bool> retryOnError = null, IScheduler scheduler = null) { strategy = strategy ?? MyRxExtensions.ExponentialBackoff; int attempt = 0; return Observable.Defer(() => { return ((++attempt == 1) ? source : source.Delay(strategy(attempt - 1), scheduler)) .Select(item => new Tuple<bool, T, Exception>(true, item, null)) .Catch<Tuple<bool, T, Exception>, Exception>(e => retryOnError(e) ? Observable.Throw<Tuple<bool, T, Exception>>(e) : Observable.Return(new Tuple<bool, T, Exception>(false, default(T), e))); }) .Retry(retryCount) .SelectMany(t => t.Item1 ? Observable.Return(t.Item2) : Observable.Throw<T>(t.Item3)); } } ``` The code makes sense to me, we try and do an operation if that fails we back off and retry. The exception type that we want to retry can be specified, and we also see that the subscriber only sees the final values once after retry which works (in the demo code above the Exception is only done (OnError'd the first time)). So generally the code works as expected except for one thing. If I look at the output the code above produces I get this: ``` ON ERROR NullReferenceException Invoked on threadId:10 On nexting 1 Invoked on threadId:11 On nexting 1 On nexting 2 On nexting 3 On complete Finished on threadId:10 On nexting 2 On nexting 3 On complete Finished on threadId:11 subscriber value is 1 which was seen on threadId:10 subscriber value is 2 which was seen on threadId:10 subscriber value is 3 which was seen on threadId:10 ``` The interesting thing for me here is that the subscriber values all come in one go, I would have expected that when the OnNext within the Generate() method were called that the Subscribers OnNext would write the OnNext'ed value to the Console output. Could anyone shed any light on why this might be?