DUnitX and testing asynchronous code

General Help
1

Hi all,

I’m wondering if there are any common patterns or approaches for unit testing asynchronous code with DUnitX?

I have a component where you call execute and then at some future point, either an event will fire with the results or another event will fire to indicate failure.

All the solutions I’ve come up with so far are more complicated than the code I’m testing.

Are there any common approaches to this?

Cheers

2

The new Delphi Multithreading book by MVP Cesar Romero has a large section on debugging of multithreaded applications which might be helpful. Page 367 onwards in the English version.

It doesn’t contain anything about unit testing of async code and tbh I’ve not seen a lot of info about that even in other languages like Python, C# and so on. Most examples ask you to write wrapper code which, of course, has the danger of being a Schrodinger’s cat; by wrapping it you change the nature of the beast.

Maybe Vince has some answers?

1 Like
3

@vincent has his Awaitable library … ?

We have IFuture …

is it possible to have TEvent (or an enum of the two) as a returned value … in (I assume) the different thread,

and then call the actual event ?

4

My async/await library won’t really solve the problem as it doesn’t block the caller (like real async/await does).

Perhaps create a TEvent to wait on in the test - have the component events set that event (after setting a status variable) to unblock the test and then assert based on the status? Hacky but might work.

Other languages use patterns like async/await or promises which make testing easier.

1 Like
5

Thanks Vincent,

I thought about this, but the issue is that the component event runs on the main thread as well (it’s called by the background thread) so I’d be waiting on the tevent on the same thread I’m sending it. I was under the, perhaps incorrect, impression that that wouldn’t work. Do you know if that is correct?

If so, I might just have to go back to my hacky polling solution.

Cheers

Malcolm

6

Oh, are you explicitly sending the event on the main thread (Synchronize/Queue)? If so then that wouldn’t work.

If it’s your component, then you could ifdef the synchronize out for unit testing and do something like this

procedure TTests.TestComponent
var
  event : TEvent;
  state : boolean; //or enum
  comp : TYourComponent;
  
  procedure OnSuccess(Sender : TObject);
  begin
	status := true;
	event.SetEvent;
  end;
  procedure OnFailure(Sender : TObject);
  begin
	status := true;
	event.SetEvent;
  end;
  
begin
  comp := TYourComponent.Create(nil);
  event := TEVent.Create(nil, true, false,'');
  try
  comp.OnSuccess := OnSuccess;
  comp.OnFailure := OnFailure;
  
  comp.SendIt;
  event.Wait(INFINITE); //or a timeout value
  
  Assert.IsTrue(state);
  
  finally
	event.Free;
	comp.Free;
  end;

end;

FWIW, issues like these are the reason I rarely ever create non visual components - makes things hard to test.

If only delphi had proper async/await - or promises (I did see a promise library somewhere but never got around to trying it) - then testing becomes a lot easier.

7

This is one: GitHub - Laurensvanrun/Delphi-Promises: Delphi implementation of promises for asynchronous programming.

1 Like
8

Thanks mate, I might give this a try, although ifdef’ing out the synchronize sticks in my throat a little bit.

I’m curious how you would use promises to simplify this? My promises understanding is very surface. Is this effectively pushing the component off into another thread and then “waiting” in the main thread for the result? It seems I’d still have to lose the synchronize. If so, I could probably do the same thing using Paul’s idea of an IFuture. Chances are I’ve misunderstood Promises though.

Cheers

9

Promises by themselves wouldn’t help, but a unit test framework could be ‘taught’ to await a promise.

That is what test frameworks like Mocha, Jest etc do - ie the test function can return a promise.

I’m open to suggestions if anyone has ideas for a pattern we could adopt and integrate into DUnitX for testing async code.

3 Likes
10

Thanks to everyone for their suggestions. I got this to work in the end but it took a few false starts.

To refresh your memory, I have a component which provides async results. You call a method to start the process, lets call it Execute and then later you either get an OnSuccess or an OnError event firing. The events run in the main thread, via a call to Synchronize.

  1. So step one was to have those event handlers both set a variable that either indicates success or failure. In addition, they also set a TEvent call FCompleted.
  2. My test code looks like this (a few names changed to protect the innocent):
var
  LStartTime : TDateTime;
begin
  FResult := Pending;

  ComponentUnderTest.Execute(Value);
  LStartTime := Now;
  while (FCompleted.WaitFor(10) = wrTimeOut) and (MillisecondsBetween(Now, LStartTime) < TIMEOUT_MS) do
  begin
    System.Classes.CheckSynchronize;
  end;

  Assert.IsFalse(MillisecondsBetween(Now, LStartTime) >= TIMEOUT_MS, 'Test timed out waiting for the asynchronous operation to complete');
  Assert.AreEqual(Success, FResult);
  Assert.AreEqual(Expected, ComponentUnderTest.RecordCount);
end;

Ordinarily I shouldn’t be using Tevent.Wait on an event set on the same thread, as it’ll block and the event setting will never happen.

The solution I came up with was to wait on the TEvent like I just said I shouldn’t, but only for a short period of time, then call CheckSynchronize to force it to allow the results back from the thread, then wait on the event again, etc, etc up until it either finishes or it times out.

The other key was CheckSynchronize. Before I had that I had a call to Application.ProcessMessages, which apart from making me feel a bit dirty, didn’t actually work.

Reflecting on it now, I possibly don’t even need the tevent, as I suspect I’m just using it as a complicated way to sleep. But if I try refactoring it later I’ll see if it’s actually needed.

So it could probably be tidied up and made easier to reuse, but I’m away. Again, thanks for all your suggestions.

Cheers

1 Like
11

I think 10ms might be a bit short for the Wait. 10ms is shorter that the default windows thread timeslice, and so is likely going to immediately reschedule the thread and starve other threads. I think the tests will perform better if you increase it to 20-50ms.

1 Like
12

Good point Geoff. I’ll play around with a view different numbers and see what the timing looks like.

Update: I did some testing with this, and any value from 1 up to around 50 ms seems to give broadly similar results. Once you get much above 50ms I start to see regular blowouts in the time taken. Not every run, but fairly consistently it’ll take 2 or 3 times as long.

Now, there’s a lot of variables in here. Even aside from local variation, the async operation is hiding a round-trip over the network, plus some processing on a remote machine, so plenty of scope for things to vary between runs. But for this case, it seems there’s marginal difference between having it at 10 vs 50, so 50ms it is.