AI on Windows: Progress Responses with Phi Silica

In my previous post I covered the basics of creating an Uno Platform application that consumes Phi Silica in order to build a very basic prompt-response based user interface. When the application was run there was a very evident latency when the button was clicked before the response was presented. In this post we’re going to make some minor changes to the calls to Phi Silica to get progress responses in order to make the user experience more responsive.

The existing code called GenerateResponseAsync in order to get a response to the prompt provided as a parameter. This method needs to be awaited and simply returns a Response that is a string, which can be displayed to the user. If we change the code to use the GenerateResponseWithProgressAsync, we can attach an event handler to the Progress event.

The following code shows the MainViewModel which has been updated to use the GenerateResponseWithProgressAsync method. There is a new constructor parameter, a DispatcherQueue, and rather than just updating the Response property, the DispatcherQueue is used to ensure the update is done on the correct thread (the Progress event currently triggers on a different thread).

using Microsoft.UI.Dispatching;
using Microsoft.UI.Xaml.Data;
#if WINDOWS
using Microsoft.Windows.AI.Generative;
#endif

namespace PhiMessagingApp;

[Bindable]
public partial class MainViewModel(DispatcherQueue dispatcher) : ObservableObject
{
    [ObservableProperty]
    private string _response = string.Empty;

    [RelayCommand]
    public async Task SendMessage(string message)
    {
#if WINDOWS

        if (!LanguageModel.IsAvailable())
        {
            var op = await LanguageModel.MakeAvailableAsync();
        }

        using LanguageModel languageModel = await LanguageModel.CreateAsync();
        var progressTask = languageModel.GenerateResponseWithProgressAsync(message);
        progressTask.Progress += 
            (s, progress) => Dispatch(() => Response += progress);

        var result = await progressTask;
        Response = result.Response;
#else
        Response = "Design-time response....";
#endif
    }

    private void Dispatch(Action action)
    {
        if (dispatcher.HasThreadAccess)
        {
            action();
        }
        else
        {
            dispatcher.TryEnqueue(() => action());
        }
    }
}

When we run this, we can see that after clicking the Button we get a progressive response.

There are a couple of questions/suggestions I would make:

• I think it would make sense for the Progress event to be triggered on the same thread that the GenerateResponseWithProgressAsync is called on.
• I’m not quite sure why the API has both GenerateResponseWithProgressAsync and GenerateResponseAsync. If you await either of them you get a response that you can use. There’s no requirement to attach a Progress event handler, so I don’t understand why we need to have two methods.