Running Android Device Tests for Xamarin.Essentials on Windows

As a developer working with Xamarin or Xamarin.Forms you should be aware of the Xamarin.Essentials package that Microsoft have been developing that “provides developers with cross-platform APIs for their mobile applications”. If you haven’t taken the time to look through the source code, it’s well worth cloning the repository and taking a look. Not only is this a great example of how to do multi-targeting, they’ve also taken the time to invest in device specific tests. In this post we’re going to look at how to run the device tests for Android in the same way that it’s run as part of the Azure Pipelines build script.

The first thing to do is of course clone a copy of the repository and then launch the solution in Visual Studio – before we try to run the device tests we’d better make sure everything is able to be built and run. Unless you’ve been using the same machine for a long time, you’re unlikely to have all the difference MonoAndroid SDKs installed. When you take a look at the Dependencies for Xamarin.Essentials in the Solution Explorer window it’s likely to look similar to what mine did.

The good news is that if you force a build, some of these are likely to go away. For example I actually did have 8.0 and 8.1 installed, so after the build they were resolved and the warning indicators disappeared. Unfortunately, the bad news is that I still had a couple of missing versions.

Luckily this is easily resolved using the Android SDK Manager (launch this from Tools > Android > Android SDK Manager from within Visual Studio), where you can install the missing SDK Platforms.

You’ll need to accept the license terms.

Installing the missing Android SDKs should resolve the Android dependencies. If you’re still seeing a warning for the UAP dependency, you need to verify that you’ve got the Windows SDK v 10.0.16299 installed.

The build process for Xamarin.Essentials is driven from the azure-pipeline.yml file in the root of the repository. This YAML file defines various jobs, one of which is the devicetests_android, which as the name suggests, runs the device tests. Actually this job invokes build.sh, specifying the test-android-emu target. The build.sh is actually just a proxy for running build.cake.

I wanted to run the device tests on Windows, so I’m going to be invoking build.ps1, which is the Powershell equivalent for invoking build.cake. The following command invokes Powershell, passes the build.ps1 as the Powershell script to invoke, and then I’ve included the other parameters that were specified in the azure-pipeline.yml.

powershell -f build.ps1 --target=test-android-emu --settings_skipverification=true --verbosity=diagnostic

Note: You need to invoke this command from the DeviceTests folder.

Unfortunately, simply running the Powershell script from a command prompt isn’t sufficient. There are a few steps you’ll need to jump through in order to get the script to run correctly. Without making any changes, if you run the Powershell script you’ll most likely see an error similar to the following image.

Whilst the error is highlighted in red, the actual cause of the error is in light grey font and indicates that JAVA_HOME hasn’t been defined. Easily fixed by using the set command (make sure the path to the JDK matches where it is on your computer).

set JAVA_HOME=C:\Program Files\Android\Jdk\microsoft_dist_openjdk_1.8.0.25

Also, whilst we’re setting environment variables, check that ANDROID_HOME and ANDROID_SDK_ROOT are set to the root folder of the Android SDK installation (typically C:\Program Files (x86)\Android\android-sdk on Windows if installed via Visual Studio installer)

Suggestion: You’ll find that invoking the Powershell script can be quite time consuming because Xamarin.Essentials has to be built (once for each supported platform) and then the emulator needs to be created and launched. To accelerate this process, after running the script the first time, you can temporarily disable the build by commenting out the “.IsDependentOn("Build-Android")” line in the build.cake file.

Now, when you run the Powershell script, you’re likely to see a different error. Again, it’s not the highlighted error that contains the useful information. Looking a few lines earlier you can see that there is an error relating the system image that the script attempts to use.

Again, fixing this issue is relatively easy. You just need to pick one of the existing system images (listed alongside the error) and use it to set the ANDROID_EMU_TARGET variable.

set ANDROID_EMU_TARGET=system-images;android-29;google_apis_playstore;x86

The next error you’ll see is at the point where the script attempts to launch the emulator. It will fail, indicating that it can’t find the file specified.

It would appear that the build script, build.cake, attempts to call “emulator.bat”. However, this file doesn’t exist. Instead, there is an emulator.exe – we just need to adjust the build.cake to use .exe instead of .bat when locating the emulator command.

Unfortunately there’s also an issue with the search logic, resulting in using a copy of emulator.exe that doesn’t work. After making the above change to .exe, you’ll see that it attempts to launch the emulator but comes up with an error “PANIC: Missing emulator engine program for ‘x86’ CPU”.

Again, this issue is easily fixed with a small change to build.exe to get it to search for the emulator.exe in the correct folder. In the following code, we’ve adjusted the search logic to look in only the emulator folder on Windows.

if (ANDROID_HOME != null) {
        var andHome = new DirectoryPath(ANDROID_HOME);
        if (DirectoryExists(andHome)) {
            if(IsRunningOnWindows()){
                emulatorPath = MakeAbsolute(andHome.Combine("emulator").CombineWithFilePath("emulator" + emulatorExt)).FullPath;
                if (!FileExists(emulatorPath))
                    emulatorPath = "emulator" + emulatorExt;
            }
            else{
                emulatorPath = MakeAbsolute(andHome.Combine("tools").CombineWithFilePath("emulator" + emulatorExt)).FullPath;
                if (!FileExists(emulatorPath))
                    emulatorPath = MakeAbsolute(andHome.Combine("emulator").CombineWithFilePath("emulator" + emulatorExt)).FullPath;
                if (!FileExists(emulatorPath))
                    emulatorPath = "emulator" + emulatorExt;
            }
        }
    }

Finally, when you run the script now, it will run through without error. However, you will see a Windows Security Alert requesting a rule be added to the firewall. You’ll need to click the Allow access button in order for the emulator to talk back to the build script – it uses a TCP listener to retrieve the results from the device tests.

After granting permissions, you should see the following output – if you want more details on the device test output, there’s an xml file that is returned which has the full details of the test execution.

One last comment: If you have the emulator running when you run the device test script, you’ll find that it generates an error at the end of running the script. When the script attempts to launch the emulator, emulator.exe will detect the running emulator and will exit immediately. At the end of the script it attempts to terminate the emulator.exe process – since this process has already ended, it throws and error. You can ignore this error as the device tests will still have executed correctly.

Getting Started with Xamarin.Forms and Unit Testing with xUnit and Moq (II)

Getting Started with Xamarin.Forms and Unit Testing with xUnit and Moq (II)

Previous posts in this sequence on Getting Started with Xamarin.Forms:
Multi-Targeting in Visual Studio, SDK Versions, MvvmCross, Authenticating with ADAL
Refactoring MvvmCross and Services, Effects, Navigation with MvvmCross
Presentation Attributes in MvvmCross, Resources and Styles, Resource Dictionaries
Platform Specific Resources with OnPlatform, Device Customization using OnIdiom
Unit Testing with xUnit and Moq

One of the hardest aspects of testing is implementing and executing platform specific unit tests. As I was putting together a post covering platform specific testing I got side tracked looking at the use of Moq, which I posted about previously. On closer inspection of the csproj of the xUnit test project that I created as part of that post, the target framework is set to netcoreapp2.1. In order to test platform specific features I need to adjust the csproj to be multi-targeted covering android, ios and uwp. By making the test project multi-targeted the code can access platform specific APIs in order to be able to verify the platform specific code in our Core library.

<TargetFrameworks>netcoreapp2.1;Xamarin.iOS10;MonoAndroid90;uap10.0.16299</TargetFrameworks>

If we compare the target frameworks with those used by the Core project of our application, we can see that we have netcoreapp2.1 instead of netstandard2.0. In order to take advantage of the xUnit test runner, the test project does need to target netcoreapp2.1. We don’t really need to add netstandard2.0, since we don’t have a test runner that is only based on netstandard2.0 and our application never actually runs as a .NET Standard application (i.e. it runs as an iOS, Android or UWP application)

After changing the target frameworks the test project doesn’t build for a couple of reasons:

– The Visual Studio xUnit test runner isn’t compatible with the platform specific target frameworks

– xUnit has a platform specific test running that needs to be referenced. I’ll cover the platform specific test runners in future posts

The csproj of the test project should be updated to selectively include package references:

<PropertyGroup>
   <TargetFrameworks>netcoreapp2.1;Xamarin.iOS10;MonoAndroid90;uap10.0.16299</TargetFrameworks>
  <IsNetCoreApp>$(TargetFramework.StartsWith(‘netcoreapp’))</IsNetCoreApp>
</PropertyGroup>


<ItemGroup Condition=” ‘$(IsNetCoreApp)’ != ‘true’ “>
   <PackageReference Include=”xunit.runner.devices” Version=”2.4.48″ />
   <PackageReference Include=”UnitTests.HeadlessRunner” Version=”2.0.0″ />
</ItemGroup>


<ItemGroup Condition=” ‘$(IsNetCoreApp)’ == ‘true’ “>
   <PackageReference Include=”Microsoft.NET.Test.Sdk” Version=”15.8.0″ />
   <PackageReference Include=”xunit.runner.visualstudio” Version=”2.4.0″>
   <PrivateAssets>all</PrivateAssets>
   <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
   </PackageReference>
</ItemGroup>

<ItemGroup>
   <PackageReference Include=”xunit” Version=”2.4.0″ />
</ItemGroup>

Even after changing the package references to make them conditional, there were issues referencing the stable release of Moq. Unfortunately the current release of Moq uses some features around reflection that aren’t supported by all platforms, and whilst it’s compatible with netcoreapp2.1, it’s not compatible with netstandard2.0. I went looking for other mocking options and almost immediately came across the work being done on the next iteration of Moq – this is a major change that does more of the heavy lifting at compile time, rather than dynamically at runtime.

At the time of writing, the next release for Moq is currently available only as source code from https://github.com/moq/moq

Once the source code has been built, the assemblies can be added as a direct reference to the test project.

<ItemGroup>
   <Reference Include=”Moq”>
     <HintPath>..LibraryMoqMoq.dll</HintPath>
   </Reference>
   <Reference Include=”Moq.Sdk”>
     <HintPath>..LibraryMoqMoq.Sdk.dll</HintPath>
   </Reference>
   <Reference Include=”Stunts”>
     <HintPath>..LibraryMoqStunts.dll</HintPath>
   </Reference>
</ItemGroup>


<ItemGroup Condition=”‘$(DesignTimeBuild)’ == ‘true'”>
   <Analyzer Include=”..LibraryMoqStunts.dll” />
   <Analyzer Include=”..LibraryMoqStunts.Sdk.dll” />
   <Analyzer Include=”..LibraryMoqRoslyn.Services.Test.Utilities.dll” />
   <Analyzer Include=”..LibraryMoqStunts.CodeAnalysis.dll” />
   <Analyzer Include=”..LibraryMoqStunts.CodeFix.dll” />
   <Analyzer Include=”..LibraryMoqnetstandard.dll” />
   <Analyzer Include=”..LibraryMoqMoq.Sdk.dll” />
   <Analyzer Include=”..LibraryMoqMoq.CodeAnalysis.dll” />
   <Analyzer Include=”..LibraryMoqMoq.CodeFix.dll” />
</ItemGroup>

In addition to assembly references, the Moq libraries are also added as Analyzers – this is to allow for the design time generation of code. Which brings me to the next step which is to adjust our test case to make use of the new Moq syntax. Currently the new version of Moq requires the manual inclusion of Mock.cs and Mock.overloads.cs (content files in the output folder that are generated by building the Moq source code) which include a bunch of helper methods that you’ll need.

The basic process for testing with a mock hasn’t changed – you still need to create and setup your mock entities, invoke a series of methods and then verify actual vs expected output:

[Fact]
public async Task AuthenticateTest()
{
     var credentialsMock = Mock.Of<ICredentialService>();
#pragma warning disable CS4014 // Don’t await – we’re calling Authenticate so that we can train the Mock to return a specific value
     credentialsMock.Authenticate().Returns(()=>Task.FromResult(“SomeRandomAccessToken”));
#pragma warning restore CS4014


    var mainViewModel = new MainViewModel(
         null,
         null,
         credentialsMock);
     Assert.True(await mainViewModel.Authenticate());
}

I’ve bolded the lines that are worth reviewing in the amended test method:

– Where previously we created a new instance of Mock<ICredentialService>, now we’re using the method Mock.Of<ICredentialService>.

– Previously we called Setup to define the mocked behaviour. Now, we call the actual method we want to mock and then use the Returns method to define what the behaviour should be. Since we actually want to mock the Task that is returned by the Authenticate method, it’s important that we don’t attempt to await it. Hence the #pragma statements

This code will not currently compile, showing an error stating that there’s currently no mock implementation on ICredentialService. This error is generated by the Moq analysers we added to the csproj.

image

Expanding the help tooltip

image

Following the tooltip suggestion we can go ahead and create the mock for the ICredentialService. This will be added to a “Mocks” folder and appropriately named ICredentialServicesMock.

With that we can go ahead and run the test case.

image

In this post we’ve taken a quick look at getting started with the next version of Moq. We’ll take this further in the next post to cover running platform tests.

Getting Started with Xamarin.Forms and Unit Testing with xUnit and Moq

Getting Started with Xamarin.Forms and Unit Testing with xUnit and Moq

Previous posts in this sequence on Getting Started with Xamarin.Forms:
Multi-Targeting in Visual Studio, SDK Versions, MvvmCross, Authenticating with ADAL
Refactoring MvvmCross and Services, Effects, Navigation with MvvmCross
Presentation Attributes in MvvmCross, Resources and Styles, Resource Dictionaries
Platform Specific Resources with OnPlatform, Device Customization using OnIdiom

When we kicked off this project we separated out the business logic from the user interface into Core and UI projects respectively. The main aim was to ensure clear separation of context and to allow for the business logic to be developed, and more importantly, tested, independently of the UI. In this post we’ll get started with unit testing our Core library using xUnit. I’ll also touch on using the mocking library, Moq, to make our testing lives easier.

To get started we’ll add another project to our solution, this time based on the xUnit Test Project (.NET Core) from the Add New Project dialog.

image

If we inspect what gets added we’ll see that it’s a regular single targeted project with references to xUnit and the Microsoft test library.

image

After creating the project, don’t forget to do the obligatory NuGet package update. If you open the UnitTest1 class you can right-click within the Test1 method and select Run Test(s).

image

When you execute tests within Visual Studio you can see the execution progress in the Output window. If it doesn’t automatically switch, you may need to select Tests for the “Show output from” dropdown.

image

Before we can write tests for our Core library, we need to add a reference to the Core library to the Testing library.

image

Now we’re ready to start creating our own tests (you can delete UnitTest1.cs since we won’t be needing it). Everyone has their own naming convention and structure for testing library. I tend to mirror the layout of the project I’m testing. So in this case I’ve created a ViewModels folder within the test project and have created a class MainViewModelTest class.

At this point I’m also going to add a reference to Moq (https://www.nuget.org/packages/Moq) which I’ll use as part of the test method

public class MainViewModelTest
{
     [Fact]
     public async Task AuthenticateTest()
     {
         var credentialsMock = new Mock<ICredentialService>();
         credentialsMock.Setup(x => x.Authenticate()).Returns(() => Task.FromResult(“SomeRandomAccessToken”));


        var mainViewModel = new MainViewModel(
             new Mock<IMvxLogProvider>().Object,
             new Mock<IMvxNavigationService>().Object,
             credentialsMock.Object);
         Assert.True(await mainViewModel.Authenticate());
     }
}

Here I’ve kept the AuthenticateTest method relatively straight forward – I created a mock instance of the ICredentialService which returns a predefined access token; this is then used along with two other mock objects when creating the MainViewModel instance. I then assert that calling the Authenticate returns true (i.e. because the Authenticate method on the ICredentialService instance returns a not-null access token).

Right-click within this test method to run the test returns a positive outcome in the Output window. You can also run and see the results of running test cases via the Test Explorer window.

image

And that’s it, you can now write as many test cases as you feel are necessary for your services, viewmodels and other Core classes.

Using IntelliTest in a .NET Standard Library

Using IntelliTest in a .NET Standard Library

Today I was reviewing some of the test cases we have for BuildIt.General – these are a bit dated now as they were created with MSTest using IntelliTest. Recently we’ve updated the BuildIt libraries to target .NET Standard and to support multi-targetting. Unfortunately, whilst the test cases continue to run, I was unable to run IntelliTest in order to add test cases for some of the new features we’ve added recently. As this Stack Overflow question confirms, IntelliTest is only supported on .NET Framework class libraries. What’s worse is that IntelliTest is only supported on old style csproj projects.

In order to get IntelliTest to work for BuildIt.General I created a new .NET Framework class library, BuildIt.General.FullFramework.csproj, which I placed into the same folder as BuildIt.General.Tests (putting it into the BuildIt.General folder causes all manner of weirdness due to different csproj formats not playing nicely together).

image

For each file in BuildIt.General that I wanted to use IntelliTest to generate test cases I added the file as a link to the BuildIt.General.FullFramework project. IntelliTest can be run by right-clicking within the method to be tested, selecting IntelliTest –> Run IntelliTest.

image

The IntelliTest output is shown in the IntelliTest Exploration Results window, from which each test can be saved.

image

More information on IntelliTest can be found on the docs website.