Self Signed Android Certificates and Certificate Pinning in Xamarin.Forms

Working with Self Signed Certificates (Certificate Pinning) in Android Applications with Xamarin.Forms

Next up is looking at working with self-signed certificates in an Android application. Previous posts in this sequence are:

In this post we’re going to briefly talk about non-secure services. Next, we’ll look at how to trust self signed certificates by adding them to the Android bundle. Then lastly we’ll look at intercepting the certificate validation process when making service calls.

One resource that is particularly useful is the network security documentation provided for Android developers. This lists the various elements of the network security configuration file that this post will reference.

Non-Secure (i.e. Http) Services

By default, Android, like iOS, doesn’t allow applications to connect to non-secure services. This means that connecting to http://192.168.1.107 or http://192.168.1.107.xip.io will not work out of the box. You’ll see an error similar to the following, if you attempt to connect to a non-secure service:

Java.IO.IOException: Cleartext HTTP traffic to 192.168.1.107 not permitted occurred

It’s important to note that this behaviour has changed between Android 8.1 and Android 9. Prior to Android 9 there were no default restrictions on calling non-secure, or plain text, services.

Adding Network Security Configuration

You can enable accessing Http/Plain text services by adjusting the network security configuration for the application. To do this we need to add an xml file to the Resources/xml folder which we’ve called network_security_config.xml:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
     <base-config cleartextTrafficPermitted="true" />
</network-security-config>

This adjusts the network security for the application both in debug and in production. If you want to access plain text services only during debugging, you should change the base-config open and close tags to debug-overrides (everything else remaining the same). The “Application (Debuggable = true/false)” assembly attribute controls whether or not the application runs in debugging mode.

#if DEBUG
[assembly: Application(Debuggable = true)]
#else
[assembly: Application(Debuggable = false)]
#endif

We also need to add a reference to the network_security_config.xml file. In the application manifest file (AndroidManifest.xml) add the networkSecurityConfig attribute.

<?xml version="1.0" encoding="utf-8"?>
< manifest http://schemas.android.com/apk/res/android%22">http://schemas.android.com/apk/res/android"
           android_versionCode="1"
           android_versionName="1.0"
           package="com.refitmvvmcross"
           android_installLocation="auto">
     <uses-sdk android_minSdkVersion="19"
               android_targetSdkVersion="28" />
     <application
         android_allowBackup="true"
         android_theme="@style/AppTheme"
         android_label="@string/app_name"
         android_icon="@mipmap/ic_launcher"
         android_roundIcon="@mipmap/ic_launcher_round"
         android:networkSecurityConfig="@xml/network_security_config"
         android_resizeableActivity="true">
         <meta-data
             android_name="android.max_aspect"
             android_value="2.1" />
     </application>
< /manifest>

You can change the filename of the network_security_config.xml file, so long as it matches the networkSecurityConfig attribute value in the AndroidManifest.xml file.

If you run the application it will connect to the Http/Plain text endpoint.

Plain-text on iOS

In addition to enabling/disabling Http/Plain text services across the entire application, it can also be controlled on a per-endpoint basis. See the documentation on the Network Security Configuration for more information.

Self Signed Certificate Endpoints

Switching the endpoint to a Https endpoint with a self signed certificate (eg https://192.168.1.107:5001) will raise the following error:

Javax.Net.Ssl.SSLHandshakeException: <Timeout exceeded getting exception details> occurred

Which of course is completely meaningless, except that we do know it’s related to establishing the SSL connection.

If we look to the Output window, we can find more information about the exception. Unfortunately when an Android app crashes it will spew a lot of mostly irrelevant data into the output window. I’m sure all that debug information is useful in a lot of cases. However, in this case it is a lot of irrelevant information that makes it hard to find the important information. The best way to find more information is to search for the error from the debug session (i.e. Javax.Net.Ssl.SSLHandshakeException). Once found, look at the next 10-15 lines of information. In this case we see

05-04 08:31:21.756 I/MonoDroid( 5948): UNHANDLED EXCEPTION:
05-04 08:31:21.768 I/MonoDroid( 5948): Javax.Net.Ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. ---> Java.Security.Cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. ---> Java.Security.Cert.CertPathValidatorException: Trust anchor for certification path not found.

This points to the fact that the application hasn’t been able to verify the certificate path for the certificate returned by the service. This result is not surprising because the service is using a self signed certificate and neither the Android device or the application trust the certificate.

Trusting Self Signed Android Certificates

In this section we’re going to use the public key from the self signed certificate. This will allow the application to connect to the secure endpoint, without having to write code to intercept the certificate validation. We’ll cover two different ways but they amount to the same thing. In both cases it’s necessary to have the public key of the certificate authority available to the application. The application will use the public key to verify the certificate path of the certificate returned by the service.

Installing to the Android Certificate Store

The first option is to install the public key of the certificate authority onto the Android device. Android maintains two different certificate stores: system and user. We’re going to be adding the certificate to the user store. Aadding to the System store requires root access and can be done using ADB as shown in various posts, such as this one.

The public key that we need to use is the public key of the certificate authority. As discussed in previous posts, we’ve used mkcert to generate our self signed certificate. The public key of the certificate authority used by mkcert is available at C:\Users\[username]\AppData\Local\mkcertrootCA.pem

The first challenge is to get the certificate onto the device, which I typically find easiest via a download link. I add the public key to dropbox and then open the link in Chrome on the Android device:

image image

Installing the Public Key

After downloading the pem file, clicking on the file in the Downloads list does nothing. You actually need to go to Settings / Security & location / Encryption & credentials / Install from SD card

image

The name of the items under settings may vary from device to device. For example Install from SD card might be Install from storage. The quickest way is to search for certificate and go to the item that says something like Install from SD card.

image

Clicking on the Install from SD card/storage settings item will display a file picker. From the burger menu you can select Downloads which will reveal the pem file you’ve just downloaded. However, this item will be disabled, presumably because of some security related to downloaded items. I initially thought that I’d downloaded the wrong certificate format. In actual fact, if you go back to the burger menu and browse the contents of the device (see third image above) and click on Download, you’ll see the same rootCA.pem file but this time it’s not dimmed out and you can click on it.

image

Saving the Root Certificate

If you have a PIN setup on the device, when you click on the rootCA.pem, you’ll need to enter your pin. After entering your PIN you’ll be asked to enter a Name for the certificate and what the certificate is to be used for.

The Name isn’t particularly useful since it doesn’t show up anywhere. It doesn’t appear in either the Trusted credentials screen (second image, showing the added certificate), nor the Security certificate popup that appears if you click on the certificate for more information.

Since we want the certificate to be used by the app we leave the default use, “VPN and apps”. At this point if you open the service endpoint in the browser you should see that the certificate is trusted.

image

Accessing the Android Certificate Store

Now that we’ve installed the certificate, there’s just one change we need to make to the Android application itself. By default Android applications will only use certificates in the system store. However, you can adjust the application to make use of the user certificate store. To do this we need to add a trust-anchors and certificates elements to the network_security_config.xml with the following contents:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
     <debug-overrides>
         <trust-anchors>
            <certificates src="user" />
         </trust-anchors>
     </debug-overrides>
</network-security-config>

In this case, the certificates element will only be used when the application is running in debug mode (ie by using the debug-overrides element). Running the application now will return data from the Https endpoint.

image

The issue with this approach is that the public key for the certificate authority has to be installed on the device. Since the public key appears in the user store, it means that at any point the user could remove it. Whilst it is unlikely that the user will delete the individual item, they may decides to clear all cached credentials.

Clearing cached credentials has the unfortunate side effect of clearing out the user store, thus preventing the application from running. If you’re only connecting to endpoints for the purpose of development or debugging, this option may be sufficient and would minimise the changes required to the application.

Adding to Application Package

The second option is to add the public key of the certificate authority to the application package itself. We’ll use the DER format for the public key which we generated from the mkcert public key file in the previous post. Add the kestrel.der file to the Resources / raw folder, with Build Action set to AndroidResource, and the Custom Tool to MSBuild:UpdateGeneratedFiles.

image

You need to link the public key to the network security configuration file. Add the trust-anchors and certificates elements to the network_security_config.xml file as follows:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
     <base-config>
         <trust-anchors>
             <certificates src="@raw/kestrel" />
         </trust-anchors>
    </base-config>
</network-security-config>

If you only want to use the certificates in debugging, exchange the base-config with debug-overrides.

That’s all you need to do to be able to access a service that uses a self-signed certificate.

Validating Server Certificates (i.e. Android Certificate Pinning)

The other alternative to working with self signed certificates is to override the certificate validation that is done as part of each service request. On Android you can override the ConfigureCustomSSLSocketFactory and GetSSLHostnameVerifier methods on the AndroidClientHandler. For example, here’s a CustomerAndroidClientHandler that inherits from the AndroidClientHandler and overrides these methods:

public class CustomAndroidClientHandler : AndroidClientHandler
{
     protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
     {
         request.Version = new System.Version(2, 0);
         return await base.SendAsync(request, cancellationToken);
     }

    protected override SSLSocketFactory ConfigureCustomSSLSocketFactory(HttpsURLConnection connection)
     {
         return SSLCertificateSocketFactory.GetInsecure(0, null);
     }

    protected override IHostnameVerifier GetSSLHostnameVerifier(HttpsURLConnection connection)
     {
         return new BypassHostnameVerifier();
     }
}

internal class BypassHostnameVerifier : Java.Lang.Object, IHostnameVerifier
{
     public bool Verify(string hostname, ISSLSession session)
     {
         return true;
    }
}

We’ve also included the class BypassHostnameVerifier, which is a basic implementation of the IHostnameVerifier that needs to be returned from the GetSSLHostnameVerifier method. In the ConfigureCustomSSLSocketFactory method we’re returning a factory generated by the GetInsecure method. According to the method documentation the GetInsecure method “Returns a new instance of a socket factory with all SSL security checks disabled”. The documentation goes on to provide a warning “Warning: Sockets created using this factory are vulnerable to man-in-the-middle attacks!”.

I would like at this point to reiterate this warning. By providing your own handling of the SSL checks, you are effectively taking ownership and responsibility of making sure your application isn’t being hacked. As such, I would recommend only overriding these methods when you need to connect to a service that uses a self signed certificate. Make sure that in the Verify method you validate the service response to ensure only responses with self signed certificates that you trust are processed (ie check for man-in-the-middle attacks).

If you’re just interested in certificate pinning (i.e. ensuring that your application is connecting to a server that is returning a known certificate) you can simply override the GetSSLHostnameVerifier method. This will leave the default SSL verification/security in place but give you the opportunity to validate that the certificate being returned is what you expect.

Http2 Handling on Android

The bad news is that Http2 combined with self signed certificates doesn’t currently play nicely with the out of the box AndroidClientHandler. The AndroidClientHandler is the default and preferred handler on Android. When you attempt to connect to a service that is Http2 only and it uses a self signed certificate, you’ll probably see an error similar to the following:

Javax.Net.Ssl.SSLHandshakeException: Connection closed by peer occurred

Unfortunately there’s no simple solution without importing another library. Luckily, the ModerHttpClient library has the code necessary to do this and is updated slightly in the modernhttpclient-updated nuget package.

Please do not include the nuget package in your application as neither the original or the updated package are being actively maintained. Instead, copy the source code that you need (in this case the NativeClientHandler) and take ownership of maintaining it within your application logic.

To round this out, here’s an example that overrides the NativeMessageHandler to set the Http version to 2.

public class CustomNativeClientHandler : NativeMessageHandler
{
     protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
     {
         request.Version = new System.Version(2, 0);
         return await base.SendAsync(request, cancellationToken);
     }
}

And when an instance of the CustomNativeClientHandler is created, the EnableUntrustedCertificates property is set to true. This effectively disables any of the SSL checking, so again opens up the risk of man-in-the-middle attacks.

Mvx.IoCProvider.LazyConstructAndRegisterSingleton<HttpMessageHandler, IServiceOptions>(options =>
{
     return new CustomNativeClientHandler
     {
         AutomaticDecompression = options.Compression,
         EnableUntrustedCertificates = true
     };
});

And when we run this, we can see that the Http protocol used is Http/2.

image

In this post we’ve touched on using both self signed certificates, as well as certificate pinning. This is not an easy topic but important for application developers to be across. If you come across issues with your application security, feel free to reach out on twitter or via Built to Roam’s contact page.


Nick Randolph @thenickrandolph

Built to Roam on building cross-platform applications


Publishing ASP.NET Core 3 Web API to Azure App Service with Http/2

Publishing ASP.NET Core 3 Web API to Azure App Service with Http/2

In my previous post I was testing a new ASP.NET Core 3 Web API that I’d created that simply returns header and http information about the request. Having got everything working locally I decided that I should push it into an Azure App Service to make it accessible from anywhere (this seemed to be easier than attempting to connect to my locally running service from a Xamarin.Forms application). Here’s the process:

Right-click on the ASP.NET Core project and select Publish.

image

In this case we’re going to select App Service (ie a Windows host) and Create New, followed by the Publish button. Next we need to give the new App Service a name and specify both a Resource Group and an App Service Plan – in this case I’m going to create all of these as part of the publishing process

image

Hitting Create will firstly create the necessary Azure resources and then it will proceed with publishing the ASP.NET Core project into the App Service. Unfortunately, once this process has finished you’ll see that the launched url doesn’t load correctly:

image

And secondly, when you return to Visual Studio you’ll see a warning prompt indicating that ASP.NET Core 3 isn’t supported in Azure App Service right now.

image

Luckily Microsoft documentation has you covered. If you go to the main documentation on publishing to Azure App Service there is a link of to deploying preview versions of ASP.NET Core applications. This document covers two different ways to fix this issue – you can either install the preview site extensions for ASP.NET Core 3, or you can simply change your deployment to be a self-contained application. In this case we’re going to go with deploying a self-contained application, since this reduces any external dependencies which seems sensible to me.

After returning to Visual Studio and dismissing the above version warning, you’ll see the Publish properties page with the default publish configuration (you can get back to this page by right-clicking your ASP.NET Core project and selecting Publish in the future).

image

We’re going to click the pencil icon along side any of the summary properties to launch the Publish dialog and change the Deployment Mode to Self-Contained, and the Target Runtime to win-x86. You may be tempted to select win-x64 but only do this if the Platform setting on your App Service is set to 64 Bit, otherwise your service won’t start and you’ll see a 503 service error.

image

Click Save and then the Publish button to publish the application using the updated publishing properties. Note that if you’re on a network that has a slow uplink (eg ADSL) this might take a while, so you might consider jumping on a fast network (eg 4G mobile) to do the upload (and yes, this does make Australia sound like an under-developed nation when it comes to access to the internet – sigh!).

Without any further messing around, the ASP.NET Core application launches correctly:

image

Hmmm, but wait, shouldn’t it be reporting HTTP/2, after all that’s what the browser was reporting when I ran the same service on Kestrel. There’s a couple of pieces to this answer but before we do, I want to remove any element on confusion as to what’s going on here by switching across to using curl – this way we have both control over what protocol we’re requesting but also detailed logging on what protocol is being used (you’ll see why this is important in a minute). Executing the following:

curl https://headerhelper.azurewebsites.net/api/values -v

image

As you can see from the image, the response was indeed done over Http/1.1, which is consistent with the Http protocol listed by the service. Ok, so let’s try requesting Http/2

curl https://headerhelper.azurewebsites.net/api/values –http2 –v

image

This call is successful but again returns Http/1.1 – this is because curl is attempting to request an upgrade to http/2 but the service isn’t willing/able to upgrade the connection.

curl https://headerhelper.azurewebsites.net/api/values –http2-prior-knowledge -v

image

This call fails because curl is forcing the use of Http/2 when in fact the service isn’t able to communicate using Http/2. So, how do we fix this? Well the good news is that Azure App Service has a simple configuration setting that can be used to enable Http/2. Here I’m just setting the HTTP version in the Configuration page for the Azure App Service.

image

This can also be set via the resource explorer, as covered by a number of other people (eg this post). After making your change, don’t forget to Save changes and then give the service 30-60seconds for it to be restarted – if you attempt to request the service immediately you’ll still get Http/1.1 responses.

After the change has been applied, here’s what we see when we use the same curl commands as above:

curl https://headerhelper.azurewebsites.net/api/values –http2 –v

curl https://headerhelper.azurewebsites.net/api/values –http2-prior-knowledge –v

image

Note that it doesn’t matter whether we attempt to negotiate the http/2 upgrade (–http2 flag) or force the point (–http2-prior-knowledge), in both cases the connection reports HTTP/2. However, what’s not cool is that the Http protocol returned by the service is HTTP/1.1 – this is what is seen by the ASP.NET Core Web API.

What we’re seeing here is that Azure is terminating the Http/2 connection and then communicating to the underlying ASP.NET Core application using Http/1.1. This is consistent with the way that SSL support is done – Azure terminates the SSL connection, meaning that your ASP.NET Core application doesn’t need to worry about fronting a secure service. This is awesome for developers that want to add SSL or HTTP/2 to their existing services – you just enable the option in the configuration page of your App Service. However, the down side is that it makes leveraging some of the underlying capabilities of HTTP/2 impossible – for example, it’s currently impossible to host a GRPC service in an App Service as this relies on HTTP/2 to function.

The question still remains – when I request the service from the browser, what protocol is being used? The response returns HTTP/1.1 because that’s what our ASP.NET Core application sees. However, if we look at the browser debugging tools, we can see that the response is indeed being handled over a HTTP/2 connection. Note that this isn’t exposed in the UI of the debugging tools but if you save the request you can see the full details:

image

Opening the HAR file in VS Code:

image

And there you have it – deploying an ASP.NET Core 3 application to Azure App Service and exposing it using HTTP/2.