Debugging Http/Https Traffic using Fiddler for Xamarin iOS, Android and Windows (UWP) Applications

Debugging Http/Https Traffic using Fiddler for Xamarin iOS, Android and Windows (UWP) Applications

One of the most frustrating things as a frontend developer working with a third party (or even sometimes your own) backend is when you seem to be getting incorrect or missing data – you don’t know whether it’s your application, or the services that are doing something weird. The easiest way to validate this is to pretend to be a hacker and stage a man-in-the-middle attack on your own application using a tool like Fiddler or Charles and thus being able to inspect the traffic from your application. Unfortunately the same effort that goes into protecting app developers from such attacks by the various platform manufacturers, also means that it is harder for developers to setup Fiddler to help diagnose failing service calls.

In this post I’ll walk through getting Fiddler setup for a Xamarin.Forms application for iOS, Android and Windows – the same basic approach will work for a native or Xamarin iOS/Android application as well. For the purpose of this post I’ve created an application using the Blank Xamarin.Forms template that comes with Visual Studio 2019, targeting all three platforms. In the OnAppearing method in the MainPage of the Xamarin.Forms application I’ve got some basic code to retrieve a string for a Https endpoint (we’ll use a Https endpoint on the assumption that if we can intercept Https then Http will work by default).

protected override async void OnAppearing()
{
     base.OnAppearing();


    var client = new HttpClient();
     var users = await client.GetStringAsync(“https://reqres.in/api/users?page=0”);
     Debug.WriteLine(users);
}

Before we get started with the individual platforms, it’s worth checking your configuration for Fiddler:

– Firstly, you need to make sure you have setup Https traffic decryption. I’m not going to repeat the documentation, so check out how to Configure Fiddler to Decrypt HTTPS Traffic

– Secondly, you’ll need to setup Fiddler for debugging remote traffic. This is as simple as going to Options, under Connections, and making sure the “Allow remote computers to connect” checkbox is checked. Note that if you’re running Skype or some other communication tools, they may jump onto the port set by Fiddler. In this case you may want to adjust the “Fiddler listens on port” number accordingly.

image

Windows (UWP)

I’ll start with Windows, mainly because it’s relatively straight forward to get setup. With Fiddler running, I just run the UWP project from within Visual Studio and it just works – I can immediately see the Https traffic.

image

Ok, so there is something that might go wrong, or that you might not have setup correctly. You need to make sure your application has been added to the exemption list so that traffic can be routed to the local machine. For more information see the blog post Revisiting Fiddler and Win8+ Immersive applications. To check this, click the WinConfig button in the top left corner of the Fiddler interface.

image

Locate your application and confirm that there is a check in the box next to your application. If there’s not, check the box, and then click Save Changes. The number of times I’ve forgotten to click the Save Changes button and then wondered why I’m still not able to see the traffic from my application….

image

Note: One other issue I’ve seen but can’t reproduce reliably, is that sometimes when you run your application from Visual Studio it unchecks the box in the WinConfig in Fiddler. If for some reason you no longer see traffic in Fiddler for your application, or your application stops being able to download data when you’re running Fiddler, go back and double check the exemption list in Fiddler.

Also it’s worth noting that when configuring Https traffic decryption, if you don’t accept all the prompts to install and trust the certificates, Windows debugging with Fiddler will not work. The reason Windows traffic debugging works without any further configuration is because the Fiddler root certificate has already been trusted on the machine running Fiddler. If you’re attempting to run the Windows application on a different machine than the one running Fiddler, you’ll need to install and trust the Fiddler root certificate (in addition to setting the remote machine as a proxy….that’s a topic for another post). Navigate to http://[ipaddress]:[port] where [ipaddress] is the ip address of the machine running Fiddler and the [port] is the port number that Fiddler is listening on (see above for configuring remote connections). Do NOT use https – this site is http only. You should see a screen similar to the following image – if you don’t, make sure you check that Fiddler is running!!

image

Click on the FiddlerRoot Certificate and install as a Trusted Certificate Authority on the Local Machine – this should allow your Windows application to trust Fiddler.

iOS

Next up is iOS and in this case we’re going to use the iOS simulator, but the same process should work with any iOS device that’s on the same local network as the machine running Fiddler. There are two steps to setup iOS for traffic debugging: trust the Fiddler root certificate and set the http proxy to use the machine running Fiddler. To setup the simulator, first launch the iOS application from Visual Studio – if you’re on Windows, this will launch the remote viewer for the simulator. Once the simulator is running, stop debugging and we’ll setup the simulator for traffic debugging.

Navigate to http:[ipaddress]:[port] (eg http://192.168.1.109:8888) to load the Fiddler echo page and then click on the Fiddler Certificate link. Follow the prompts to download and install the certificate

image image image

In addition to downloading the certificate you also need to install it. Go to Settings / General / Profile and click through on the FiddlerRoot profile in order to Install it.

image image image image image image image

As we need the Fiddler certificate to be trusted as a root certificate (Fiddler generates certificates for each site you go to from this root certificate) go to Settings / About / Certificate Trust Settings and toggle the switch next to the FiddlerRoot certificate.

image image image image

The only difference between a real iOS device and the simulator is that on a real iOS device you can set a network proxy (see online tutorials such as this one for instructions on setting a proxy). Unfortunately this isn’t configurable on the simulator (although you could set it on the mac running the simulator but this would affect all traffic on the mac). When running on the simulator, we can adjust the HttpClient to use a WebProxy using the following code:

var handler = new HttpClientHandler();
handler.Proxy = new WebProxy(“192.168.1.109”, 8888);
var client = new HttpClient(handler);

Running the iOS application should show network traffic in Fiddler and you should still see the returned data printed out in the Output window (ie from the Debug.WriteLine statement).

Android

For Android I’m going to use the Android Simulator but again a real device should behave similarly again assuming it’s connected to the same local network as the machine running Fiddler.

Unlike iOS that will use any proxy configured for the device, for Android you need to explicitly opt in to use a proxy in your code – you’ll need to use code similar to the following on both emulator and real devices:

var handler = new HttpClientHandler();
handler.Proxy = new WebProxy(“192.168.1.109”, 8888);
var client = new HttpClient(handler);

What’s a little scary about this code is that it “just works”. You might be thinking that this is a good thing, and I guess from a debugging perspective it is but it does raise the question of how much of the system security model does the Microsoft built HttpClientHandler respect. What I would have expected here is an SSL fail because the Fiddler root certificate isn’t trusted by the emulator, nor is the application configured to use any user certificates.

The other thing to point out here is that you should not be using the HttpClientHandler as I’ve discussed in my previous post on working with the HttpClient. Let’s change our code by moving it into the OnCreate method of the Android head project and change over to using the AndroidClientHandler.

protected override async void OnCreate(Bundle savedInstanceState)
{
     …
     var handler = new AndroidClientHandler();
     handler.Proxy = new WebProxy(“192.168.1.109”, 8888);
     var client = new HttpClient(handler);
     var users = await client.GetStringAsync(“
https://reqres.in/api/users?page=0″);
     System.Diagnostics.Debug.WriteLine(users);
}

Now when we run the application we see the very familiar SSL handshake exception being raised, which is what we should expect. To get things to work, we now need to install the Fiddler certificate and configure the application to use user certificates.

To install the Fiddler root certificate, navigate to http:[ipaddress]:[port] (eg http://192.168.1.109:8888) to load the Fiddler echo page and then click on the Fiddler Certificate link. Follow the prompts to download and install the certificate

image image image image image  image

After installing the certificate you can go to Setting / Security and Location / Encryption & credentials / User credentials to inspect the certificate

image  image

With the certificate installed into the user store, you need to configure the Android project to allow the use of certificates from the user store. In my post Working with Self Signed Certificates (Certificate Pinning) in Android Applications with Xamarin.Forms, I covered this in detail but in summary you need to create a network_security_config.xml file which sets the trust-anchors property (set using the debug-overrides element) to include certificates from the user store and then you need to reference this xml file from the networkSecurityConfig attribute on the application element in the AndroidManifest.xml file.

After installing the certificate and adding the network security configuration to the Android application you should now see network requests from the application appear within Fiddler.

Working with Self Signed Certificates (Certificate Pinning) in Android Applications with Xamarin.Forms

Working with Self Signed Certificates (Certificate Pinning) in Android Applications with Xamarin.Forms

Next up in the sequence of posts talking about app security is looking at working with self-signed certificates in an Android application. Previous posts in this sequence are:

Accessing ASP.NET Core API hosted on Kestrel over Https from iOS Simulator, Android Emulator and UWP Applications.
Publishing ASP.NET Core 3 Web API to Azure App Service with Http/2
Xamarin and the HttpClient For iOS, Android and Windows
Working with Self Signed Certificates (Certificate Pinning) in Windows (UWP) Application with Xamarin.Forms
Working with Self Signed Certificates (Certificate Pinning) in iOS Application with Xamarin.Forms

Similar to the post on working with self signed certificates on iOS, in this post we’re going to briefly talk about non-secure services, followed by looking at how to trust self signed certificates by adding them to the Android bundle and then lastly look at intercepting the certificate validation process when making service calls.

One resource that is particularly useful is the network security documentation provided for Android developers which lists the various elements of the network security configuration file that will be used and referenced in this post.

Non-Secure (i.e. Http) Services

By default, Android, like iOS doesn’t allow applications to connect to non-secure services. This means that connecting to http://192.168.1.107 or http://192.168.1.107.xip.io will not work out of the box and you’ll see an error similar to the following, if you attempt to connect to a non-secure service:

Java.IO.IOException: Cleartext HTTP traffic to 192.168.1.107 not permitted occurred

It’s important to note that this behaviour has changed between Android 8.1 and Android 9 – prior to Android 9 there was no default restrictions on calling non-secure, or plain text, services.

Accessing Http/Plain text services can be enabled again by adjusting the network security configuration for the application. To do this we need to add an xml file to the Resources/xml folder which we’ve called network_security_config.xml with the following contents:

<?xml version=”1.0″ encoding=”utf-8″?>
< network-security-config>
     <base-config cleartextTrafficPermitted=”true” />
< /network-security-config>

Note: this adjusts the network security for the application both in debug and in production. If you want to access plain text services during debugging only, you should change the base-config open and close tags to debug-overrides (everything else remaining the same). Whether the application is running in debugging mode is controlled by the Application (Debuggable = true/false) assembly attribute, which in this case we have on the MvvmCross setup file in the Android project.

#if DEBUG
[assembly: Application(Debuggable = true)]
#else
[assembly: Application(Debuggable = false)]
#endif

We also need to add a reference to the network_security_config.xml file into the application manifest file (AndroidManifest.xml) by adding the networkSecurityConfig attribute.

<?xml version=”1.0″ encoding=”utf-8″?>
< manifest http://schemas.android.com/apk/res/android%22″>http://schemas.android.com/apk/res/android”
           android_versionCode=”1″
           android_versionName=”1.0″
           package=”com.refitmvvmcross”
           android_installLocation=”auto”>
     <uses-sdk android_minSdkVersion=”19″
               android_targetSdkVersion=”28″ />
     <application
         android_allowBackup=”true”
         android_theme=”@style/AppTheme”
         android_label=”@string/app_name”
         android_icon=”@mipmap/ic_launcher”
         android_roundIcon=”@mipmap/ic_launcher_round”
         android:networkSecurityConfig=”@xml/network_security_config”
         android_resizeableActivity=”true”>
         <meta-data
             android_name=”android.max_aspect”
             android_value=”2.1″ />
     </application>
< /manifest>

Note that the network_security_config.xml filename can be changed so long as the name matches the networkSecurityConfig attribute value in the AndroidManifest.xml file. Now if you run the application it will connect to the Http/Plain text endpoint.

image

In addition to enabling/disabling Http/Plain text services across the entire application, it can also be controlled on a per-endpoint basis. See the documentation on the Network Security Configuration for more information.

Switching the endpoint to a Https endpoint with a self signed certificate (eg https://192.168.1.107:5001) we see the following error being thrown

Javax.Net.Ssl.SSLHandshakeException: <Timeout exceeded getting exception details> occurred

Which of course is completely meaningless, except that we do know it’s related to establishing the SSL connection. If we look to the Output window we can find more information about the exception. Unfortunately when an Android app crashes there is a spew of mostly irrelevant output that’s dumped into the window (I’m sure it’s useful in a lot of cases but in this case it is irrelevant information that makes it hard to find the important information). The best way to find more information is to search for the error that was thrown in the debug session (i.e. Javax.Net.Ssl.SSLHandshakeException and then look at the next 10-15 lines of information. In this case we see

05-04 08:31:21.756 I/MonoDroid( 5948): UNHANDLED EXCEPTION:
05-04 08:31:21.768 I/MonoDroid( 5948): Javax.Net.Ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. —> Java.Security.Cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. —> Java.Security.Cert.CertPathValidatorException: Trust anchor for certification path not found.

This points to the fact that the application hasn’t been able to verify the certificate path for the certificate returned by the service – no surprises there because it’s a self signed certificate and we’ve done nothing to tell Android or the application about the certificate.

Trusting Self Signed Certificates

In this section we’re going to look at using the public key from the self signed certificate in order to allow the application to connect to the secure endpoint, without having to write code to intercept the certificate validation. We’ll cover two different ways but they amount to the same thing – having the public key of the certificate authority available for the application so that it can verify the certificate path of the certificate returned by the service.

Installing to User Certificate Store

The first option is to install the public key of the certificate authority onto the Android device. Android maintains two different certificate stores: system and user. We’re going to be adding the certificate to the user store (adding to the System store requires root access and can be done using ADB as shown in various posts such as this one). The public key that we need to use is the public key of the certificate authority. As discussed in previous posts, we’ve used mkcert to generate our self signed certificate, so the public key of the certificate authority used by mkcert is available at C:Users[username]AppDataLocalmkcertrootCA.pem

The first challenge is to get the certificate onto the device, which I typically find easiest via a download link. Here I’ve added the public key to dropbox and have opened the link in Chrome on the Android device:

image image

After downloading the pem file, clicking on the file in the Downloads list does nothing. You actually need to go to Settings / Security & location / Encryption & credentials / Install from SD card

image
Note that the settings items may be named slightly differently. For example Install from SD card might be Install from storage. The quickest way to get there is to search for certificate and go to the item that says something like Install from SD card or storage.

image

When you click on the Install from SD card/storage settings item you’ll be presented with what amounts to a file picker, typically showing Recent files. From the burger menu you can select Downloads which will reveal the pem file you’ve just downloaded. However, this item will be disabled, presumably because of some security related to downloaded items. My initial thought was that I’d downloaded the wrong certificate format but in actual fact if you go back to the burger menu and browse the contents of the device (see third image above) and click on Download, you’ll see the same rootCA.pem file but this time it’s not dimmed out and you can click on it.

image

When you click on the rootCA.pem, if you have a PIN setup on the device you’ll most likely be asked to enter your pin. After entering your PIN you’ll be asked to enter a Name for the certificate and what the certificate will be used for. The Name isn’t particularly useful since it doesn’t show up in either the Trusted credentials screen (second image, showing the added certificate), nor the Security certificate popup that appears if you click on the certificate for more information. Since we want the certificate to be used by the app we leave the default use which should be VPN and apps. At this point if you open the service endpoint in the browser you should see that the certificate is trusted.

image

Now that we’ve installed the certificate, there’s just one change we need to make to the Android application itself to make use of the certificate. By default Android applications will only use certificates in the system store. However, you can adjust the application to make use of the user certificate store. To do this we need to add a trust-anchors and certificates elements to the network_security_config.xml with the following contents:

<?xml version=”1.0″ encoding=”utf-8″?>
<network-security-config>
     <debug-overrides>
         <trust-anchors>
            <certificates src=”user” />
         </trust-anchors>

     </debug-overrides>
</network-security-config>

In this case we’ve setup the certificates element to only be used when running application in debug mode (ie by using the debug-overrides element). Running the application now will return data from the Https endpoint.

image

The issue with this approach is that the public key for the certificate authority has to be installed on the device in order for the application to work. Since the public key appears in the user store, it means that at any point the user could remove it. Whilst it is unlikely that the user will go in and specifically delete the individual item, it is possible that the user decides to clear our all cached credentials – this has the unfortunate side effect of clearing out the user store, thus preventing the application from running. This said, if you’re only connecting to endpoints for the purpose of development or debugging, and that in production you use certificates from a well known certificate authority, this option may be sufficient and minimises the changes required to the application.

Adding to Application Package

The second option is to add the public key of the certificate authority to the application package itself. We’ll use the DER format for the public key which we generated from the mkcert public key file in the previous post. The kestrel.der file is added to the Resources / raw folder with Build Action set to AndroidResource (if the Custom Tool isn’t set, you can updated it also).

image 

Now that the public key has been added to the package, it’s important that there is a linkage between the network security configuration file and the public key. This is done by adding the trust-anchors and certificates elements to the network_security_config.xml file as follows:

<?xml version=”1.0″ encoding=”utf-8″?>
<network-security-config>
     <base-config>
         <trust-anchors>
             <certificates src=”@raw/kestrel” />
         </trust-anchors>
    </base-config>
</network-security-config>

(again if you only want to use the certificates in debugging, exchange the base-config with debug-overrides)

That’s all you need to do to be able to access a service that uses a self-signed certificate.

Validating Server Certificates (i.e. Certificate Pinning)

The other alternative to working with self signed certificates is to override the certificate validation that goes on as part of each service request. On Android this can be done by overriding the ConfigureCustomSSLSocketFactory and GetSSLHostnameVerifier methods on the AndroidClientHandler. For example, here’s a CustomerAndroidClientHandler that inherits from the AndroidClientHandler and overrides these methods:

public class CustomAndroidClientHandler : AndroidClientHandler
{
     protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
     {
         request.Version = new System.Version(2, 0);
         return await base.SendAsync(request, cancellationToken);
     }


    protected override SSLSocketFactory ConfigureCustomSSLSocketFactory(HttpsURLConnection connection)
     {
         return SSLCertificateSocketFactory.GetInsecure(0, null);
     }


    protected override IHostnameVerifier GetSSLHostnameVerifier(HttpsURLConnection connection)
     {
         return new BypassHostnameVerifier();
     }
}


internal class BypassHostnameVerifier : Java.Lang.Object, IHostnameVerifier
{
     public bool Verify(string hostname, ISSLSession session)
     {
         return true;
    }
}

We’ve also included the class BypassHostnameVerifier, which is a basic implementation of the IHostnameVerifier that needs to be returned from the GetSSLHostnameVerifier method. In the ConfigureCustomSSLSocketFactory method we’re returning a factory generated by the GetInsecure method, which according to the method documentation “Returns a new instance of a socket factory with all SSL security checks disabled” and then goes on to provide a warning “Warning: Sockets created using this factory are vulnerable to man-in-the-middle attacks!” I would like at this point to reiterate this warning – by providing your own handling of the SSL checks, you are effectively taking ownership and responsibility of making sure your application isn’t being hacked. As such I would recommend only overriding these methods only when you need to connect to a service that uses a self signed certificate and that you put more validation in the Verify method to ensure only the self signed certificate that you trust is being returned in the service call (ie check for man-in-the-middle attacks).

Note that if you’re just interested in certificate pinning (ie ensuring that your application is connecting to a server that is returning a known certificate) you can simply override the GetSSLHostnameVerifier method. This will leave the default SSL verification/security in place but give you the opportunity to validate that the certificate being returned is what you expect.

Http2 Handling on Android

The bad news is that Http2 combined with self signed certificates doesn’t play nicely with the out of the box AndroidClientHandler, which is the default and preferred handler on Android. When you attempt to connect to a service that is Http2 only and it uses a self signed certificate (ie you’ve had to either install or add the certificate to the application package as above), you’ll probably see an error similar to:

Javax.Net.Ssl.SSLHandshakeException: Connection closed by peer occurred

Unfortunately there’s no simple solution without importing another library. Luckily, the ModerHttpClient library has the code necessary to do this and has been updated slightly in the modernhttpclient-updated nuget package. Please do not include the nuget package in your application as neither the original or the updated package are being actively maintained. Instead, copy the source code that you need (in this case the NativeClientHandler) and take ownership of maintaining it within your application logic.

To round this out, here’s an example that overrides the NativeMessageHandler to set the Http version to 2.

public class CustomNativeClientHandler : NativeMessageHandler
{
     protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
     {
         request.Version = new System.Version(2, 0);
         return await base.SendAsync(request, cancellationToken);
     }
}

And when an instance of the CustomNativeClientHandler is created, the EnableUntrustedCertificates property is set to true – this effectively disables any of the SSL checking, so again opens up the risk of man-in-the-middle attacks

Mvx.IoCProvider.LazyConstructAndRegisterSingleton<HttpMessageHandler, IServiceOptions>(options =>
{
     return new CustomNativeClientHandler
     {
         AutomaticDecompression = options.Compression,
         EnableUntrustedCertificates = true
     };
});

And when we run this, we can see that the Http protocol used is Http/2.

image

In this post we’ve touched on using both self signed certificates, as well as certificate pinning. This is not an easy topic but important for application developers to be across. If you come across issues with your application security, feel free to reach out on twitter or via Built to Roam’s contact page.

———-

Nick Randolph @thenickrandolph

Built to Roam on building cross-platform applications

———-

Working with Self Signed Certificates (Certificate Pinning) in iOS Application with Xamarin.Forms

Working with Self Signed Certificates (Certificate Pinning) in iOS Application with Xamarin.Forms

Next up in the sequence of posts talking about app security is looking at working with self-signed certificates in an iOS application. Previous posts in this sequence are:

Accessing ASP.NET Core API hosted on Kestrel over Https from iOS Simulator, Android Emulator and UWP Applications.
Publishing ASP.NET Core 3 Web API to Azure App Service with Http/2
Xamarin and the HttpClient For iOS, Android and Windows
Working with Self Signed Certificates (Certificate Pinning) in Windows (UWP) Application with Xamarin.Forms

In this post we’re going to cover 1) accessing non-secure services 2) trusting a self-signed certificate and 3) handling certificate validation. This gives you all the options you should need when accessing which security option to use during development and how you might want to implement certificate pinning in the production version of your app.

Non-Secure (i.e. Http) Services

iOS is secure by default – this means that by default an iOS application can only connect to services over Https with a certificate that can be verified against one of the well known certificate authorities. For most services this is not a problem, as most production services will use a certificate that has been issued by a well know certificate authority (eg when you deploy a service to Azure App Service the generated endpoint (eg myservices.azurewebsites.net) already has a Https endpoint with a certificate that is trusted). However, this may be an issue when you’re running services locally, or you’re attempting to connect to a service in an environment where either there is no https endpoint. In these cases, you may want to adjust the behaviour of the iOS application so that it can connect to a non-secure (ie http) endpoint.

Let’s see this in action by changing our the endpoint of our service request to http://192.168.1.107:5000 – when we setup the endpoint it was configured for both https on port 5001 and http on port 5000. If you happen to be trying this out with a new ASP.NET Core 3 project, don’t forget that the template comes with the line UseHttpRedirection in startup.cs so if you want to expose an http endpoint you’ll need to remove that line.

image

In the iOS application if you simply change to http://192.168.1.107:5000 the application will operate correctly, despite all the concern that http connections aren’t supported. This is because there’s a clear set of exceptions to the Https rule on iOS:

image

So what happens if you do want to use http but instead of using an IP address (ie the exclusion we just saw) you have a fully qualified domain name. Let’s try this by changing the endpoint to http://192.168.1.107.xip.io:5000. BIG shout out to the xip.io service which is super cool – you can use any ip address before the xip.io and the returned ip address from doing a DNS lookup will be the same ip address.

image

Now when we run the iOS application and it attempts to make the service call we get the following exception raised within Visual Studio:

Unhandled Exception:
System.Net.WebException: <Timeout exceeded getting exception details> occurred

This isn’t very meaningful. However, in the Output window there’s much more information:T

Unhandled Exception:
System.Net.WebException: The resource could not be loaded because the App Transport Security policy requires the use of a secure connection. —> Foundation.NSErrorException: Error Domain=NSURLErrorDomain Code=-1022 “The resource could not be loaded because the App Transport Security policy requires the use of a secure connection.” UserInfo={NSLocalizedDescription=The resource could not be loaded because the App Transport Security policy requires the use of a secure connection., NSErrorFailingURLStringKey=http://192.168.1.107.xip.io:5000/api/values

This exception we can combat by including an exception in the Info.plist:

<key>NSAppTransportSecurity</key>
<dict>
   <key>NSExceptionDomains</key>
   <dict>
     <key>192.168.1.107.xip.io</key>
     <dict>
       <key>NSExceptionAllowsInsecureHTTPLoads</key>
       <true />
     </dict>
   </dict>
</dict>

Adding this to the plist will exclude the listed domain from the App Transport Security policy. Another alternative is to use the NSAllowArbitraryLoads attribute – this is not recommended as it effectively disables the security policy for any endpoint that the app connects to.

<key>NSAppTransportSecurity</key>
<dict>
   <key>NSAllowsArbitraryLoads</key>
   <true />

</dict>

So that’s it for accessing non-secure, or Http, endpoints. Simply add the endpoint to the NSExceptionDomains element in the Info.plist file and you’re good to go.

Trusting Self-Signed Certificates

Now let’s go back to connecting to a secure endpoint but this time we’re going to keep with using a xip.io address to ensure any security policies are enforced. The secure endpoint would be https://192.168.1.107.xip.io:5001. Note that I’ve reissued the certificate used by the ASP.NET Core application to include 192.168.1.107.xip.io in the alternative names section:

image 

You’d imagine this would just work since the endpoint domain is already listed in the Info.plist file (see earlier on doing this using the NSExceptionDomains) but instead you get an error similar to the following.

System.Net.WebException: The certificate for this server is invalid. You might be connecting to a server that is pretending to be “192.168.1.107.xip.io” which could put your confidential information at risk.

The information in this error is only partially correct – the certificate for the server is actually value, it’s just that the app/device isn’t able to verify the integrity of the certificate. In order to use this endpoint we’re going to need to find a way for the device to trust the ceritifcate being returned by the server. By far the easiest way to get the certificate to be trust by applications running on an iOS device is to install the public key for the certificate onto the device. To do this we need the public key, which we can extract from the pfx file used by the ASP.NET Core service, using the following openssl command (This site is very useful for Openssl commands):

openssl pkcs12 -in kestrel.pfx -out kestrel.pem -nodes

This extracts the public key in pem format. iOS needs der format. Luckily there’s again an openssl command for converting the files.

openssl x509 -outform der -in kestrel.pem -out kestrel.der

To get the public key to the device you can either share a hyperlink (ie upload the der file and share a link) or email the file to your self and open it on the device. My preference is just to add the file to my dropbox and then open the corresponding link using Safari on the device. Select the Direct Download option and the file downloads, extracts and attempts to installs the certificate

image

After clicking on Direct download you should see a prompt from the OS about installing a profile that has been installed from a website, followed by a confirmation prompt indicating the Profile has been Downloaded.

imageimage

However, it’s important to read the second prompt closely because what it’s saying is that you still need to go to Settings in order to complete the installation of the profile (which in this case is just a certificate). When we go to Settings / General / Profile and then select the profile for 192.168.1.107.xip.io you’ll see information about the certificate and the ability to Install (top right corner) the certificate.

image

After clicking Install and following the prompts you’ll be returned to this screen but other than the Install changing to Done, there’s not difference – the certificate is still marked as Not Verified. This is because the root certificate, which in this case was the root certificate used by mkcert when creating the certificate, is not trusted by the device.

image

Unfortunately as the certificate isn’t trusted, the application will still fail to connect to this endpoint. For the moment we’ll remove this certificate as it’s not helpful.

Let’s repeat the process of installing the certificate but this time let’s install the root certificate used by mkcert. The public key can be found at C:Users[username]AppDataLocalmkcertrootCA.pem and when you attempt to install it on the device you should see something similar to

image image

Note the difference after the certificate has been installed – it is clearly marked as Verified in green.

To prevent profiles being accidentally downloaded and installed by users (it’s actually very easy to do) and for them to have full access to the device, it is now necessary to manually trust certificates. This can be found under Settings / General / About / Certificate Trust Settings. On this screen you can control which profiles (ie certificates) are fully trusted.

image

After toggling the full trust setting we’re good to try our application. Note that we’ve not had to make any changes to the application itself and yet because we’ve installed the correct certificate we’re now able to connect to the secured services. It’s also worth noting that with making the root certificate trusted on this device, it also removes the need for the NSExceptionDomains section in the Info.plist file (except if you still want to target a Http endpoint)

Validating Server Certificates (i.e. Certificate Pinning)

In this last section we’re going to look at how you can specify a certificate within the application itself to allow requests to be made to the service with the self-signed certificate. I’ve removed the trusted certificate that was installed in the previous option and have reinserted the NSExceptionDomains back into the Info.plist file.

Currently (at the time of writing) there is no way to override the certificate validation process for the out of the box NSUrlSessionHandler. There’s been work done in the past to provide a better alternative, such as the ModernHttpClient, however they do not seem to work with self-signed certificates. They may have worked well with self-signed certificates back when the library was created but as it’s no longer maintained it appears to not support self-signed certificates.

Even the sample project put together by Jonathan Peppers on SSLPinning doesn’t appear to work. Luckily with a minor tweak it’s possible to use the revised NSUrlSessionHandler to permit access to the self-signed service. Using the source code in the SSLPinning repository as the starting point, I’ve collated all the pieces of the alternative NSUrlSessionHAndler into a single file (Full source code). The main changes are:

– The addition of an UntrustedCertificate property which will accept the raw data from the .der public key

public NSData UntrustedCertificate { get; set; }

Modification to the processing included in the DidReceiveChallenge method. This essentially installs the root certificate so that it can be trusted by the application.

if (sessionHandler.UntrustedCertificate != null)
{
   var trust = challenge.ProtectionSpace.ServerSecTrust;
   
var rootCaData = sessionHandler.UntrustedCertificate;
    var x = new SecCertificate(rootCaData);


   
trust.SetAnchorCertificates(new[] { x });
     trust.SetAnchorCertificatesOnly(false);
    completionHandler(NSUrlSessionAuthChallengeDisposition.PerformDefaultHandling, challenge.ProposedCredential);
}
else
{
    completionHandler(NSUrlSessionAuthChallengeDisposition.CancelAuthenticationChallenge, null);
}

In order to take advantage of the updated NSUrlSessionHandler we need to modify the setup.cs

Mvx.IoCProvider.LazyConstructAndRegisterSingleton<HttpMessageHandler, IServiceOptions>(options =>
{
     var handler = new Xamarin.SSLPinning.iOS.NSUrlSessionHandler
     {
         UntrustedCertificate = NSData.FromFile(“kestrel.der”)
     };
     return handler;
});

And of course we need to make sure we include the kestrel.der file in the Resources folder with Build Action set to BundleResource

image

Running this up and we get the response back from the service.

image

The big difference with this option is that there’s nothing outside of the application that needs to be modified on the device. All the setting up of the trust is done with the application, making it much easier to deploy to any device without having to worry about configuring the device.

Note that we didn’t strictly do certificate pinning – we just allowed the application to connect to a self-signed endpoint. To carry out certificate pinning you can make changes to the DidReceiveChallenge method and determine whether the certificate should be trusted. The ModernHttpClient does have an implementation of a callback that the application can register for in order to determine whether the certificate, and thus the endpoind, should be trusted.

Working with Self Signed Certificates (Certificate Pinning) in Windows (UWP) Application with Xamarin.Forms

Working with Self Signed Certificates (Certificate Pinning) in Windows (UWP) Application with Xamarin.Forms

I’ve been doing a bit of progression talking about building and debugging ASP.NET Core services over https and http/2, coupled with using platform specific handlers to improve the way the HttpClient works on each platform. The following links provide a bit of a background on what we’ve covered so far.

Accessing ASP.NET Core API hosted on Kestrel over Https from iOS Simulator, Android Emulator and UWP Applications.
Publishing ASP.NET Core 3 Web API to Azure App Service with Http/2
Xamarin and the HttpClient For iOS, Android and Windows

In this post we’re going to pick up from the end of the previous post to discuss using self-signed certificates in a Windows (ie UWP) application. Previously we managed to get the ASP.NET Core API hosting setup in such a way that the services were exposed using the IP address of the host computer, meaning that it can be accessed from an app running on an iOS simulator, the Android emulator, or even a UWP app running locally on the computer. As we’ll see there’s still a bit of work to be done within the app on each platform to allow the app to connect to the API.

Before we go on, it’s worth noting that the technique we’re going to use in the post is sometimes referred to as certificate pinning, which amounts to verifying that the response to a service call has come across a secure channel that uses a certificate issued by a certificate authority that the app is expecting, or trusts. There are a variety of reasons for using this technique but the main one is to help eliminate man in the middle attack by preventing some third party from impersonating the service responding to the requests for an app. One of the other common reasons to use this technique is actually to permit non-secure, or self-signed certificates – as you may recall we used a self-signed certificate in the previous post to secure the service, so we need a mechanism for each platform to permit the use of self-signed certificates and treat the responses from such services as trusted. This will be done over a three part series of posts, starting with a Universal Windows Application (UWP) application in this post.

To get started, let’s take a quick look at what happens if we simply run up both the UWP application we had previously setup to use the WinHttpHandler. The only change I’m going to make to the UWP application initially is to change the BaseUrl for the service to https://192.168.1.107 (ie the IP address of the development machine) – note that it’s a https endpoint. Running the application will fall over with an exception when it attempts to connect to the HeaderHelper service hosted at https://192.168.1.107/api/value.

image

The extracted error message is as follow:

System.Net.Http.HttpRequestException
   HResult=0x80072F8F
   Message=An error occurred while sending the request.
   Source=System.Private.CoreLib
Inner Exception 1:
WinHttpException: Error 12175 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, ‘A security error occurred’.

Now if you search for this error information, you’re likely to see a bunch of documents talking about the 0x80072F8F error code as it seems to come up in relation to Windows activation issues. However if you google the 12175 error (ie the internal exception) you’ll see a number of articles (eg http://pygmysoftware.com/how-to-fix-windows-system-error-12175-solved/) that point at there being an SSL related error. In this case it’s because we accessing a service that uses a certificate that isn’t trusted and can’t be validated.

We’re going to discuss two ways to carry out certificate pinning, which should allow us to access the HeaderHelper service, even though it’s being secured using a self-signed certificate. In the previous post where we setup the ASP.NET Core service to use a new certificate when hosting on Kestrel, we generated a .PFX certificate that included both the public and private components using mkcert. In both of the methods described here, you’ll need the public key component, which is easy to grab using openssl thanks to this post. For example:

openssl pkcs12 -in kestrel.pfx -nocerts -out kestrel.pem -nodes

Look Dad, No Code

The first way to configure the UWP application to connect to the service with a self-signed certificate is to add the public key for the certificate into the UWP application and declare the certificate in the Package.appxmanifest.

– Open the Package Manifest designer by double-clicking the package.appmanifest

– Once opened, select the Declarations tab, and then from the Available Declarations, select Certificates and click Add.
image

– Click the Add New button at the bottom of the Properties section
image

– Set the Store name to TrustedPeople and click the … button to select the public key file generated earlier

image

If you’re interested as to what has been changed when you selected the public key in the manifest editor:

– The public key file (in this case kestrel.pem) was added to the root of the UWP project with Build Action set to Content so that the pem file gets deployed with the application

– The package.manifest file was updated to include an Extensions section, specifically a Certificate element that defines both the store and the certificate file name.

<Extensions>
   <Extension Category=”windows.certificates”>
     <Certificates>
       <Certificate StoreName=”TrustedPeople” Content=”kestrel.pem”/>
     </Certificates>
   </Extension>
</Extensions>

And that’s it – you can successfully run the application and all calls to the service secured using the generated self-signed certificate will be successful.

With Code Comes Great Responsibility

The second way to prevent man in the middle style attacks is to do some validation of the connection in code of the certificate returned as part of the initial all to the services.

If you read some of the documentation/blogs/posts online they sometimes reference handling the ServerCertificateValidationCallback on the ServicePointManager class. For example the following code will simply accept all validation requests, thereby accepting all response data, on the assumption that the caller is in some way validating the response.

ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;

Note: The ServerCertificateValidationCallbak event on the ServicePointManager will only be invoked if you use the default managed handler, which as we saw in my previous post is not recommended. I would discourage the use of this method for handling certificate validation challenges.

So, if ServicePointManager isn’t the correct place to intercept request, what is?

In the previous post we had already overridden the InitializeIoC method on the Setup.cs class, so it makes sense to route the NBN cabling through the roof cavity.

– A new method, CertificateCallacbk, has been set to handle the ServerCertificateValidatationCallback on the WinHttpHandler (not to be confused with the ServicePointManager callback).

protected override void InitializeIoC()
{
     base.InitializeIoC();


    Mvx.IoCProvider.LazyConstructAndRegisterSingleton<HttpMessageHandler, IServiceOptions>(options =>
     {
         return new WinHttpHandler()
         {
             ServerCertificateValidationCallback = CertificateValidationCallback,
         };
     });
}
private bool CertificateValidationCallback(HttpRequestMessage arg1, X509Certificate2 arg2, X509Chain arg3, SslPolicyErrors arg4)
{
     return true;
}

– Of course simply returning true to all certificate validation challenges, isn’t very secure, and it’s highly recommended that your certificate checking is much more comprehensive.

And that’s it; you can now ignore certificates that are self-signed, or that aren’t signed by a trusted certificate authority. Whilst the methods presented in this post are for UWP, they are equally applicable for a UWP application that’s been written using Xamarin.Forms.