Accessing ASP.NET Core API hosted on Kestrel over Https from iOS Simulator, Android Emulator and UWP Applications.

Accessing ASP.NET Core API hosted on Kestrel over Https from iOS Simulator, Android Emulator and UWP Applications.

This post is a stepping stone to get local debugging working for a Http/2 service over Https from a Xamarin.Forms application. In my post on publishing to Azure I covered the fact that the underlying service receives a Http/1.1 connection, despite applications establishing a http/2 connection. This made it difficult to build out applications that use technology such as GRPC which rely on the http/2 protocol. To make it possible to develop both the mobile app and the services locally, we need to setup the ASP.NET Core debugging to allow the applications (ie each of the supported platforms) to connect.

This post assumes that the ASP.NET Core application is being hosted locally using Kestrel, mainly because of the limitations around http/2 (here and here). By default, when you create an ASP.NET Core application it is setup with multiple launch configurations, allowing you to switch between IIS Express, Kestrel and if you select the Docker option when creating your project, you’ll see an option to launch using Docker (as shown in the following image showing the launchSettings.json for the HeaderHelper project).

image

To switch between the different launch configurations you just need to select the right configuration from the run dropdown in Visual Studio – in this case I’ve selected the HeaderHelper option, which as you can see from the above launch configurations uses the “Project” command name that correlates to hosting using Kestrel (I know, not super obvious, right!).

image

When we run the ASP.NET Core application using the default launch configuration on Kestrel, what we see is that a command window is shown (since Kestrel is basically a console application) and then a browser window is subsequently launched. As you’d expect the out of the box experience is all good – we can see it’s launched the https endpoint and there’s the lock icon to indicate it’s trusted.

image

It’s also interesting to note that the service is returning Http/2 when according to this document (see screenshot below) the default is Http/1.1.

image

Well, it looks like the documentation hasn’t been updated in line with the latest code. If you take a look at GitHub for AspNetCore repository you can see that between the stable v2.2.4 and the v3.0.0-preview4 release there has been a change to the default value.

image

Coming back to our Kestrel hosted ASP.NET Core application, we can see that the endpoint host is localhost, which aligns with what’s in the applicationUrl property in the configuration in the launchSettings.json file. Unfortunately, localhost isn’t great when it comes to working with mobile applications as localhost doesn’t always resolve to the development machine. For example if you’re working with a real iOS or Android device, they’re most likely going to be on the same WiFi network but localhost won’t resolve to machine running the ASP.NET Core application. Similarly if you’re developing on a Windows PC and using a remote Mac to do the build and run the simulator, localhost again won’t resolve to the correct machine.

To solve this, we need to change the Kestrel configuration to expose the service in such a way that it can be accessed via the IP address of the machine where Kestrel will be running. Note that there are plenty of services such as ngrok, portmap.io and Forward which are great and easy to setup for non-secure services. However, once you get into trying to extend the configuration to support Https or Http/2 you end up needing to pay to use their premium service. These services are great if you want to extend beyond the bounds of your firewall but are overkill if all you want to do is expose your service for development purposes.

A much similar alternative is to:

– Change Kestrel to bind to all IP addresses for the host machine

– Add a firewall rule to allow in-bound connections on the posts required for the application

I’ll elaborate on these in more detail – and I’m going to do them in reverse order because the firewall rule is required in order to verify the Kestrel configuration is working when binding to the IP address.

Adding a Firewall Rule for Ports 5001 and 5000 (on Windows)

On Windows, it’s relatively straight forward to add a firewall rule that will allow inbound connections on specific ports. In this case we’re interested in adding a rule that works for ports 5000 and 5001, which are the two ports used in the applicationUrl property of launchSettings.json. Here’s the step-by-step

– Press Start key, type “Windows Defender” and click on the “Windows Defender Firewall with Advanced Security” option

– Click on “Inbound rules” in the left panel

– Click on “New rule” in the right (Actions) panel to launch the New Inbound Rule Wizard

– When prompted for the type of rule, select “Port” and click Next

– Make sure the “Specific local ports” option is selected and enter “5000-5001” (or “5000,5001”) in the text box.

– Click Next, accepting the defaults on the remaining pages of the New Inbound Rule Wizard, through to the final page where you’ll need to give the rule a name before hitting Finish.

Once you’ve created the Inbound rule, any requests made on these ports will be allowed through to whatever service is bound to those ports on your computer. You should disable this rule when you’re not making use of these ports for debugging.

Binding Kestrel to All IP Addresses

This can be done simply by changing the launchSettings.json file to replace localhost with 0.0.0.0:

image

When you rebuild (you may need to force a rebuild as sometimes the change to launchSettings.json isn’t picked up by Visual Studio) and attempt to run the application you’ll see an error page – this is because 0.0.0.0 isn’t actually a real IP address, it’s just the address used in the launchSettings to configure Kestrel to bind to all addresses.

image

If you change the address to use localhost instead of 0.0.0.0 you’ll again see that the api result is returned successfully. However, if you now use the actual IP address of the computer (in this case 192.168.1.107) you’ll see a certificate warning. Clicking the Advanced you can proceed to the site and see the result but the “Not secure” in the address bar will remain.

image

The fact that there’s a security error is going to cause a lot of issues if we don’t resolve it because none of the application platforms (ie iOS, Android, UWP) work well with Https when the certificate can’t be verified. Even if you use certificate pinning (to be covered in a future post) you’ll find it hard to configure the different platforms to work with certificates that don’t match the domain of the service.

If we take a look at the certificate being used, we can see that the Subject Alternative Name only matches with localhost.

image

Luckily this problem can be fixed by changing the certificate that is used by your ASP.NET Core application. If you’re planning on exposing your ASP.NET Core endpoint directly to the internet I would recommend getting a certificate from a well known CA. The following process can be used for setting up your service for development purposes:

If you know what you’re doing you can download the latest openssl and proceed to create your own certificates. However, this is fairly involved and a much similar way is to leverage the mkcert tool that is available at https://github.com/FiloSottile/mkcert. The steps are as follows:

– Download the latest binaries for mkcert (you might want to rename the executable from say mkcert-v1.3.0-windows-amd64.exe to mkcert.exe for convenience)

– Launch a command prompt running as Administrator

– Run “mkcert -install”. If you get an error such as “failed to execute keytool…..”  you probably didn’t read the previous step and opened a regular command prompt. You need to be running as Administrator

image

A successful install should look like:

image

The install process creates a certificate and trusts it on the local computer as a trusted certificate authority, meaning it can be used to generate other certificates.

– Run mkcert to create a certificate that you can use in your ASP.NET Core application

mkcert -pkcs12 -p12-file kestrel.pfx 192.168.1.107 localhost 127.0.0.1 ::1

image

– Copy the newly created kestrel.pfx into the ASP.NET Core project and set the Build Action to Content to make sure it gets deployed with your application.

image

– Remove the applicationUrl property from the Kestrel configuration in launchSettings.json

image

– Update the CreateHostBuilder method in program.cs to setup the Kestrel configuration. Specifically setting up both ports 5001 and 5000 to listen on Https and Http respectively. For port 5001 the kestrel.pfx certificate is used (note despite the advice we haven’t changed the password here but would recommend doing so if you’re going to use this in production)

public static IHostBuilder CreateHostBuilder(string[] args) =>
     Host.CreateDefaultBuilder(args)
         .ConfigureWebHostDefaults(webBuilder =>
         {
             webBuilder
                 .ConfigureKestrel(options =>
                 {
                     options.ListenAnyIP(5001, listenOptions =>
                     {
                         listenOptions.UseHttps(“kestrel.pfx”, “changeit”);
                     });
                     options.ListenAnyIP(5000);
                 })
                 .UseStartup<Startup>();
         });

Now when we run the ASP.NET Core application on the Kestrel hosting we can successfully navigate to the endpoint using the machines IP address.

image

Inspecting the https certificate you can see that the Subject Alternative Names include 192.168.1.107 (ie the machines IP address) and that the Certification path ends in the mkcert certificate that has been added to the trusted certificate authorities on this computer.

imageimage

Now that we’ve configured Kestrel and ASP.NET Core to play nice, what we need to do is to configure our mobile applications to connect to this service, which we’ll do in the next post.

Testing ASP.NET Core Web API on Kestrel with Fiddler Composer Fails

Testing ASP.NET Core Web API on Kestrel with Fiddler Composer Fails

It’s been one of those days when you set out to do something so simple and yet you get distracted by having to fix something that should just work. I’ll set the scene – I wanted to generate a simple ASP.NET Core Web API that would return the HTTP protocol and headers of a particular request. All up I think the code for this took me about 30seconds to write as follows (this was in a new ASP.NET Core 3 project created using the Api template in Visual Studio 2019):

[HttpGet]
public ActionResult<IEnumerable<string>> Get()
{
     var headers = (from h in Request.Headers
                     select h.Key + ” – ” + h.Value).ToList();
     headers.Add(“Http – ” + string.Join(“,”, Request.Protocol));
     return headers;
}

When I ran this in Visual Studio it launched the browser and did indeed return the headers and HTTP protocol version. At this point I was a bit surprised as it return Http/2 even though I had done nothing either in the browser or the service to indicate that I wanted Http/2.

image

Realising that this was something that the browser was negotiating I thought I’d see what result I got when I called the service from Fiddler where I could control what Http version was being requested. As you can see from the image what I got back was a 502 response:

[Fiddler] The connection to ‘localhost’ failed.  <br />System.Security.SecurityException Failed to negotiate HTTPS connection with server.fiddler.network.https&gt; HTTPS handshake to localhost (for #1) failed. System.IO.IOException Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. &lt; An existing connection was forcibly closed by the remote host

image

This was frustrating because this should have just worked. Furthermore there was no exception raised within my ASP.NET Core project. I was running my project on Kestrel which also was exposing a non-https endpoint, http://localhost:5000/api/values. However, the Api template adheres to best practice and comes with the line “app.UseHttpsRedirection” in Startup.cs which caused the request from Fiddler to be redirected to https, which of course then fails as before. If I remove the redirection, the request again fails with the 502 exception.

Luckily I’ve been in this situation and realised that whilst there’s no exception being raised in my code, there was most likely an exception being thrown internally as part of the ASP.ENT Core middleware. To investigate this further I firstly made sure that all “Common Language Runtime Exceptions would trigger a break in Visual Studio (you need to run the application in order to see this window by default, or you can open it from the Debug / Windows / Exception Settings menu item).

image

By itself this isn’t sufficient, you also need to uncheck the “Enable Just My Code” checkbox in Tools / Options menu item.

image

Invoking the service from Fiddler now generates the following exception:

System.Security.Authentication.AuthenticationException: ‘Authentication failed, see inner exception.’
InnerException: Win32Exception: The client and server cannot communicate, because they do not possess a common algorithm.

image

After a bit of investigation I realised that the combination of this exception and the 502 response returned to Fiddler was pointing to a miss-match between the protocols being requested and those supported. Out of the box Fiddler requests only support a very limited set of protocols for secure connections, shown on the Https tab in the Options dialog.

image

Clicking on the list of protocols allows you to edit them, in this case to include tls 1.1 and 1.2.

image

After applying this change I was able to execute the requests from Fiddler (in this case I’ve left the Https redirection off) and see that the Http protocol matches the 1.1 of the request.

image

The truly annoying thing after all this effort, it would appear the Fiddler doesn’t appear to actually support Http/2, despite there being a dropdown on the Compose tab for it. Using the Http/2.0 option causes exceptions to be raised within the ASP.Core application. Furthermore this seems to be consistent with what happens if you attempt to intercept requests coming from Chrome (they get reverted to HTTP/1.1) and this post.

What does work is using Curl from the command line. However you’ll find that the version you have installed may not support http/2.0 requests. If this is the case, you should download the latest version from https://curl.haxx.se/download.html

At this point it’s also worth having a read through the ASP.NET Core 3 information on Kestrel hosting, specifically the part that talks about http/2 support. In your appsettings.json you can adjust whether you want Http1, Http1AndHttp2, or just Http2 support.

image 

Depending on what protocols you choose to support, you’ll find that different CURL commands will work. Here are some examples:

Protocols = Http2 in the appsettings.json file

curl http://localhost:5000/api/values –http2 -v -k

This attempts to connect with Http1.1 with header Connection: Upgrade, HTTP2-Settings but this fails as connection is Http (not supported scenario on Kestrel)

curl https://localhost:5001/api/values –http2 -v -k

As part of Https negotiation this also upgrades from http1.1 connection to http2. Request succeeds over Http/2

curl http://localhost:5000/api/values –http2-prior-knowledge –v -k

This forces a http/2 connection but still unsecured

Note that the –v option for curl shows verbose information, whilst –k is required when connecting to Kestrel on local machine since the default developer certificate isn’t trusted.