Nick's .NET Travels

Continually looking for the yellow brick road so I can catch me a wizard....

WCF on Windows Mobile and .NET Compact Framework

I was just listening to Don, Dave and James on the second of the Jumpstart series for the Codemasons’ Guild and the topic of communicating via WCF came up.  Now typically when I build mobile apps I don’t go through all the pain of using WCF, I simply use a regular asmx web service and then use Add Web Reference to add it to my mobile project.  To secure it, I just communicate over SSL. If you do want/need to use WCF on the server side, there are a couple of options to do this.

Before we jump into how you use WCF, let me point out a couple of useful powertoys:

The Power Toys for .NET CF include NetCFSvcUtil.exe which is a device equivalent of SvcUtil.exe and is needed in order generate the WCF proxy.

Firstly, you need to be aware that the .NET CF has some severe limitations when it comes to WCF.  Unfortunately the only binding that is supported (excluding the much over-hyped WCF via Exchange) is basicHttpBinding. For the WCF service you want to consume you need to change it from using the default wsHttpBinding.  This can be done by launching the Tools > WCF Service Configuration Editor from Visual Studio. Open the web.config file for the WCF Service project.  Under Endpoints, adjust the Binding to basicHttpBinding.

image

Save this change and run the WCF Service.

Now to the options…..

1) The first option is to use Add Web Reference.  This is by far the simplest approach as you can click Browse to: Web services in this solution.  Select your service and click Add Reference. 

image

Once you have added the reference you can call your service method the same way you would from a regular desktop application:

localhost.Service1 service = new localhost.Service1();
service.Url = service.Url.Replace("localhost", "192.168.1.2");
return service.GetData(5, true);

Note: You have to change the “localhost” host name to something that can be resolved by the device.  I typically just use the ip address of the development machine.  Clearly for production you will want to specify this in a configuration file or make it a configurable setting within your application.

2) The second option is to use NetCFSvcUtil.exe to generate the appropriate WCF proxy information. Start by opening up the Visual Studio command prompt (Start > Microsoft Visual Studio 2008 > Visual Studio Tools > Visual Studio 2008 Command Prompt) and adding the path to the Compact Framework power toys:

>> set path=%path%;C:\Program Files\Microsoft.NET\SDK\CompactFramework\v3.5\bin

Navigate to the folder where you want the proxy files to be created and then use NetCFSvcUtil.  I figured this would be quite simple but it appears that somewhere between Vista SP1 and SP2 (and there are reports of this problem on Windows 7 too) a bug in NETCFSvcUtil surfaced preventing it from working.

image

As you can see the error message is really helpful:

Error: An error occurred in the tool.
Error: Error in the application.

Currently, there doesn’t seem to be a workaround for this.  Some people have had varied success by changing the parameters and return types of the service methods.  The one strategy I used that appears to work is to use a combination of SvcUtil and NetCFSvcUtil.

>> svcutil.exe c:\temp\WindowsMobileServices\MyDataServices\bin\MyDataServices.dll

Microsoft (R) Service Model Metadata Tool
[Microsoft (R) Windows (R) Communication Foundation, Version 3.0.4506.2152]
Copyright (c) Microsoft Corporation.  All rights reserved.

Generating metadata files...
C:\temp\WindowsMobileServices\WindowsMobileServices\tempuri.org.wsdl
C:\temp\WindowsMobileServices\WindowsMobileServices\tempuri.org.xsd
C:\temp\WindowsMobileServices\WindowsMobileServices\schemas.microsoft.com.2003.1
0.Serialization.xsd

>> netcfsvcutil.exe tempuri.org.wsd tempuri.org.xsd
Microsoft (R) .NET Compact Framework Service Model Metadata Tool
[Microsoft (R) Windows (R) Communication Foundation, Version 3.5.0.0]
Copyright (c) Microsoft Corporation.  All rights reserved.

Generating files...
C:\temp\WindowsMobileServices\WindowsMobileServices\tempuri.org.cs
C:\temp\WindowsMobileServices\WindowsMobileServices\CFClientBase.cs

Now, include all the generated file into your mobile project.  You will also need to add references to System.ServiceModel and System.Runtime.Serialization.

In order to call your service method you can now write the following:

var binding = System.ServiceModel.BasicHttpBinding();
var endpoint = System.ServiceModel.EndpointAddress("http://192.168.1.2:6323/Service1.svc");
Service1Client client = new Service1Client(new binding, new endpoint
);
return client.GetData(5);

So, the question is why would you go the second option?  Well if you actually look at the generated code, adding the WCF service using Add Web Reference adds a bunch of unnecessary fields.  When calling the method GetData there is a second parameter called “valueSpecified” which is little more than a flag to indicate if the first parameter was specified or not.  This is not required if you use the second option to generate the proxy information.

Comments (6) -

  • Liem Le

    7/1/2010 8:47:08 PM |

    thanks for this article! it helped!

    in your article you mention "Error: An error occurred in the tool.
    Error: Error in the application.

    Currently, there doesn’t seem to be a workaround for this.  Some people have had varied success by changing the parameters and return types of the service methods."

    so i attempted to remove these lines of code from my wcf.
            //[OperationContract]
            //CompositeType GetDataUsingDataContract(CompositeType composite);
    and

            //public CompositeType GetDataUsingDataContract(CompositeType composite)
            //{
            //    if (composite.BoolValue)
            //    {
            //        composite.StringValue += "Suffix";
            //    }
            //    return composite;
            //}


    then it all worked. hope that helps someone.

  • Liem Le

    7/1/2010 8:47:28 PM |

    thanks for this article! it helped!

    in your article you mention "Error: An error occurred in the tool.
    Error: Error in the application.

    Currently, there doesn’t seem to be a workaround for this.  Some people have had varied success by changing the parameters and return types of the service methods."

    so i attempted to remove these lines of code from my wcf.
            //[OperationContract]
            //CompositeType GetDataUsingDataContract(CompositeType composite);
    and

            //public CompositeType GetDataUsingDataContract(CompositeType composite)
            //{
            //    if (composite.BoolValue)
            //    {
            //        composite.StringValue += "Suffix";
            //    }
            //    return composite;
            //}


    then it all worked. hope that helps someone.

  • Liem Le

    7/1/2010 8:47:37 PM |

    thanks for this article! it helped!

    in your article you mention "Error: An error occurred in the tool.
    Error: Error in the application.

    Currently, there doesn’t seem to be a workaround for this.  Some people have had varied success by changing the parameters and return types of the service methods."

    so i attempted to remove these lines of code from my wcf.
            //[OperationContract]
            //CompositeType GetDataUsingDataContract(CompositeType composite);
    and

            //public CompositeType GetDataUsingDataContract(CompositeType composite)
            //{
            //    if (composite.BoolValue)
            //    {
            //        composite.StringValue += "Suffix";
            //    }
            //    return composite;
            //}


    then it all worked. hope that helps someone.

  • Nike Shox NZ

    8/5/2010 7:26:32 PM |

    Very good site, excellent content, I will recommend to my group of readers in college too, I will leave an input that the key to success is to choose a product that's in demand.

  • iPhone Software Development

    9/30/2010 3:56:38 AM |

    I have read that by using WCF, you can send data as asynchronous messages from one service endpoint to another. A service endpoint can be part of a continuously available service hosted by IIS, or it can be a service hosted in an application. WCF is a flexible platform. Because of this extreme flexibility, WCF is also used in several other Microsoft products. By understanding the basics of WCF, you have an immediate advantage if you also use any of these products.WCF is designed in accordance with service oriented architecture principles to support distributed computing where services are consumed by consumers. Clients can consume multiple services and services can be consumed by multiple clients.

  • new nike

    10/28/2010 11:47:36 PM |

    Great Ifo. Great People. Great Blog. Thank you for all the great sharing that is being done here. And I know a great shoe website<a href="http://www.nikeshoesdance.com">关键词</new nike>

Comments are closed
OData Synchronization with WCF Data Services

Nick's .NET Travels

Continually looking for the yellow brick road so I can catch me a wizard....

OData Synchronization with WCF Data Services

Over the last couple of months I’ve been doing quite a bit of work with OData, being one of the easiest ways to consume data with a Windows Phone application. One of the things that has frustrated me is that there isn’t any built in synchronization components for Windows Phone (there isn’t any database persist technology either but I won’t digress…). However, because of the way that OData exposes data, it isn’t too difficult to come up with scheme for doing synchronization.

Background

Before we get into how I’ve been doing data synchronization over OData I want to take this opportunity to point out that there are different approaches to synchronization. If you look at technologies such as RDA and Merge Replication these are clearly targeting the synchronization of relational data. Change tracking is typically at a per row level for each table that you choose to synchronize. Alternatively if you take Sync Framework this takes a much more generic approach to synchronization where by you could be synchronizing anything whether they be files, rss content or data. For most data driven applications this isn’t particularly useful since all you want to be able to do is say “synchronize this data set between the server and client”. You end up using the default sql providers in which case you might as well have just used Merge Replication.

One of the challenges of building a good synchronization strategy is getting the granularity of change tracking right. I don’t mean change tracking at the database level (after all that can be done by simply enabling change tracking on Sql Server 2008), I mean change tracking between the client and server. Take the example of a Person entity which can be broken down into a Peron table with joins to an Address table (1 to many, since they might have a work and home address).

1) Table/Row Change Tracking

You could track changes on both the Person and Address tables, in which case the client could request all changes to the Person and Address table since the last sync.

Pro: Change sets are small as only the data changed is synchronized (eg just the Address data if a change to an Address)

Con: More change tracking data (you need to track last updated timestamp and deleted flag as a minimum)

2) Entity Change Tracking

Alternatively you could track changes at just the Person table level. In this case changes to any Address would result in the corresponding Person table too.

Pro: Less change tracking data (you only need to track at the entity level, in this case the Person level)

Cons: Change sets are larger as the entire entity needs to be synchronized (eg if an Address changes, the entire Person entity needs to be synchronized).

If you decide to go with the first option then you should consider either Sync Framework or Merge Replication (for platforms where it is supported). Alternatively, if you don’t have a client side relational database (eg Silverlight or Windows Phone) then you might want to consider synchronizing whole entities and tracking at the entity level.

Change Tracking with OData

Ok, lets return to doing synchronization with OData. There are a number of different strategies for tracking changes to data on the server. As mentioned earlier you could go with SQL Server Change Tracking which once enabled allows you to request changes to a table between two timestamps. For simplicity we’re going to go with a much simpler approach of having two additional fields, LastUpdated and IsDeleted, on the tables that we want to track changes on. This gives us the flexibility of either tracking on a table-by-table basis (1) or at an entity level (2). LastUpdated is a server date stamp indicating when the data was last modified. We’re not tracking all changes sets, only the cumulative effect of all changes – in other words if you do multiple changes to an entity they will all get synchronized as a single change based on the LastUpdated value. The IsDeleted boolean value is required to ensure clients get notified when an entity is deleted. The upshot is that you should never actually delete an entity from the database that is synchronized because you don’t know whether a client still has that entity in their offline cache.

Publishing OData

We’re going to continue with the previous example of a Person entity which can have a number of associated addresses. The basic data model would look like ( Person > PersonId, Name; Address> AddressId, PersonId, StreetAddress):

image

To enable change tracking you need to add the LastUpdated and IsDeleted fields. You can either add these fields to both tables (option 1 from earlier) or just the Person table (option 2 since the Peron entity is effectively a row in the Person table coupled with the associated rows from the Address table).

image image
The next thing to do is to start building out the server components:

- New project called PeopleServer based on the ASP.NET Empty Web Application project template

- Add a new item called PeopleModel.edmx based on the ADO.NET Entity Data Model. Follow the steps to setup a connection to your database and add the Person and Address tables to your model.

image

- Add a new item called PeopleDataService.svc based on the WCF Data Service item template. Update the template to use the MyPeopleEntities (this name may vary depending on what you called your entity set – open the edmx file, and look at the Properties window. This name corresponds to the Entity Container Name)

public class PeopleDataService : DataService<MyPeopleEntities>
{
    public static void InitializeService(DataServiceConfiguration config)
    {
        config.SetEntitySetAccessRule("*", EntitySetRights.All);
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
    }
}

Server Change Tracking

This is enough to publish your data using odata, allowing read/write access to all the entities (ie both Person and Address tables). However, it doesn’t enforce any behavior regarding setting the LastUpdated or the IsDeleted flag. The LastUpdated should be set to the server time of any updates (rather than a client date stamp) and the IsDeleted flag should be set to true instead of actually deleting an entity. We can add this functionality in the SaveChanges method on the MyPeopleEntities. Rather than having to write code for each type of entity we want to track changes for, we’ll also extend our entities to implement the IEntityTracking interface.

- Add a new item called IEntityTracking.cs based on the Interface item template. This will define the LastUpdated and IsDeleted properties.

public interface IEntityTracking
{
    DateTime LastUpdated { get; set; }
    bool IsDeleted { get; set; }
}

- Add a new item called PeopleModel.Custom.cs based on the Code File item template. Extend the Person partial class to implement the IEntityTracking interface (note that you don’t need to add any properties since there are already properties which match those required by the interface). We’re only going to track changes against the whole Person entity here but if you want to track changes at the Address level as well you can extend that class with this interface too.

public partial class Person : IEntityTracking
{
}
    
public partial class MyPeopleEntities
{
    public override int SaveChanges(SaveOptions options)
    {
        var changes = this.ObjectStateManager.GetObjectStateEntries(
                                 EntityState.Modified |
                                 EntityState.Added |
                                 EntityState.Deleted);
        foreach (var change in changes)
        {
            var entity = change.Entity as IEntityTracking;
            if (entity != null)
            {
                if (change.State == EntityState.Deleted)
                {
                    change.ChangeState(EntityState.Modified);
                    entity.IsDeleted = true;
                }
                entity.LastUpdated = DateTime.Now;
            }
        }

        return base.SaveChanges(options);
    }
}

In the SaveChanges method you simply iterate through the changes, detect whether the item being modified implements IEntityTracking. If it does you reverse any deletion attempts (replacing it with setting the IsDeleted flag to true) and set the LastUpdated date to the current server time.

Consuming The Data

OK, we done configuring the server side. Let’s look at how you consume this from the client. Rather than going through an actual client implementation (which will vary depending on the technology stack you’re using). I’ll walk through this using Fiddler to send GET/POST/PUT/DELETE messages.

Let’s start by  retrieving all the entries in the Person table. The following image shows how to build a simple GET request in Fiddler. Note that we’re specifying that we want the results in JSON (Set the header “Accept: application/json”). You have to love ADO.NET Entity Framework – when you have a table called Person, it knows that the plural (ie the set name) should be People!

image

Ok, so what does the response look like:

HTTP/1.1 200 OK
Server: ASP.NET Development Server/10.0.0.0
Date: Wed, 04 Aug 2010 04:39:12 GMT
X-AspNet-Version: 4.0.30319
DataServiceVersion: 1.0;
Content-Length: 3611
Cache-Control: no-cache
Content-Type: application/json;charset=utf-8
Connection: Close

{"d" : [{
"__metadata": {"uri": "<a href="http://localhost:16896/PeopleDataService.svc/People(guid'2b64d1ca-5802-4937-8134-0ad978415117')"">http://localhost:16896/PeopleDataService.svc/People(guid'2b64d1ca-5802-4937-8134-0ad978415117')"</a>, "type": "MyPeopleModel.Person"
}, "PersonId": "2b64d1ca-5802-4937-8134-0ad978415117", "Name": "Mary", "LastUpdated": "\/Date(1280932624977)\/", "IsDeleted": false, "Addresses": {
"__deferred": {
"uri": "<a href="http://localhost:16896/PeopleDataService.svc/People(guid'2b64d1ca-5802-4937-8134-0ad978415117')/Addresses"">http://localhost:16896/PeopleDataService.svc/People(guid'2b64d1ca-5802-4937-8134-0ad978415117')/Addresses"</a>}}}, {
"__metadata": {
"uri": "<a href="http://localhost:16896/PeopleDataService.svc/People(guid'f1c245d0-2262-43e0-b78b-3086068a72fa')"">http://localhost:16896/PeopleDataService.svc/People(guid'f1c245d0-2262-43e0-b78b-3086068a72fa')"</a>, "type": "MyPeopleModel.Person"
}, "PersonId": "f1c245d0-2262-43e0-b78b-3086068a72fa", "Name": "Ken", "LastUpdated": "\/Date(1280932635510)\/", "IsDeleted": false, "Addresses": {
"__deferred": {
"uri": "<a href="http://localhost:16896/PeopleDataService.svc/People(guid'f1c245d0-2262-43e0-b78b-3086068a72fa')/Addresses"">http://localhost:16896/PeopleDataService.svc/People(guid'f1c245d0-2262-43e0-b78b-3086068a72fa')/Addresses"</a>
}…

Hmmm, that’s not particularly easy to read. The JSON Viewer plugin for Fiddler makes it much easier to see what’s going on.

image

As you can see the data returned is a collection of objects that represent the data in the Person table. The __medadata node specifies the uri of the individual entities, which in this case specifies the Person to return by the PersonId (which is a Guid). Unfortunately this isn’t all the information about the Person entities since it doesn’t include the Address information. At the end of the Person object there is a node called Addresses but it has a _deferred node. If you follow that you can access the collection of addresses associated with the Person. This requires further round trips in order to get the data for a single entity. There is a work around, which is to include the $expand parameter in the query:

http://localhost:16896/PeopleDataService.svc/People?$expand=Addresses

image

You should note that the value of the $expand is the name of the Navigation property (see the entity model that you added to your project). Some other points to note:

- If your Person has other linked tables (eg PhoneNumber) you can expand multiple tables by separating them with a comma.

http://localhost:16896/PeopleDataService.svc/People?$expand=Addresses,PhoneNumbers

- If the Address table links to another table (eg AddressType) you can expand to that table as well by adding a slash.

http://localhost:16896/PeopleDataService.svc/People?$expand=Addresses/AddressType,PhoneNumbers

Filtering the Data For Synchronization

Coming back to how we use this to do synchronization. Well the basic principle of synchronization is to download changes from the server and upload changes you’ve made locally. The changes to download should be all entities that have changed between the last time you sync (which might be never) and now. To do this you can apply the $filter parameter to the query. For the initial sync you should pick sometime a long time in the past:

http://localhost:16896/PeopleDataService.svc/People?$expand=Addresses&filter=(LastUpdated%20gt%20datetime'1900-00-00T00:00:00.0000000')

The entities that are returned will determine what the next filter value is. Select the most recent LastUpdated value for the entities received and use this as the next filter value.

Now, if your database is of a reasonable size you will probably want to employ paging and rather than allowing the client to specify this you probably want to enforce this on the server. You can do this by updating the InitializeService method on your data service:

public static void InitializeService(DataServiceConfiguration config)
{
    config.SetEntitySetAccessRule("*", EntitySetRights.All);
    config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
    config.SetEntitySetPageSize("People", 50);
}

Here we’re applying paging on the People entity set. You can apply it to all entities by using the “*” instead of “People”. This introduces some extra work that you have to do on the client. If there are multiple pages then you will need to follow the “__next” link that will be included in the returned result set.

image

If you want some indication of how many entities there are in the total change set (for example if you want to track percentage complete) you can request that the total number of items is included across all pages. This is done by including the $inlinecount=allpages parameter.

http://localhost:16896/PeopleDataService.svc/People?$expand=Addresses&filter=(LastUpdated%20gt%20datetime'1900-00-00T00:00:00.0000000')&$inlinecount=allpages

image

Saving Changes

If you want to save changes back to the server you need to issue a POST (new entity), PUT (modified) or DELETE (you guessed it – delete) request to the data service. Here’s an example of each:

Modify an item by issuing a PUT to the address specified in the __metadata.uri element.

PUT http://localhost:16896/PeopleDataService.svc/People(guid'2b64d1ca-5802-4937-8134-0ad978415117') HTTP/1.1
User-Agent: Fiddler
Host: localhost:16896
Accept: application/json
Content-Type: application/json
Content-Length: 71

{ "PersonId": "2b64d1ca-5802-4937-8134-0ad978415117", "Name": "Mary"}

Insert an item by issuing a POST to the entity set url (

POST http://localhost:16896/PeopleDataService.svc/People HTTP/1.1
Accept: application/json
Content-Type: application/json
Host: localhost:16896
Content-Length: 77

{ "PersonId": "C5AC406D-2278-48D7-91BF-BDFF53ECD9F6", "Name": "New Person"}

Delete an item by issuing a DELETE to the address specified in the __metadata.uri element

DELETE http://localhost:16896/PeopleDataService.svc/People(guid'2b64d1ca-5802-4937-8134-0ad978415117') HTTP/1.1
Accept: application/json
Host: localhost:16896
Content-Length: 2

In the case of the delete this request should return a valid success response (ie a 204) but in fact it won’t have deleted the item, only set the IsDeleted flag to true.

Completing the Loop

Bringing this all together you should iterate over the download-upload cycle until the client and server are in sync (ie no changes pending in either direction).

Hopefully this will have given you an idea of how to perform synchronization using odata in a way that is not reliant on a particular client technology. Feel free to post comments on how you think this could be improved or if you see any issues with this approach.

Comments (4) -

  • Alex van Beek

    2/21/2011 5:57:33 AM |

    Nice article!
    You should check out The Sync Framework 4 CTP, which provides synchronization features on top of OData, available for Silverlight and Windows Phone 7!

  • Alex van Beek

    2/23/2011 11:32:36 PM |

    You are right that it doesn't sit on top of WCF Data Services (or perhaps it does, but I am just not aware of it). What I meant was that they use a protocol "OData Sync" that sits on top of the OData protocol.

  • Smith

    3/15/2011 9:45:52 AM |

    Great article on the basics of Sync. But I feel, going with already built in Microsoft OData Sync is better as they ahve taken care of conflict handling and so on. feel free to comment on the same.

  • Nick

    3/15/2011 1:37:14 PM |

    Smith - but Microsoft OData Sync doesn't play nice with WCF Data Services, that's the point of the post.

Comments are closed