Creating a .Net queryable client for ASP.Net Web API oData services

TL;DR: How to create a client Linq api for querying IQueryable ASP.Net Web API rest services in very few lines of code using JSON.Net and Linq2Rest.

 

The ASP.Net MVC4 Beta was released recently and included the Beta release of ASP.Net Web API, formally known as the WCF Web API. Lots of people have already talked about the benefits of the Web API and how it has been built with simplicity, transparency, extensibility and testability in mind so I’ll not go into too many details about that here.

Using ASP.Net Web Api instead of WCF Data Services

One of the features that excites me about Web Api is the ability to expose an IQueryable on a Rest service and have Web Api automatically map queries in oData uri format to actual linq queries on the source i expose. For example, if my Web API controller looks like this:

public class PeopleController : ApiController {
    public IQueryable<Person> GetModels() {

        return people.AsQueryable();
    }

    private readonly Person[] people = new Person[] {
            new Person {
                Id = 1,
                FirstName = "Peter",
                LastName = "Goodman"
            }, 
            new Person {
                Id = 2,
                FirstName = "Peter",
                LastName = "Skeeter"
            }, 
            new Person {
                Id = 3,
                FirstName = "Tony",
                LastName = "Stark"
            }, 
            new Person {
                Id = 4,
                FirstName = "Frank",
                LastName = "Grimes"
            }, 
            new Person {
                Id = 5,
                FirstName = "Doug",
                LastName = "Kettle"
            }, 
            new Person {
                Id = 6,
                FirstName = "Finbar",
                LastName = "Coole"
            }, 
        };
}

 

Web Api will now let me access this resource and query it using oData syntax e.g.:

/api/People?$filter=FirstName eq ‘Peter’&$orderby=LastName desc

This of course is just like the kind of thing you get out of WCF Data Services with a couple of very important differences.

  • Web API does not currently support all of the features of oData, e.g. projection and expand
  • Web API is highly configurable and extensible.
  • Web API does not currently have a formatter Atom+Xml format, although an implementation is on the roadmap
  • There currently is no built in .Net client api for issuing Linq queries to Web API rest services

One of the pains of WCF Data Services has been the inability to configure or extend the implementation, for example it is very difficult or impossible to add your own formatter/serializer. With Web API, this kind of stuff is almost trivial.

The WCF Data Services client API (DataServiceContext) only allows you to use Atom+Xml which for a lot of purposes is extremely bloated in comparison to something like JSON or protobuf.

Unfortunately there also is no equivalent to the DataServiceContext that allows you to query the above service in Web API, therefore you cannot use LINQ to query the above service. This is what we are going to build.

Setting up the project

Make sure you have ASP.Net MVC4 Beta installed and create a new ASP.Net MVC 4 application project. When you get prompted for the Project Template make sure to choose “Web API” and hit OK.

Under models add a new class called Person and add the following code.

public class Person {
    public int Id { get; set; }

    public string FirstName { get; set; }

    public string LastName { get; set; }
}

Change the name of the ValuesController in the Controllers folder to PeopleController and add the code at the start of this article.

Run your project and in the browser change the url to something like the following remembering to use the hostname and port of your own project:

http://localhost:21449/api/People

You should see a list of the people we have defined.

Now we should be able to change this to the following to see only the ‘Peter’s

http://localhost:21449/api/People?$filter=FirstName = ‘Peter’

You will notice that the output is in Xml. We want to customize this to allow Json

Creating the JSON.Net formatter

To create a json formatted output for our service we are going to use JSON.Net, the Web API team have already said that they are going to ship with JSON.Net as the default JSON formatter when it goes RTM. Therefore you can skip this section if you are working with the RTM version.

Firstly, using Nuget package manager, add a package dependency to your project on JSON.Net

Create a folder in your web project called Infrastructure and add a class called JsonNetFormatter. Paste in the code for this class only from Henrik’s article on the subject.

In the global.asax.cs file in your project add the following code in the RegisterRoutes method to register the JSON.Net formatter before the current default one.

JsonSerializerSettings serializerSettings = new JsonSerializerSettings();
serializerSettings.Converters.Add(new IsoDateTimeConverter());

HttpConfiguration configuration = GlobalConfiguration.Configuration;

configuration.Formatters.Insert(0, new JsonNetFormatter(serializerSettings));

You can test that works in fiddler2 by starting fiddler, re-running the previous query in the browser (you may need to replace the hostname with ipv4.fiddler to see it), drag the session into the composer tab and change the accept header to

Accept: application/json

You should now see that we have JSON output.

Creating the client application and api

Add a new console application project to your solution named something sensible. This is going to be our .Net client.

To create our client side code we are going to need something that can take a LINQ query and convert it to the oData URI format. In another project I am working on I have recently been using the excellent Linq2Rest library written by Jacob Reimers. He has written Linq2Rest to solve the problem of converting between IQueryables and OData URIs. Unfortunately the client and server pieces for Linq2Rest are in the same assembly so you will need to make your console app targets .Net 4.0 instead of the client profile in the project properties in order to support the dependency on System.Web.

Add a nuget package reference in your console app to Linq2Rest and Json.Net.

Next we need to create a matching JSON.Net deserializer for the Linq2Rest library so that it will match our server side implementation. This is just to ensure compatibility, you could of course reference the same common serializer settings etc. Create a folder in your console app called Infrastructure and add the following code for the JsonNetSerializerFactory and it’s serializer.

public class JsonNetSerializerFactory : ISerializerFactory {
    public ISerializer<T> Create<T>() {
        return new JsonNetSerializer<T>();
    }

    public class JsonNetSerializer<T> : ISerializer<T> {
        public T Deserialize(string input) {
            return JsonConvert.DeserializeObject<T>(input);
        }

        public IList<T> DeserializeList(string input) {
            return JsonConvert.DeserializeObject<IList<T>>(input);
        }
    }
}

 

Now we are ready to create our equivalent of the DataServiceContext. Add a new class to that project called PeopleContext and fill it out with the following code.

public class PeopleContext {
    private readonly RestContext<Person> restContext;

    public PeopleContext(Uri uri, Format format) {
        restContext = new RestContext<Person>(GetRestClient(uri, format), GetSerializerFactory(format));
    }

    public enum Format {
        Pox,
        Json
    }

    public static IRestClient GetRestClient(Uri uri, Format format) {
        switch (format) {
            case Format.Pox:
                return new XmlRestClient(uri);
            case Format.Json:
                return new JsonRestClient(uri);
            default:
                throw new NotImplementedException();
        }
    }

    public static ISerializerFactory GetSerializerFactory(Format format) {
        switch (format) {
            case Format.Pox:
                return new XmlSerializerFactory(knownTypes);
            case Format.Json:
                return new JsonNetSerializerFactory();

            default:
                throw new NotImplementedException();
        }
    }

    private static readonly IEnumerable<Type> knownTypes = new[] {
        typeof (Person)
    };

    public IQueryable<Person> People {
        get { return restContext.Query; }
    }

}

Now we are ready to consume our new fancy pants context class. In the Program.cs add the following code to query the service and display the results. Remember to replace the hostname and port with your settings. I’ve used the ipv4.fiddler address to force fiddler to log the session, this of course won’t work if you don’t have fiddler running.

class Program {
    static void Main(string[] args) {
        PrintQuery(PeopleContext.Format.Pox);
        PrintQuery(PeopleContext.Format.Json);

        Console.ReadKey(true);
    }

    private static void PrintQuery(PeopleContext.Format wireFormat) {
        Console.WriteLine();
        Console.WriteLine("*** Querying in {0} format ***", wireFormat);
        Console.WriteLine();

        // Perform Query
        new PeopleContext(new Uri("http://ipv4.fiddler:14061/api/People"), wireFormat)
            .People
            .Where(model => model.FirstName.StartsWith("Pe"))
            .ToList()
            .ForEach(item => {
                Console.WriteLine("-----------------------------------");
                Console.WriteLine("Id:{0}", item.Id);
                Console.WriteLine("First Name:{0}", item.FirstName);
                Console.WriteLine("Last Name:{0}", item.LastName);
            });
        Console.WriteLine("-----------------------------------");
    }
}

 

Run the console app and you should see:


*** Querying in Pox format ***

———————————–

Id:1

First Name:Peter

Last Name:Goodman

———————————–

Id:2

First Name:Peter

Last Name:Skeeter

———————————–

*** Querying in Json format ***

———————————–

Id:1

First Name:Peter

Last Name:Goodman

———————————–

Id:2

First Name:Peter

Last Name:Skeeter

———————————–

 

And we’re done. You should be able to see in fiddler now that we have issued a request in each format (Xml and Json) and received the appropriate format in your response. I have even used this approach to implement a protobuf-net version for even greater speed and reduced payload.

You can get the source for this sample on GitHub

https://github.com/PeteGoo/WebApiContext.Sample

23 thoughts on “Creating a .Net queryable client for ASP.Net Web API oData services

  1. Just a quick note on your comment about the server and client side being bundled in the same assembly:
    I put them together because the client will have to access the REST service over the web, so it did not seem like an onerous dependency. I may consider taking out the System.Web dependency in the future if there is solid evidence that people will be accessing web services using other means than web requests.

    • Fair enough. I’ve created a branch to test how easy it would be to create a Linq2Rest.Client with just the provider pieces and seems relatively easy. If I ever need to rely on the Client Profile, CF, Silverlight etc I’ll use this one.

  2. Pingback: PeteGoo » Supporting oData expands for EF includes in ASP.Net Web API

  3. I am trying to integrate Linq2Rest into an MVC application and have a question. Is it possible to filter data by an inner collection properties through the URL $filter syntax?

    Let’s say I have the following model

    class Company
    {
    int ID { get; set; }
    Contact[] Contacts { get; set; }
    }

    class Contact
    {
    string FirstName;
    string LastName;
    }

    What would be the $filter clause to find all companies where a contact named John Smith works?

    Thank you,
    Dmitry

    • The current version of the OData protocol does not support filtering inside a collection. This has been proposed in the form of “any” and “all” for the next version but not finalised yet. Therefore Linq2Rest only supports the current version.

  4. Hi Pete,

    Thanks for the informative article. I’m trying to implement this to use protobuf-net, however I’m having trouble. Any help would be appreciated. Here what I have so far.

    public class ProtobufNetFormatter : MediaTypeFormatter
    {
    public ProtobufNetFormatter() { }

    protected override bool CanReadType(Type type)
    {
    return type != typeof(IKeyValueModel);
    }

    protected override bool CanWriteType(Type type)
    {
    return true;
    }

    protected override Task OnReadFromStreamAsync(Type type, Stream stream, HttpContentHeaders contentHeaders, FormatterContext formatterContext)
    {
    return Task.Factory.StartNew(() =>
    {

    return Serializer.Deserialize(stream); //this does not work??? because of type
    });
    }

    protected override Task OnWriteToStreamAsync(Type type, object value, Stream stream, HttpContentHeaders contentHeaders, FormatterContext formatterContext, TransportContext transportContext)
    {
    return Task.Factory.StartNew(() =>
    {
    Serializer.Serialize(stream, value);
    });
    }
    }

  5. Thanks! Good stuff. I didn’t know how to access the non generic versions of those functions. You still have to register protobufnetformatter with the HttpConfiguration.Formatters correct? Also, is Queryable cross-platform compatible?

  6. Yes, I’ve updated the Gist with the Global.asax.cs for adding the formatter. As for Queryable being cross-platform compatible, which platforms are you interested in?

  7. Yep, the server pieces are fine for that. You should be able to issue the OData URI queries without any concern for the fact that the data is coming from an IQueryable.

    Of course you will not be able to use the standard OData client libraries for android and Obj-C as they expect Atom+XML format rather than protobuf. I guess though that you are planning to handle the deserialization yourself.

  8. Indeed that was the case. How would I go about implementing the RestClient for protobuf since it isn’t JSON or xml? Thanks for the continued help.

  9. Ah I missed it. Thank you. Also, when I navigate to {server address}/api/{controllername} I get a XML output instead of some binary file. Why is that?

  10. Ahhh. That is due to the fact that your browser is sending the accepts header for xml etc. The Web API will pick the formatter according to the accepts header. To be able to see the appropriate format you need to modify the accepts header.

    The best way to do this is to use Fiddler http://www.fiddler2.com/fiddler2/version.asp

    With fiddler open, issue the request in your browser. You should see it appear in fiddler. Drag the row from the sessions in fiddler to the Composer tab on the right. Then change the Accept: line to read:

    Accept: application/x-protobuf

    With that done hit execute in fiddler and it should execute the request. If everything worked ok then you should see a reduced message size in the request that just got served in fiddler.

  11. That makes sense. Alright, one more thing. I can use other methods of accessing the Web API data other than Linq2Rest, correct? I could use something like WebClient or something like RestSharp, correct?

  12. Of course. What I wanted to show here was the conversion of an IQueryable expression to OData URIs. You could just issue the uri query yourself against the service using WebClient or RestSharp or HttpClient etc.

  13. @Dmitry Starosta Peter contributed the feature to query collections using Any and All queries. It is not currently included in the published Nuget package because of some difficulties coordinating versions with Microsoft’s Rx Team. I’m hoping it will be included soon, so you will be able to query:
    Company.Where(x=>x.Contacts.Any(y=>y.LastName == “Smith”));

  14. Great post. Will Querying capability work well when I need to perform some business validations before returning the data on large data? Eg: Table – Employee, Total no. of records – 100000, Scenario – my service need to respond with CURRENT employees(approx count 50000) only and expose the querying capability to it. I visualize as, service receives the oData queries, business layer performs the validation and an ORM performs the execution on the DB based on the query cascaded from business layer.

    • It should be fine.It depends on the ORM but LINQ queries should be composable. In other words if you have some filtering in your controller (to current employees only ) and a request comes in that further filters your entity set then you should only get a single query for the composed filter. Of course you need to check this with a SQL profiler but this should be the case as long as you are not using any evaluating functions in your controller method (e.g. ToList, ToArray etc)

  15. Great post. I’ve added $inline count to a project POC that I’ve started based on your work. The only problem with my (and your) solution is the lack of projection and __deferred metadata for lazy loading. This is something I can live with in a POC, but I’m going to want it before I start producing production code.

    • Yes, this stuff is probably getting out of date with the arrival of the fall update. When I fid some time I’ll give it another shot with the latest bits. Unfortunately the open source story for the client bits is not quite there.

  16. Pingback: Blue Ray Plus - Latest Technology News

Got a comment? Go on...