Back in March I wrote a quick howto on supporting $expand for entities in ASP.Net Web API. That method used an extension method to pull the $expand clause out of the query string an apply it to the ObjectSet.

I’ve now added to that sample an ActionFilter which will do this for you without any code in your controller.

To use it, change your global.asax and add the ActionFilter as part of your configuration.

public static void RegisterRoutes(RouteCollection routes) {
    ...
    ...

    HttpConfiguration configuration = GlobalConfiguration.Configuration;
    configuration.Filters.Add(new EFExpandActionFilter());

    ...
    ...
    ...
}

 

Now your api methods need only return an ObjectSet or ObjectQuery without any extension methods.

public IQueryable<Customer> GetCustomers() {
    return northwindEntities.Customers;
}

Note that if you are using the default XML serializer you will find that the navigation properties do not serialize due to the XmlIgnore attribute, either customize your entities or use Json.Net (as in the sample project)

WCF Data Services 5 has been RTM’d and adds support for a whole bunch of scenarios:

• Any/All
• Actions
• Collections
• Named Streams/Stream Properties
• PATCH Verb
• Prefer Headers
• Properties on Derived Types
• Support for DateTimeOffset and TimeSpan Data Types
• Support for DbContext as the DataService Source
• Spatial
• Vocabularies

The most interesting bit for me initially was Any/All. Basically this has been one of those things that I get asked from time to time and have to explain that you could not (until now) filter a entity set based on some values in a sub-collection. E.g. give me all the customers who have ordered “x”.

With the inclusion of Any and All the following code is now possible

// http://foo/?filter=Orders/any(o: o/OrderItems/any(i: i.Id eq 5))
var customers = Customers
                .Where(c =>
                    c.Orders.Any(o =>
                        o.OrderItems.Any(i =>
                            i.Id == 5));

 

I like the lambda syntax in the URI $filter parameter. It makes it easy for anyone that is used to modern languages to see what is going on.

So right now you can get WCF Data Services 5 and start using the Any/All support in OData.

Jacob Reimers has also gracefully accepted and published my pull request so Linq2Rest now fully supports Any/All queries in both the server and client APIs.

I’ve also submitted an issue on the ASP.Net Web API for support so please up vote that issue if it’s something that’s important to you. From that issue you can also go see the implementation that I’ve done for my fork of aspnetwebstack that implements server support for Any/All in ASP.Net Web API.

Recently I was using a new AppDomain from within a Visual Studio package to reflect over a dll and get some metadata. I then unloaded the AppDomain expecting that the file locks on the dll file would be released…..but they weren’t.

I could see that another AppDomain in VS was holding a lock on my file. You can see this by switching on Native debugging when attaching another visual studio instance and from the immediate window use:

.load sos

!dumpdomain

It turns out this is due to the LoaderOptimization attribute. Set this property to SingleDomain on your AppDomainSetup object to allow the file lock to be freed when unloading your AppDomain. This translates as.

Indicates that the application will probably have a single domain, and loader must not share internal resources across application domains.

Pushqa is a smarter pub/sub model, allowing the subscriber to use LINQ to declare the messages they want to receive.

Background

LINQ has revolutionised the way we work with data in .Net, allowing us to compose and query data sources of all types. When Microsoft introduced OData it opened up the ability to query server resources using a simple URI syntax or LINQ if you are using a .Net client. This was great news for accessing data stores like SQL using Entity Framework, NHibernate etc over HTTP such that the data manipulation and querying happens on the server before being sent to the client. A kind of LINQ remoting if you like.

Then came Reactive Extensions (Rx), a LINQ implementation that instead of accessing data sources, allows querying and composition of future events or event streams to produce new event streams. Rx revolutionises the event concept in .Net and makes dealing with asynchrony so much easier.

Introducing Pushqa

Suppose then, that we could also use Rx LINQ or OData uri syntax to filter an event stream of push messages before they leave the source. A kind of smarter Pub/Sub that allows the subscriber to specify which messages they want to receive from the server in a more natural declarative style. This is the goal of Pushqa, to give subscribers power over their own subscriptions.

Imagine a log viewer client that can filter to only the error level messages that happen to a specific user. Or a client cache that listens to specific domain events in order to evict only the items it cares about from it’s cache when they change.

By combining Rx, SignalR and OData Pushqa is able to translate LINQ queries to OData compliant URI conventions. The uri is used to initiate a persistent connection to the server that will then filter the events exposed by an Rx event stream into tailored streams for each client.

e.g.

The following code

eventProvider.Stocks
    .Where(s =>
        s.Name == "GOOG" ||
        s.Name == "MSFT")
    .AsObservable()
    .Subscribe(s => Console.WriteLine(“{0} – {1}”, s.Name, s.Price))

results in the following OData query

http://my.domain.com/Stocks/?filter=(Name eq 'GOOG') or (Name eq 'MSFT')

and the following output

MSFT – 24.11

GOOG – 457.52

MSFT – 30.11

GOOG – 576.65

…

The stock events generated by the source on the server will be filtered to only Google and Microsoft before being sent to the subscriber one at a time using HTTP long polling or similar HTTP based push messaging as provided by SignalR.

Fetch the early bits from GitHub. Be warned these bits are early but hopefully the sample projects will give you some ideas. Samples include:

• Using Skip and Take to complete event streams.
• A Process Viewer showing the server’s process information with live updates.
• A Stock ticker.
• WPF and Web examples.

Following on from my previous post on creating a client API for ASP.Net Web API queryable oData services, I wanted to prove that expand clauses could also work for Entity Framework object sets.

The oData URI conventions specifies that the desired depth of an entity can be represented in the $expand query string parameter. When using the ASP.Net Web API we need a way to pull this information from the querystring and convert it into Include statements on the ADO.Net Entity Framework object set. To achieve this I created an extension method that will pull the $expand query string parameter and translate it into multiple Include method calls.

public class NorthwindController : ApiController
{
    private readonly NorthwindEntities northwindEntities = new NorthwindEntities();

    public NorthwindController() {
        northwindEntities.ContextOptions.LazyLoadingEnabled = false;
    }

    public IQueryable<Customer> GetCustomers() {
        return northwindEntities.Customers.ProcessExpands();
    }
}

public static class ObjectQueryExtensions {
    public static ObjectQuery<T> ProcessExpands<T>(this ObjectSet<T> source) where T : class {
        string expandsQueryString = HttpContext.Current.Request.QueryString["$expand"];
        if(string.IsNullOrWhiteSpace(expandsQueryString)) {
            return source;
        }

        ObjectQuery<T> query = source;

        expandsQueryString.Split(',').Select(s => s.Trim()).ToList().ForEach(
            expand => {
                query = query.Include(expand.Replace("/", "."));
            });

        return query;
    }
}

Now all that’s left is to figure out how to provide the client side support for this….

Meanwhile the full source for the above project is available on github. https://github.com/PeteGoo/WebApiContext.Sample

TL;DR: How to create a client Linq api for querying IQueryable ASP.Net Web API rest services in very few lines of code using JSON.Net and Linq2Rest.

 

The ASP.Net MVC4 Beta was released recently and included the Beta release of ASP.Net Web API, formally known as the WCF Web API. Lots of people have already talked about the benefits of the Web API and how it has been built with simplicity, transparency, extensibility and testability in mind so I’ll not go into too many details about that here.

Using ASP.Net Web Api instead of WCF Data Services

One of the features that excites me about Web Api is the ability to expose an IQueryable on a Rest service and have Web Api automatically map queries in oData uri format to actual linq queries on the source i expose. For example, if my Web API controller looks like this:

public class PeopleController : ApiController {
    public IQueryable<Person> GetModels() {

        return people.AsQueryable();
    }

    private readonly Person[] people = new Person[] {
            new Person {
                Id = 1,
                FirstName = "Peter",
                LastName = "Goodman"
            },
            new Person {
                Id = 2,
                FirstName = "Peter",
                LastName = "Skeeter"
            },
            new Person {
                Id = 3,
                FirstName = "Tony",
                LastName = "Stark"
            },
            new Person {
                Id = 4,
                FirstName = "Frank",
                LastName = "Grimes"
            },
            new Person {
                Id = 5,
                FirstName = "Doug",
                LastName = "Kettle"
            },
            new Person {
                Id = 6,
                FirstName = "Finbar",
                LastName = "Coole"
            },
        };
}

 

Web Api will now let me access this resource and query it using oData syntax e.g.:

/api/People?$filter=FirstName eq ‘Peter’&$orderby=LastName desc

This of course is just like the kind of thing you get out of WCF Data Services with a couple of very important differences.

• Web API does not currently support all of the features of oData, e.g. projection and expand
• Web API is highly configurable and extensible.
• Web API does not currently have a formatter Atom+Xml format, although an implementation is on the roadmap
• There currently is no built in .Net client api for issuing Linq queries to Web API rest services

One of the pains of WCF Data Services has been the inability to configure or extend the implementation, for example it is very difficult or impossible to add your own formatter/serializer. With Web API, this kind of stuff is almost trivial.

The WCF Data Services client API (DataServiceContext) only allows you to use Atom+Xml which for a lot of purposes is extremely bloated in comparison to something like JSON or protobuf.

Unfortunately there also is no equivalent to the DataServiceContext that allows you to query the above service in Web API, therefore you cannot use LINQ to query the above service. This is what we are going to build.

Setting up the project

Make sure you have ASP.Net MVC4 Beta installed and create a new ASP.Net MVC 4 application project. When you get prompted for the Project Template make sure to choose “Web API” and hit OK.

Under models add a new class called Person and add the following code.

public class Person {
    public int Id { get; set; }

    public string FirstName { get; set; }

    public string LastName { get; set; }
}

Change the name of the ValuesController in the Controllers folder to PeopleController and add the code at the start of this article.

Run your project and in the browser change the url to something like the following remembering to use the hostname and port of your own project:

http://localhost:21449/api/People

You should see a list of the people we have defined.

Now we should be able to change this to the following to see only the ‘Peter’s

http://localhost:21449/api/People?$filter=FirstName = ‘Peter’

You will notice that the output is in Xml. We want to customize this to allow Json

Creating the JSON.Net formatter

To create a json formatted output for our service we are going to use JSON.Net, the Web API team have already said that they are going to ship with JSON.Net as the default JSON formatter when it goes RTM. Therefore you can skip this section if you are working with the RTM version.

Firstly, using Nuget package manager, add a package dependency to your project on JSON.Net

Create a folder in your web project called Infrastructure and add a class called JsonNetFormatter. Paste in the code for this class only from Henrik’s article on the subject.

In the global.asax.cs file in your project add the following code in the RegisterRoutes method to register the JSON.Net formatter before the current default one.

JsonSerializerSettings serializerSettings = new JsonSerializerSettings();
serializerSettings.Converters.Add(new IsoDateTimeConverter());

HttpConfiguration configuration = GlobalConfiguration.Configuration;

configuration.Formatters.Insert(0, new JsonNetFormatter(serializerSettings));

You can test that works in fiddler2 by starting fiddler, re-running the previous query in the browser (you may need to replace the hostname with ipv4.fiddler to see it), drag the session into the composer tab and change the accept header to

Accept: application/json

You should now see that we have JSON output.

Creating the client application and api

Add a new console application project to your solution named something sensible. This is going to be our .Net client.

To create our client side code we are going to need something that can take a LINQ query and convert it to the oData URI format. In another project I am working on I have recently been using the excellent Linq2Rest library written by Jacob Reimers. He has written Linq2Rest to solve the problem of converting between IQueryables and OData URIs. Unfortunately the client and server pieces for Linq2Rest are in the same assembly so you will need to make your console app targets .Net 4.0 instead of the client profile in the project properties in order to support the dependency on System.Web.

Add a nuget package reference in your console app to Linq2Rest and Json.Net.

Next we need to create a matching JSON.Net deserializer for the Linq2Rest library so that it will match our server side implementation. This is just to ensure compatibility, you could of course reference the same common serializer settings etc. Create a folder in your console app called Infrastructure and add the following code for the JsonNetSerializerFactory and it’s serializer.

public class JsonNetSerializerFactory : ISerializerFactory {
    public ISerializer<T> Create<T>() {
        return new JsonNetSerializer<T>();
    }

    public class JsonNetSerializer<T> : ISerializer<T> {
        public T Deserialize(string input) {
            return JsonConvert.DeserializeObject<T>(input);
        }

        public IList<T> DeserializeList(string input) {
            return JsonConvert.DeserializeObject<IList<T>>(input);
        }
    }
}

 

Now we are ready to create our equivalent of the DataServiceContext. Add a new class to that project called PeopleContext and fill it out with the following code.

public class PeopleContext {
    private readonly RestContext<Person> restContext;

    public PeopleContext(Uri uri, Format format) {
        restContext = new RestContext<Person>(GetRestClient(uri, format), GetSerializerFactory(format));
    }

    public enum Format {
        Pox,
        Json
    }

    public static IRestClient GetRestClient(Uri uri, Format format) {
        switch (format) {
            case Format.Pox:
                return new XmlRestClient(uri);
            case Format.Json:
                return new JsonRestClient(uri);
            default:
                throw new NotImplementedException();
        }
    }

    public static ISerializerFactory GetSerializerFactory(Format format) {
        switch (format) {
            case Format.Pox:
                return new XmlSerializerFactory(knownTypes);
            case Format.Json:
                return new JsonNetSerializerFactory();

            default:
                throw new NotImplementedException();
        }
    }

    private static readonly IEnumerable<Type> knownTypes = new[] {
        typeof (Person)
    };

    public IQueryable<Person> People {
        get { return restContext.Query; }
    }

}

Now we are ready to consume our new fancy pants context class. In the Program.cs add the following code to query the service and display the results. Remember to replace the hostname and port with your settings. I’ve used the ipv4.fiddler address to force fiddler to log the session, this of course won’t work if you don’t have fiddler running.

class Program {
    static void Main(string[] args) {
        PrintQuery(PeopleContext.Format.Pox);
        PrintQuery(PeopleContext.Format.Json);

        Console.ReadKey(true);
    }

    private static void PrintQuery(PeopleContext.Format wireFormat) {
        Console.WriteLine();
        Console.WriteLine("*** Querying in {0} format ***", wireFormat);
        Console.WriteLine();

        // Perform Query
        new PeopleContext(new Uri("http://ipv4.fiddler:14061/api/People"), wireFormat)
            .People
            .Where(model => model.FirstName.StartsWith("Pe"))
            .ToList()
            .ForEach(item => {
                Console.WriteLine("-----------------------------------");
                Console.WriteLine("Id:{0}", item.Id);
                Console.WriteLine("First Name:{0}", item.FirstName);
                Console.WriteLine("Last Name:{0}", item.LastName);
            });
        Console.WriteLine("-----------------------------------");
    }
}

 

Run the console app and you should see:

*** Querying in Pox format ***

———————————–

Id:1

First Name:Peter

Last Name:Goodman

———————————–

Id:2

First Name:Peter

Last Name:Skeeter

———————————–

*** Querying in Json format ***

———————————–

Id:1

First Name:Peter

Last Name:Goodman

———————————–

Id:2

First Name:Peter

Last Name:Skeeter

———————————–

 

And we’re done. You should be able to see in fiddler now that we have issued a request in each format (Xml and Json) and received the appropriate format in your response. I have even used this approach to implement a protobuf-net version for even greater speed and reduced payload.

You can get the source for this sample on GitHub

https://github.com/PeteGoo/WebApiContext.Sample

I hit a problem recently where a singleton I was calling from an assembly resolver was throwing an exception. It turned out that the sync object used in the lock for the singleton pattern was null. How was this possible? Look at the code below:

using System;

public sealed class Singleton
{
   private static volatile Singleton instance;
   private static readonly object syncRoot = new Object();

   private Singleton() {}

   public static Singleton Instance
   {
      get
      {
         if (instance == null)
         {
            lock (syncRoot)
            {
               if (instance == null)
                  instance = new Singleton();
            }
         }

         return instance;
      }
   }
}

How can this be, my static readonly field should be set by the field initializer. This should be run before it’s first usage. Normally, this is true but for some reason this was not the case. The C#spec though gives us a workaround, implement a static constructor. The section on static constructors states:

If a class contains any static fields with initializers, those initializers are executed in textual order immediately prior to executing the static constructor.

Therefore, if we add a static constructor to our Singleton class, we get guaranteed initialization of the static field initializers before it is run, also:

The static constructor for a class executes at most once in a given application domain. The execution of a static constructor is triggered by the first of the following events to occur within an application domain:

• An instance of the class is created.
• Any of the static members of the class are referenced.

Therefore our static constructor will run before our singleton Instance property.

I have been meaning to get into Rx for a while now and haven’t quite found the excuse or opportunity to do so. While playing with SignalR it became apparent that Rx was something that could help and while I was researching it I seen a lot of people talking about using Rx in UI frameworks and I thought, hmmm, that sounds interesting.

So I set myself a challenge, I wanted to use Rx to create an auto-complete WPF control with some of the major use cases I have seen in these types of controls.

The App Shell

First thing I was going to need was a WPF application with a MainWindow that would hold my text box, the results and a log window.

 

The XAML looks something like this:

<Window
    x:Class="AutoCompleteWithRx.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="AutoComplete with Rx"
    Height="451"
    Width="825"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:AutoCompleteWithRx="clr-namespace:AutoCompleteWithRx"
    mc:Ignorable="d"
    d:DataContext="{d:DesignInstance AutoCompleteWithRx:AutoCompleteViewModel}"
    FocusManager.FocusedElement="{Binding ElementName=SearchBox}">
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition
                Height="Auto" />
            <RowDefinition />
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition />
            <ColumnDefinition />
        </Grid.ColumnDefinitions>
        <TextBlock
            FontFamily="Segoe WP Light"
            FontSize="26.667"
            Text="Auto-Complete Sample"
            Grid.ColumnSpan="2" 

            />

        <StackPanel
            Grid.Row="2">
            <Label>Search</Label>
            <TextBox
                x:Name="SearchBox"
                Text="{Binding SearchText, UpdateSourceTrigger=PropertyChanged}"/>
            <ProgressBar Height="19" IsIndeterminate="True"/>
            <ListBox
                ItemsSource="{Binding SearchResults}" />
        </StackPanel>
        <StackPanel Grid.Column="1" Orientation="Vertical" Grid.Row="1" d:LayoutOverrides="Height">
            <Label>Log</Label>
            <ListBox
                ItemsSource="{Binding LogOutput}" />
        </StackPanel>
    </Grid>
</Window>

So that is basically the view for now. I have cheated and just instantiated my viewmodel in the code behind class for now.

public partial class MainWindow : Window {
    public MainWindow() {
        InitializeComponent();
        DataContext = new AutoCompleteViewModel();
    }
}

Now that we have a window I need to get my view model up to scratch. I started with simply satisfying the data properties required by my form.

public class AutoCompleteViewModel : INotifyPropertyChanged {

    private string searchText;

    public string SearchText {
        get { return searchText; }
        set {
            if (searchText == value) {
                return;
            }
            searchText = value;
            NotifyPropertyChanged("SearchText");
        }
    }

    private readonly ObservableCollection<string> logOutput = new ObservableCollection<string>();

    public ObservableCollection<string> LogOutput {
        get { return logOutput; }
    }

    private readonly ObservableCollection<string> searchResults = new ObservableCollection<string>();

    public ObservableCollection<string> SearchResults {
        get { return searchResults; }
    }

    public event PropertyChangedEventHandler PropertyChanged;

    protected virtual void NotifyPropertyChanged(string propertyName) {
        if (PropertyChanged != null) {
            PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
        }
    }
}

 

Subscribing to Property Changed Events using Rx

OK. So that is simple enough but now I need to make my textbox do something when the user types into it. Before I start doing this though I will need to add Rx to my project. Right click the project choose “Manage Nuget Packages” and search for Rx, install Rx-Main and Rx-WPF.

Let’s start by simply logging the fact that the user has typed something. Add a constructor to our view model class with the following code:

public AutoCompleteViewModel() {
    // Listen to all property change events on SearchText
    var searchTextChanged = Observable.FromEventPattern<PropertyChangedEventHandler, PropertyChangedEventArgs>(
        ev => PropertyChanged += ev,
        ev => PropertyChanged -= ev
        )
        .Where(ev => ev.EventArgs.PropertyName == "SearchText");

    // Transform the event stream into a stream of strings (the input values)
    var input = searchTextChanged
        .Select(args => SearchText);

    // Log all events in the event stream to the Log viewer
    input.ObserveOn(DispatcherScheduler.Instance)
        .Subscribe(e => LogOutput.Insert(0,
            string.Format("Text Changed. Current Value - {0}", e)));
}

The first section of slightly unwieldy syntax creates an IObservable from the PropertyChanged event of INotifyPropertyChanged which our view model implements. You can think of the IObservable as a stream of events that will happen or a collection of future objects that will be added to each time the event is raised. Our code will then filter those events to only those concerning the SearchText that our textbox is bound to.

The next line transforms the type of the events in our IObservable from PropertyChangeEventsArgs to string, the content of the textbox. Much more useful.

Finally we need to subscribe to the event stream and do something with the string, firstly though we tell the subscription that we want to observe the events on the WPF dispatcher (This helper singleton comes from the Rx-WPF package). Then we simply provide an inline function to execute for each event, in our case inserting into the log stack which our LogOutput list box is bound to.

The result looks like this.

OK. So far so good. Now let’s add an actual search to the mix.

Chaining the Asynchronous Search

Before I add my search function I want to create a structure that will encapsulate the search term and it’s result.

public struct SearchResult {
    public string SearchTerm { get; set; }
    public IEnumerable<string> Results { get; set; }
}

Next I add my search function. For now I’ve just created a synchronous function but chances are you would be calling an external service with Async methods (BeginXXX and EndXXX) for which you should use the FromAsyncPattern method instead of the ToAsync I am using.

private SearchResult DoSearch(string searchTerm) {
    return new SearchResult {
        SearchTerm = searchTerm,
        Results =
            phrases.Where(item => item.ToUpperInvariant().Contains(searchTerm.ToUpperInvariant())).ToArray()
    };
}

private readonly string[] phrases = new[] {
    "The badger knows something",
    "Your head looks like a pineapple",
    "etc...",
};

And now for the magic. To invoke our search we add the following to our constructor:

// Setup an Observer for the search operation
var search = Observable.ToAsync<string, SearchResult>(DoSearch);

// Chain the input event stream and the search stream
var results = from searchTerm in input
              from result in search(searchTerm)
              select result;

// Log the search result and add the results to the results collection
results.ObserveOn(DispatcherScheduler.Instance)
.Subscribe(result => {
    searchResults.Clear();
    LogOutput.Insert(0, string.Format("Search for '{0}' returned '{1}' items", result.SearchTerm, result.Results.Count()));
    result.Results.ToList().ForEach(item => searchResults.Add(item));
    }
);

Here we firstly call ToAsync, this will create a function that takes a string (our search term) and returns an Observable that will only ever produce one event (the Completed / End) and then close.

We chain the incoming input stream with the result of the search function using simplified LINQ syntax and produce a new observable of SearchResult. We then observe the result, logging and updating the autocomplete items.

The result is this:

OK. So this is round about where my mind exploded and grey matter started to leak out of my ears. Basically what is happening is we are treating these incoming events as collections before they actually happen, then we can manipulate them using LINQ and even chain the events much like pipelining in UNIX / Powershell.

We’re not done yet. We still have a number of issues.

Using Throttle to enforce an idle period before searching

One of the issues you can see in the above screenshot is that the search was done after every single key press. The owner of the service we are calling will not be too happy with us flooding them with unnecessary searches, so how do we stop this from happening. Typically we would create a timer that would wait for a period of inactivity before searching, we would have to reset the timer every time the user typed something so that the search is not performed for each of the previous characters and there would be quite a few lines of code required to actually do this.

With Rx we can use the Throttle operator which will quite effectively do exactly the behaviour we want for free. We change our previous declaration of the input stream to add the Throttle method.

var input = searchTextChanged
    .Throttle(TimeSpan.FromMilliseconds(400))
    .Select(args => SearchText);

Now if we test we should get something like the following:

That was too easy.

Merging two event streams

The next requirement was that I may want a larger throttle timeout for strings of less than 3 characters as people tend to type slower at the start of a search. To do this I needed to split my input stream into two different event streams that could be throttled differently.

var input = searchTextChanged
    .Where(ev => SearchText == null || SearchText.Length < 4)
    .Throttle(TimeSpan.FromSeconds(3))
    .Merge(searchTextChanged
        .Where(ev => SearchText != null && SearchText.Length >= 4)
        .Throttle(TimeSpan.FromMilliseconds(400)))
    .Select(args => SearchText);

So above I have simply cut the searchTextChanged stream 2 different ways, throttled them differently and then I can use the Merge operator to join the streams back together again into one.

Creating a Reactive ICommand for MVVM

The next requirement is that the search should be executed immediately if the user hits the enter key. To do this I decided to use the MVVM ICommand pattern. We need to add the following Input KeyBinding to the TextBox in our XAML so that the enter key will be picked up and we point it at the command we will write below.

<TextBox
    x:Name="SearchBox"
    Text="{Binding SearchText, UpdateSourceTrigger=PropertyChanged}">
    <TextBox.InputBindings>
        <KeyBinding
            Command="{Binding TextBoxEnterCommand}"
            Key="Enter" />
    </TextBox.InputBindings>
</TextBox>

The next thing we need to do is create our own command type. If you use an MVVM framework that understands Rx like ReactiveUI then you should get this for free but implementing a simple version yourself is not difficult. We simply take the RelayCommand defined by Josh Smith and add a Subject<T> exposed as an Observable.

public class ReactiveRelayCommand : ICommand {
    private readonly Action<object> execute;
    private readonly Predicate<object> canExecute;

    public ReactiveRelayCommand(Action<object> execute)
        : this(execute, null) {
    }

    public ReactiveRelayCommand(Action<object> execute, Predicate<object> canExecute) {
        if (execute == null)
            throw new ArgumentNullException("execute");

        this.execute = execute;
        this.canExecute = canExecute;
    }

    [DebuggerStepThrough]
    public bool CanExecute(object parameter) {
        return canExecute == null || canExecute(parameter);
    }

    public event EventHandler CanExecuteChanged {
        add { CommandManager.RequerySuggested += value; }
        remove { CommandManager.RequerySuggested -= value; }
    }

    public void Execute(object parameter) {
        execute(parameter);
        executed.OnNext(parameter);
    }

    private readonly Subject<object> executed = new Subject<object>();

    public IObservable<object> Executed {
        get { return executed; }
    }
}

Notice that it exactly the same as the standard RelayCommand except that we use a Subject<object> as the backing implementation for our IObservable. We call OnNext whenever the command is Executed.

Add the following field and property to the viewmodel class to store our command:

private ReactiveRelayCommand textBoxEnterCommand;
public ReactiveRelayCommand TextBoxEnterCommand {
    get { return textBoxEnterCommand; }
    set { textBoxEnterCommand = value; }
}

Then at the start of our constructor we setup the command itself:

// Setup the command for the enter key on the textbox
textBoxEnterCommand = new ReactiveRelayCommand(obj => { });

Next we need to change our input event stream to take into account our command and merge that with our SearchText property change events so that either can fire our search:

// Transform the event stream into a stream of strings (the input values)
var input = searchTextChanged
    .Where(ev => SearchText == null || SearchText.Length < 4)
    .Throttle(TimeSpan.FromSeconds(3))
    .Merge(searchTextChanged
        .Where(ev => SearchText != null && SearchText.Length >= 4)
        .Throttle(TimeSpan.FromMilliseconds(400)))
    .Select(args => SearchText)
    .Merge(
        textBoxEnterCommand.Executed.Select(e => SearchText));

Notice we have just added the last line to merge the Executed observable on our command. Now when we run we can type a single character and hit enter if we do not want to wait for the idle period. How easy was that? Are we done? Almost.

Removing duplicate consecutive events

We still have an issue that when the user presses Enter, the idle timeout will expire some time later and we will get another event firing, essentially performing our search twice. So how do we stop the second duplicate event from being fired, easy, we just use DistinctUntilChanged.

We want a distinct search but we still want to allow the same search to be done later if the user tries another search in between. In other words we can have the same search done twice as long as they aren’t consecutive searches.

// Transform the event stream into a stream of strings (the input values)
var input = searchTextChanged
    .Where(ev => SearchText == null || SearchText.Length < 4)
    .Throttle(TimeSpan.FromSeconds(3))
    .Merge(searchTextChanged
        .Where(ev => SearchText != null && SearchText.Length >= 4)
        .Throttle(TimeSpan.FromMilliseconds(400)))
    .Select(args => SearchText)
    .Merge(
        textBoxEnterCommand.Executed.Select(e => SearchText))
    .DistinctUntilChanged();

 

Cancelling previous searches with TakeUntil

The last issue here is that when searches return out of sequence, we could get the wrong results displayed. For example, the user searches for cr hits enter but then types crazy and hits enter. If the first search is taking a long time because of the number of results then the results for it will end up being displayed after the actual current search term is displayed. To simulate this we could add some latency to our search.

private readonly Random random = new Random(5);
private SearchResult DoSearch(string searchTerm) {
    Thread.Sleep(random.Next(100, 500)); // Simulate latency
    return new SearchResult {
        SearchTerm = searchTerm,
        Results =
            phrases.Where(item => item.ToUpperInvariant().Contains(searchTerm.ToUpperInvariant())).ToArray()
    };
}

To solve this we need to stop the Async stream when more input is given, this is pretty easy with the TakeUntil operator. Take operators define a completion for an event stream. In the case of an Async operation this will effectively cancel the Async subscription. TakeUntil takes an observable as input such that an event on that stream will complete the current stream. To implement this change we simply say the we TakeUntil the input stream in our chaining code.

// Chain the input event stream and the search stream, cancelling searches when input is received
var results = from searchTerm in input
              from result in search(searchTerm).TakeUntil(input)
              select result;

Summary

So we have managed to create the guts of an auto-complete control using Rx in a much more declarative way by responding to events in observables instead of writing lots of imperative code with little items of state sitting about to represent our state machine.

All of the heavy work can be easily seen in a single declaration of intent:

// Setup the command for the enter key on the textbox
textBoxEnterCommand = new ReactiveRelayCommand(obj => { });

// Listen to all property change events on SearchText
var searchTextChanged = Observable.FromEventPattern<PropertyChangedEventHandler, PropertyChangedEventArgs>(
    ev => PropertyChanged += ev,
    ev => PropertyChanged -= ev
    )
    .Where(ev => ev.EventArgs.PropertyName == "SearchText");

// Transform the event stream into a stream of strings (the input values)
var input = searchTextChanged
    .Where(ev => SearchText == null || SearchText.Length < 4)
    .Throttle(TimeSpan.FromSeconds(3))
    .Merge(searchTextChanged
        .Where(ev => SearchText != null && SearchText.Length >= 4)
        .Throttle(TimeSpan.FromMilliseconds(400)))
    .Select(args => SearchText)
    .Merge(
        textBoxEnterCommand.Executed.Select(e => SearchText))
    .DistinctUntilChanged();

// Log all events in the event stream to the Log viewer
input.ObserveOn(DispatcherScheduler.Instance)
    .Subscribe(e => LogOutput.Insert(0,
        string.Format("Text Changed. Current Value - {0}", e)));

// Setup an Observer for the search operation
var search = Observable.ToAsync<string, SearchResult>(DoSearch);

// Chain the input event stream and the search stream, cancelling searches when input is received
var results = from searchTerm in input
              from result in search(searchTerm).TakeUntil(input)
              select result;

// Log the search result and add the results to the results collection
results.ObserveOn(DispatcherScheduler.Instance)
.Subscribe(result => {
    searchResults.Clear();
    LogOutput.Insert(0, string.Format("Search for '{0}' returned '{1}' items", result.SearchTerm, result.Results.Count()));
    result.Results.ToList().ForEach(item => searchResults.Add(item));
    }
);

 

Lots of the ideas and even the odd piece of code were taken from:

• Bart De Smet on Curing Asynchronous Blues
• Stack Overflow
  AutoCompleteBlog.zip

This may be basics to some people but I thought it was worth mentioning. I was writing some simple code recently to compare the files in a remote directory with those in a local directory to determined if the remote was newer in any way. This seemed like a really simple problem so I made my first stab at it.

return (from remoteFile in Directory.GetFiles(remotePath)
        join localFile in Directory.GetFiles(localPath)
            on Path.GetFileName(remoteFile) equals Path.GetFileName(localFile)
        where File.GetLastWriteTimeUtc(remotePath) > File.GetLastWriteTimeUtc(localPath)
        select remoteFile).Any();

 

It became very apparent that there were some performance issues when running this code, especially when the remote location introduces any form of latency. The first issue is with Directory.GetFiles, if you actually reflect this method you will see that it ultimately calls the following:

return new List<string>(FileSystemEnumerableFactory.CreateFileNameIterator(path, userPathOriginal, searchPattern, includeFiles, includeDirs, searchOption)).ToArray();

The existence of the ToArray means the full file list will be enumerated before the method returns, however we only care if any file has changed so it would be better if we had an enumerator so we could exit early when the first newer file is found. This is also true when performing the join as this will enumerate the joined collection first before enumerating the full original collection of remote files and then joining them.

We could solve the above issues using the new Directory.EnumerateFiles method which will return an enumerator that will yield each file as it is enumerated. We could also get rid of the join and enumerate all the local files first before in a separate operation to allow the more expensive remote enumeration happen in sequence.

The next problem in the original attempt is that we are returning to the remote files to call another operation to evaluate the last write time stamp on the file metadata. This requires us to return to the remote file system and incur the same latency per file. I originally tried to solve this problem using the Fast Directory Enumerator on code project which allowed me to enumerate the list of remote files at the same time as retrieving the file info, thus removing the need to return to the scene of the crime. This seemed to work ok but I was having an issue with it not completing the remote file enumeration sequence and discovered that there is actually an implementation introduced in .Net 4. The very well hidden EnumerateFiles on DirectoryInfo was the answer.

var localFiles = new DirectoryInfo(localPath).GetFiles();

return new DirectoryInfo(remotePath).EnumerateFiles()
    .Any(remoteFile => (from localFile in localFiles
                        where localFile.Name == remoteFile.Name
                        && remoteFile.LastWriteTimeUtc > localFile.LastWriteTimeUtc
                        select remoteFile).Any());

Notice the remote files retrieval now uses the EnumerateFiles method on DirectoryInfo. This allows us to exit when the first item matches the condition in the Any clause.

I’ve been playing with SignalR recently and it drives me nuts that I can’t have more than a few simultaneous long polling requests open to the IIS server on Windows 7. It stops responding after about 2 clients. Even IIS Express supports unlimited simultaneous connections but we need AppFabric support.

We would seriously consider using it for our messaging solution but on a single machine developer install, we would need to come up with another solution for our many client apps.