Session data is evil

I’ve been working in ASP.MVC recently after working in Java for a long time. One of the things that struck me was the common use of session data in web application.

Now I know that people can and do use abuse sessions in Java, but the default routing and ease of access make using it more tempting in ASP.MVC. The standard routing convention of “/Controller/Action/:id” means you need to explicitly code to use RESTful paths that give you multipleĀ IDs in URLs like “order/2/item/3” for non-trivial scenarios, and out of the box convenience methods like “TempData” seem to offer magical persistence between requests. These incentives combine to make using session data the path of least resistance in ASP.MVC.

1487ta

Any data stored in session is inherently unreliable and use of it makes load balancing and scaling your application much more difficult. Once you use it, each instance of your web application must be able to find the users session data to reliably handle requests. Since it’s now extremely common to use multiple instances even for small applications (irresponsible not to for disaster recovery and redundancy) you will need to think about this before you deploy into production.

It also adds hidden complexity to testing your application. Each endpoint which relies on state stored in session needs to be tested with that application state simulated. This means you have at least two places in your code defining and using the same semi-structured data, which makes your tests complex/fragile and your code harder to maintain.

Stelhi_Silk_Mill_Lanco_broken_windows

Once you make using session data part of your architecture it’s very hard to refactor and remove it. That little innocent use of TempData to store details from the last request will spread as Developers think “If it was ok there then it’s ok here…” and “one more can’t hurt” (the broken windows theory). Now your user flows in the web application rely on session stored details to go from screen A to B to C, and refactoring them means re-writing and testing a lot of the view/controller logic to replace the data held in session.

There are acceptable uses for session data in web application, authentication is the obivous one. What they have in common is having alternative flows to cope if session data is not found without breaking functionality.

If you have an over reliance of session in your application you are making a flakey, hard to scale and maintain application that will at best limp into production. At worst it will fall over and take your users data with it.

There are common patterns and methods to avoid needing session data, below are some links to help:

ASP.MVC Datatables server-side

This is an example implementation of JQuery Datatables with server-side processing. The source is here.

datatables

Introduction

JQuery Datatables is a great tool, attach it to a table of results and it gives you quick and easy sorting/searching. Against a small dataset this works fine, but once you start to have >1000 records your page load is going to take a long time. To solve this Datatables recommend server-side processing.

This code is an example of implementing server-side processing for an ASP.MVC web appliction, using a generic approach with Linq so that you can re-use it for different entities easily with little code repetition. It also shows an implementation of full word search across all columns, which is something that the Javascript processing version offers but is very tricky to implement on the database side with decent performance. It’s a C# .NET implementation but you can take the interfaces and calls from the controllers and convert the approach for Java or Ruby (missing the nice Linq stuff tho).

Details

I’ll skip the basic view/js details as that is easily available on the datatables documentation.

The request comes into the controller as a GET with all the sort/search details as query parameters (see here), it expects a result matching this interface:

public interface IDatatablesResponse<T>
{
    int draw { get; set; }
    int recordsTotal { get; set; }
    int recordsFiltered { get; set; }
    IEnumerable<T> data { get; set; }
    string error { get; set; }
}

The controller extracts the parameters, creates the DB context and repository and makes three calls asynchronously:

  • get the total records
  • get the total filtered records
  • get the searched/sorted/paged data

The data is returned and Datatables Javascript uses it to render the table and controls for the correct searched/sorted/paged results.

The magic happens in the DatatablesRepository objects which handle those calls.

DatatablesRepository classes

Interface:

public interface IDatatablesRepository<TEntity>
{
    Task<IEnumerable<TEntity>> GetPagedSortedFilteredListAsync(int start, int length, string orderColumnName, ListSortDirection order, string searchValue);
    Task<int> GetRecordsTotalAsync();
    Task<int> GetRecordsFilteredAsync(string searchValue);
    string GetSearchPropertyName();
}

The base class DatatablesRepository has a default implementation which provides generic logic for paging, searching and ordering an entity:

protected virtual IQueryable<TEntity> CreateQueryWithWhereAndOrderBy(string searchValue, string orderColumnName, ListSortDirection order)
{
    ...
    query = GetWhereQueryForSearchValue(query, searchValue);
    query = AddOrderByToQuery(query, orderColumnName, order);
    ...
}

protected virtual IQueryable<TEntity> GetWhereQueryForSearchValue(IQueryable<TEntity> queryable, string searchValue)
{
    string searchPropertyName = GetSearchPropertyName();
    if (!string.IsNullOrWhiteSpace(searchValue) && !string.IsNullOrWhiteSpace(searchPropertyName))
    {
        var searchValues = Regex.Split(searchValue, "\\s+");
        foreach (string value in searchValues)
        {
            if (!string.IsNullOrWhiteSpace(value))
            {
                queryable = queryable.Where(GetExpressionForPropertyContains(searchPropertyName, value));
            }
        }
        return queryable;
    }
    return queryable;
}

protected virtual IQueryable<TEntity> AddOrderByToQuery(IQueryable<TEntity> query, string orderColumnName, ListSortDirection order)
{
    var orderDirectionMethod = order == ListSortDirection.Ascending
            ? "OrderBy"
            : "OrderByDescending";

    var type = typeof(TEntity);
    var property = type.GetProperty(orderColumnName);
    var parameter = Expression.Parameter(type, "p");
    var propertyAccess = Expression.MakeMemberAccess(parameter, property);
    var orderByExp = Expression.Lambda(propertyAccess, parameter);
    var filteredAndOrderedQuery = Expression.Call(typeof(Queryable), orderDirectionMethod, new Type[] { type, property.PropertyType }, query.Expression, Expression.Quote(orderByExp));

    return query.Provider.CreateQuery<TEntity>(filteredAndOrderedQuery);
}

The default implementation for creating the Where query (for searching) will only work if you provide a SearchPropertyName for a property that exists in the database that is a concatenation of all the values you want to search in the format displayed.

You can implement and override to use a custom method if your Entity does not support this, here is an example from the Person Entity:

public class PeopleDatatablesRepository : DatatablesRepository<Person>
{
    ...
    protected override IQueryable<Person> GetWhereQueryForSearchValue(IQueryable<Person> queryable, string searchValue)
    {
        return queryable.Where(x =>
                // id column (int)
                SqlFunctions.StringConvert((double)x.Id).Contains(searchValue)
                // name column (string)
                || x.Name.Contains(searchValue)
                // date of birth column (datetime, formatted as d/M/yyyy) - limitation of sql prevented us from getting leading zeros in day or month
                || (SqlFunctions.StringConvert((double)SqlFunctions.DatePart("dd", x.DateOfBirth)) + "/" + SqlFunctions.DatePart("mm", x.DateOfBirth) + "/" + SqlFunctions.DatePart("yyyy", x.DateOfBirth)).Contains(searchValue));
    }
}

The same is true of the order by query, which may need customisation to sort correctly for data, i.e. dates. Here is an example from the PersonDepartmentListViewRepository, which replaces the formatted date column being formatted with the raw date:

public class PersonDepartmentListViewRepository : DatatablesRepository<PersonDepartmentListView>
{
    ...
    protected override IQueryable<PersonDepartmentListView> AddOrderByToQuery(IQueryable<PersonDepartmentListView> query, string orderColumnName, ListSortDirection order)
    {
        if (orderColumnName == "DateOfBirthFormatted")
        {
            orderColumnName = "DateOfBirth";
        }
        return base.AddOrderByToQuery(query, orderColumnName, order);
    }
}

Using a view will make life much easier, as the data can be pre-formatted and you can supply a search column to do the full word searching, here’s the view I used to combine results from two tables:

CREATE VIEW [dbo].[PersonDepartmentListView]
AS
SELECT dbo.Person.Id, 
dbo.Person.Name, 
dbo.Person.DateOfBirth,
CONVERT(varchar(10), CONVERT(date, dbo.Person.DateOfBirth, 106), 103) AS DateOfBirthFormatted,
dbo.Department.Name AS DepartmentName,
CONVERT(varchar(10), dbo.Person.Id) + ' ' + dbo.Person.Name + ' ' + CONVERT(varchar(10), CONVERT(date, dbo.Person.DateOfBirth, 106), 103) + ' ' + dbo.Department.Name AS SearchString
FROM  dbo.Person INNER JOIN
       dbo.Department ON dbo.Person.DepartmentId = dbo.Department.Id

Notes

  • If you are displaying date values be aware that you will need to format the date for display before returning in JSON, and the date format will affect how you sort the column on the backend as you will need to identify the actual date column property rather than the formated string
  • For effort and performance you are better off creating view than using complex Linq queries
  • I created the initial example with the help of Stephen Anderson

Using JQuery UI to create a complex ordered reference data editor

JQuery is great, as is JQuery UI.

Sortable1

I had a requirement in a project to create an admin screen to edit reference data used in drop downs for other screens. The drop down options had to be ordered, enabled/disabled and allow users to add new options. Before JQuery I would have implemented this using a very message control with multiple buttons for Enable/Disable/Up/Down, all causing postbacks or using very large amounts of Javascript.

Using JQuery UI Sortable control I was able to do this very cleanly, just two ordered lists setup as connected and some neat Javascript to add/remove. Now I have a very UX friendly drag/drop list that users can re-order the options as they like.

Sortable2

Source (zip embedded in word doc as Posterous doesn’t like zips):

Links:

http://jqueryui.com/demos/sortable/#connect-lists

Microsoft Web Deploy Tool – Sample solution for msbuild single build multiple environment deployment with environmental configuration

Following on from my earlier post about Microsoft Web Deploy tool I’ve put together a sample solution with MSBuild scripts to build and deploy a web application and a WCF service with environmental configuration, so with a single build you can deploy to multiple environments with the right configuration and no manual config changes.

Previously I couldn’t figure out how to get the environment configuration to work for WCF endpoints and certain settings, but now I’ve got that figured out I would recommend using this over the alternatives (xml transformations, build script copying).

 

Pre-requisites:

  • VS 2010 (or latest MSBuild if deploying on server)
  • Web Deploy tool (included in VS 2010, only needed for server deploy)
  • MVC3 (included in VS 2010, only needed for server deploy)

 

The project consists of a Model class library, WCF service and an MVC3 application. The Model has a service reference to the WCF service and a custom settings file, the MVC application references the Model, both the WCF service and MVC application have environment configuration, like appsettings keys, settings sections, endpoint addresses and connectionstrings (in dbproperties.config). The master build script builds the solution and creates the web deployment packages, the Deployer script checks/creates the target host web site in IIS, substitutes the correct environment configuration into the package and deploys.

Web

Important files in the project are:

  • Parameters.xml – In both web applications, contains the list of environment configuration settings for each app, with a description and instructions for the deployer on how to find/set them (either xpath or simple match substitution)
  • SrcConfig{env}Kainos.WebBRIC.{app}.SetParameters.xml – Environmental settings used in deployment for app, contains the value to be used for each parameter in Parameters.xml and the target Virtual Directory (deployer will create if does not exist)
  • Master.build – Builds the deployment packages (ln38)
  • Deployer.build – Checks/Creates host website (using MSBuild Extension Package tasks) for IIS6 or IIS7, copies target environment configuration to package and deploys

Parameters
Setparameters

You can use the MSBuild Extension Package tasks to do a lot more than I’ve got in the sample, create host headers/app pools and setup the website correctly, but I think in the majority of cases the web site will already exist with correct configuration so this may be redundant. One thing people using the deploy tool need to be aware of is it will delete all files/folders in the existing virtual directory folder which don’t exist in the package, so they will need to backup any log files etc. if they want to keep them after a re-deployment.

Attached zipped solution embedded into word doc, as Posterous will not accept zips.

Using .NET Web Deployment Packages for single build releases parameterized for multiple environment deployments

The titles a bit of a mouth full but I think this is a common requirement for deployments, where a release is built once and then deployed for different environments (e.g. first UAT then Production when finished testing).

When I first used Visual Studio 2010 I noticed the new Web.config.Debug/Web.config.Release transformation and thought it would be a handy feature, but after looking at build/deployment scripts for a project I’ve realised it’s useless for the kind of releases I do. I’ve never deployed directly from Visual Studio, it’s always been build the release via msbuild then copy the release onto target environment and use msbuild to perform deployment tasks. Setting environment specific configuration values was either done manually (nasty) or by xml transformations in msbuild script (messy). Web.config transformations seemed to fix this but require one unacceptable compromise, separate builds per environment for a single release, since the transformation is only applied at on build and it’s apparently not possible to use the same internal targets to transform the configs afterwards for deployments (please correct me if you know how). Since multiple builds makes tracing issues more difficult I wouldn’t want to risk using it.

As an alternative you can use Web Deployment Packages, which build and zip up your web application (asp/mvc/wcf etc.) with a neat script for deploying it straight onto the web server (creating virtual directories automatically) and using a “SetParameters.xml” file to hold environment specific configuration. If in your application you include a file called “parameters.xml” with all environment specific configuration (see here for details) when you build deployment package (by msbuild in the release or right clicking in VS) your “SetParameters.xml” will contain the names/default values of the configuration values. Its then a matter of maintaining environment specific versions of the SetParameters.xml file and using them when deploying the web packages by command line.

 

Capture
Capture1

See this blog post for details on how to set parameters correctly and get the web deployment package from here.

ASP.MVC – JQuery – Hide/Show elements using class

Not specifically ASP.MVC related, but I used JQuery because of it so I’m grouping it in anyway.

Created for a screen which needed to hide/show controls based on a radio button list selection, something which I’ve done before but the simplicity of how to implement it with JQuery impressed me so I’m blogging it.

It uses two bits of standard JQuery functionality, selecting elements based on class and applying css styles to the selection of elements.

  • Decorate the elements to hide/show with css classes which will control when they are displayed, table rows in this example:

Capture

  • Then create a Javascript function which will use the JQuery class selector to find those elements and set the css to hide/show, this function will be called when the document is loaded to setup the page based on the selected radio button value.

Capture

Simple! Doing it using plain JS would have been much longer and harder to maintain (changing the elements to display just requires modifying the classes).

 

ASP.MVC – Custom validation without using Data Annotations

This is intended as one of a series of posts on ASP.MVC 3 (Razor) about tips and traps as I learn, feel free to comment with advice or anything to help me or others.

All the ASP.MVC tutorials I’ve seen show validation being performed by using Data Annotations on the Model properties and calling ModelState.IsValid in the controller. This is great for normal validation but has some drawbacks when you need to do some complicated stuff.

I had a screen containing a radio button list, clicking a radio button hide or displayed certain fields, changing the mandatory fields for the screen. This made using the base MVC 3 Data Annotations not an option, as marking any of the fields as required would cause them to always be required regardless of the selected radio button. If this was something I would reuse I would probably create a custom validation attribute (link on how to do that) but it was only needed for that screen and quite specific.

Instead I used the ModelState.AddModelError, which allowed me to put my custom validation in the controller:

Capture

Some points to note:

  • The field name you supply in AddModelError is used by your “ValidationMessageFor” tags in the view, so will affect how control styles. Use a field name for a button/hidden field if you don’t want validation styles applied to a specific field control for the error.
  • If the field uses DataType validation (e.g. is an int/DateTime in model) and the value entered was invalid for the DataType using AddModelError will cause the DataType validation error message to display as well as your custom message.