How to implement real time data for a web page

Question

(This is intended as a Q/A style question, intended to be a go-to resource for people that ask similar questions. A lot of people seem to stumble on the best way of doing this because they don't know all the options. Many of the answers will be ASP.NET specific, but AJAX and other techniques do have equivalents in other frameworks, such as socket.io and SignalR.)

I have a table of data that I have implemented in ASP.NET. I want to display changes to this underlying data on the page in real time or near real time. How do I go about it?

My Model:

public class BoardGame
    {
    public int Id { get; set;}
    public string Name { get; set;}
    public string Description { get; set;}
    public int Quantity { get; set;}
    public double Price { get; set;}

    public BoardGame() { }
    public BoardGame(int id, string name, string description, int quantity, double price)
        {
        Id=id;
        Name=name;
        Description=description;
        Quantity=quantity;
        Price=price;
        }
    }

In lieu of an actual database for this example, I'm just going to store the data in the Application variable. I'm going to seed it in my Application_Start function of my Global.asax.cs.

var SeedData = new List<BoardGame>(){
    new BoardGame(1, "Monopoly","Make your opponents go bankrupt!", 76, 15),
    new BoardGame(2, "Life", "Win at the game of life.", 55, 13),
    new BoardGame(3, "Candyland", "Make it through gumdrop forrest.", 97, 11)
    };
Application["BoardGameDatabase"] = SeedData;

If I were using Web Forms, I'd display the data with a repeater.

<h1>Board Games</h1>
        <asp:Repeater runat="server" ID="BoardGameRepeater" ItemType="RealTimeDemo.Models.BoardGame">
            <HeaderTemplate>
                <table border="1">
                    <tr>
                        <th>Id</th>
                        <th>Name</th>
                        <th>Description</th>
                        <th>Quantity</th>
                        <th>Price</th>
                    </tr>
            </HeaderTemplate>
            <ItemTemplate>
                <tr>
                    <td><%#: Item.Id %></td>
                    <td><%#: Item.Name %></td>
                    <td><%#: Item.Description %></td>
                    <td><%#: Item.Quantity %></td>
                    <td><%#: Item.Price %></td>
                </tr>
            </ItemTemplate>
            <FooterTemplate></table></FooterTemplate>
        </asp:Repeater>

And load that data in the code behind:

protected void Page_Load(object sender, EventArgs e)
    {
    BoardGameRepeater.DataSource = Application["BoardGameDatabase"];
    BoardGameRepeater.DataBind();
    }

If this were MVC using Razor, it's just a simple foreach over the model:

@model IEnumerable<RealTimeDemo.Models.BoardGame>
<h1>Board Games</h1>
<table border="1">
    <tr>
        <th>
            @Html.DisplayNameFor(model => model.Id)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.Name)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.Description)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.Quantity)
        </th>
        <th>
            @Html.DisplayNameFor(model => model.Price)
        </th>
    </tr> 
@foreach (var item in Model) {
    <tr>
        <td>
            @Html.DisplayFor(modelItem => item.Id)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.Name)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.Description)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.Quantity)
        </td>
        <td>
            @Html.DisplayFor(modelItem => item.Price)
        </td>
    </tr>
} 
</table>

Let's use Web Forms to have a little page for adding data so we can then watch the data update in real time. I recommend you create two browser windows so you can see the form and table at the same time.

<h1>Create</h1>
<asp:Label runat="server" ID="Status_Lbl" /><br />
Id: <asp:TextBox runat="server" ID="Id_Tb" /><br />
Name: <asp:TextBox runat="server" ID="Name_Tb" /><br />
Description: <asp:TextBox runat="server" ID="Description_Tb" /><br />
Quantity: <asp:TextBox runat="server" ID="Quantity_Tb" /><br />
Price: <asp:TextBox runat="server" ID="Price_Tb" /><br />
<asp:Button runat="server" ID="SubmitBtn" OnClick="SubmitBtn_Click" Text="Submit" />

And the code behind:

protected void SubmitBtn_Click(object sender, EventArgs e)
    {
    var game = new BoardGame();
    game.Id = Int32.Parse(Id_Tb.Text);
    game.Name = Name_Tb.Text;
    game.Description = Description_Tb.Text;
    game.Quantity = Int32.Parse(Quantity_Tb.Text);
    game.Price = Int32.Parse(Price_Tb.Text);
    var db = (List<BoardGame>)Application["BoardGameDatabase"];
    db.Add(game);
    Application["BoardGameDatabase"] = db;
    //only for SignalR
    /*var context = GlobalHost.ConnectionManager.GetHubContext<GameHub>();
    context.Clients.All.addGame(game); */           
    }

Answer 1

AJAX Polling, better implementation

Similar to the other AJAX based answer, you can continually poll the server. But this time, instead of responding with the data to display, we're going to reply with a list of ID's of the data. The client side is going to keep track of the data it's already retrieved in an array then it will make a separate GET request to the server for data when it sees a new ID has been added.

Here's our page code:

<h1>Board Games (AJAX Polling Good)</h1>
        <table id="BoardGameTbl" border="1">
            <thead>
                <tr>
                    <th>Id</th>
                    <th>Name</th>
                    <th>Description</th>
                    <th>Quantity</th>
                    <th>Price</th>
                </tr>
            </thead>
            <tbody id="BoardGameTblBody">
            </tbody>
        </table>
        <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
        <script type="text/javascript">
            var loadedGames = [];
            function getListOfGames() {
                $.ajax({
                    type: "GET",
                    url: "api/GamesApi/GetGameIds",
                    dataType: "json"
                })
                .done(function (data) {
                    for (i = 0; i < data.length; i++) {
                        if (loadedGames.indexOf(data[i]) == -1) {
                            loadedGames[loadedGames.length] = data[i];
                            getGame(data[i]);
                        }
                    }
                    setTimeout(getListOfGames, 5000);
                });
            }
            function getGame(id) {
                $.ajax({
                    type: "GET",
                    url: "api/GamesApi/GetGame/" + id,
                    dataType: "json"
                })
                .done(function (game) {
                    $("#BoardGameTblBody").append("<tr><td>" + game.Id + "</td><td>" + game.Name + "</td><td>" + game.Description + "</td><td>" + game.Quantity + "</td><td>" + game.Price + "</td></tr>");
                });
            }
            getListOfGames();
        </script>

Here's the Web API Controller:

namespace RealTimeDemo.Controllers
{
public class GamesApiController : ApiController
    {
    [Route("api/GamesApi/GetGameIds")]
    public IEnumerable<int> GetGameIds()
        {
        var data = HttpContext.Current.Application["BoardGameDatabase"] as List<BoardGame>;
        var IDs = data.Select(x => x.Id);
        return IDs;
        }

    [Route("api/GamesApi/GetGame/{id}")]
    public BoardGame GetGame(int id)
        {
        var data = HttpContext.Current.Application["BoardGameDatabase"] as List<BoardGame>;
        return data.Where(x => x.Id == id).SingleOrDefault();
        }
    }

Now, this is a much better implementation than my other AJAX based answer and the Timer/UpdatePanel answer. Since we're only sending the ID's across every 5 seconds, it's a much lighter strain on network resources. It'd also be fairly trivial to handle no network connection situations, or to execute some sort of notification when new data has been loaded, such as throwing up a noty.

Advantages

• Doesn't send viewstate with request.
• Doesn't execute entire page lifecycle
• Only ID's are sent across the wire (could be improved if you sent a timestamp with the request, and only replied with data that's changed since the timestamp) as part of the polling. Only new objects are retrieved from the database.

Disadvantages - We're still polling, generating a request every few seconds. If the data doesn't change very often, you're needlessly using up bandwidth.

Answer 2

Timer/UpdatePanel

If you are using Web Forms, you can use a control called an UpdatePanel. The UpdatePanel is capable of refreshing sections of the page asynchronously, without causing a postback of the entire page. Combined with an asp:Timer, you can have the table update as often as you like. Here's the code:

<asp:ScriptManager runat="server" />
        <h1>Board Games (using Update Panel)</h1>
        <asp:Timer runat="server" ID="UP_Timer" Interval="5000" />
        <asp:UpdatePanel runat="server" ID="Game_UpdatePanel">
            <Triggers>
                <asp:AsyncPostBackTrigger ControlID="UP_Timer" EventName="Tick" />
            </Triggers>
            <ContentTemplate>
                <asp:Repeater runat="server" ID="BoardGameRepeater" ItemType="RealTimeDemo.Models.BoardGame">
                    <HeaderTemplate>
                        <table border="1">
                            <tr>
                                <th>Id</th>
                                <th>Name</th>
                                <th>Description</th>
                                <th>Quantity</th>
                                <th>Price</th>
                            </tr>
                    </HeaderTemplate>
                    <ItemTemplate>
                        <tr>
                            <td><%#: Item.Id %></td>
                            <td><%#: Item.Name %></td>
                            <td><%#: Item.Description %></td>
                            <td><%#: Item.Quantity %></td>
                            <td><%#: Item.Price %></td>
                        </tr>
                    </ItemTemplate>
                    <FooterTemplate></table></FooterTemplate>
                </asp:Repeater>
            </ContentTemplate>
        </asp:UpdatePanel>

And the code behind:

    protected void Page_Load(object sender, EventArgs e)
        {
        BoardGameRepeater.DataSource = Application["BoardGameDatabase"];
        BoardGameRepeater.DataBind();
        }

So let's talk about how this one works. Every 5 seconds, the timer is going to fire a Tick event. This is registered as an asynchronous postback server with the UpdatePanel, so a partial postback occurs, the entire page lifecycle runs again, so it reloads the data on the Page Load event, then the entire contents of of the UpdatePanel's content template is replaced with freshly generated data from the server. Let's look at how the network traffic might look:

+5s Client => Server | Run the page lifecycle, send me the contents of the ContentPanel.
Server => Client | Here's the entire contents of the ContentPanel.
+10s Client => Server | Run the page lifecycle, send me the contents of the ContentPanel.
Server => Client | Here's the entire contents of the ContentPanel.
+15s Client => Server | Run the page lifecycle, send me the contents of the ContentPanel.
Server => Client | Here's the entire contents of the ContentPanel.

Advantages:

• Simple to implement. Just add a timer, a script manager, and wrap the repeater in an update panel.

Disadvantages:

• Heavy: The ViewState is sent to the server with every request. The impact of this can be lightened if you disable ViewState however (which you should do anyways).
• Heavy: Whether the data has changed or not, you're sending all of the data over the line every 5 seconds. That's a huge chunk of bandwidth.
• Slow: Takes a long time with each partial postback since all the data goes across the wire.
• Tough to work with: When you begin adding more functionality, it can be tricky to correctly handle the partial postbacks properly.
• Not smart: Even if the previous request didn't finish, it'll continue posting back thanks to the timer.
• Not smart: No easy way to handle network interruption.

Answer 3

AJAX Polling, poor implementation

If you are using MVC or Web Forms, you can implement a technique called AJAX polling. This will constantly send an AJAX request to the server. The server will send a response containing the latest data. It is incredibly simple to implement. You don't have to use jQuery to use AJAX, but it makes it a lot easier. This example is going to use Web API for the server side functionality. Web API is similar to MVC, it uses routing and controllers to process requests. It's the replacement for ASMX Web Services.

This is the web forms code, but it's very similar to the MVC code so I'm going to omit that:

<h1>Board Games (AJAX Polling Bad)</h1>
        <table id="BoardGameTbl" border="1">
            <thead>
                <tr>
                    <th>Id</th>
                    <th>Name</th>
                    <th>Description</th>
                    <th>Quantity</th>
                    <th>Price</th>
                </tr>
            </thead>
            <tbody id="BoardGameTblBody">
            </tbody>
        </table>
        <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
        <script type="text/javascript">
            function getData() {
                $.ajax({
                    type: "GET",
                    url: "api/GamesApi/GetGameData",
                    dataType: "json"
                })
                .done(function (data) {
                    $("#BoardGameTblBody").empty();
                    for (i = 0; i < data.length; i++) {
                        $("#BoardGameTblBody").append("<tr><td>" + data[i].Id + "</td><td>" + data[i].Name + "</td><td>" + data[i].Description + "</td><td>" + data[i].Quantity + "</td><td>" + data[i].Price + "</td></tr>");
                    }
                    setTimeout(getData, 5000);
                });
            }
            getData();
        </script>

This is making a request to a Web API. The API is returning a JSON representation of all the games.

public class GamesApiController : ApiController
    {
    [Route("api/GamesApi/GetGameData")]
    public IEnumerable<BoardGame> GetGameData()
        {
        var data = HttpContext.Current.Application["BoardGameDatabase"] as List<BoardGame>;
        return data;
        }
    }

The overall result of this method is similar to the Timer/UpdatePanel method. But it doesn't send any viewstate data with the request, and it doesn't execute a long page lifecycle process. You also don't have to dance around detecting if you're in a postback or not, or if you're in a partial postback or not. So I consider this an improvement over Timer/UpdatePanel.

However, this method still has one of the major disadvantages of the Timer/UpdatePanel method. You're still sending all of the data over the wire with each AJAX request. If you look at my other AJAX based answer, you'll see a better way to implement AJAX polling.

Advantages

• Doesn't send viewstate with request.
• Doesn't execute entire page lifecycle

Disadvantages

• Generates a request every few seconds
• Response includes all data, even if it hasn't changed

Answer 4

SignalR

This is the answer I'm most excited to share, because it represents a much cleaner implementation that is lightweight and works well in today's mobile (data constricted) environment.

There have been several methods over the years to provide "realtime" pushing of data from the server to the client (or the appearance of pushing data). Rapid Short Polling (similar to my AJAX based answers), Long Polling, Forever Frame, Server Sent Events, and WebSockets are different transport mechanisms used to achieve this. SignalR is an abstraction layer capable of selecting an appropriate transport mechanism based on the client and server's capabilities. The best part of using SignalR is that it's simple. You don't have to worry about the transport mechanism, and the programming model is easy to understand.

I'm going to define a SignalR hub, but just leave it empty.

public class GameHub : Hub
    {
    }

When I add data to the "database", I'm going to run the below bit of code. If you read the question, you'll see I commented it out in the "create" form. You'll want to uncomment that.

var context = GlobalHost.ConnectionManager.GetHubContext<GameHub>();
context.Clients.All.addGame(game);

Here's my page code:

<h1>SignalR</h1>
        <asp:Repeater runat="server" ID="BoardGameRepeater" ItemType="RealTimeDemo.Models.BoardGame">
            <HeaderTemplate>
                <table border="1">
                    <thead>
                        <tr>
                            <th>Id</th>
                            <th>Name</th>
                            <th>Description</th>
                            <th>Quantity</th>
                            <th>Price</th>
                        </tr>
                    </thead>
                    <tbody id="BoardGameTblBody">
            </HeaderTemplate>
            <ItemTemplate>
                <tr>
                    <td><%#: Item.Id %></td>
                    <td><%#: Item.Name %></td>
                    <td><%#: Item.Description %></td>
                    <td><%#: Item.Quantity %></td>
                    <td><%#: Item.Price %></td>
                </tr>
            </ItemTemplate>
            <FooterTemplate></tbody></table></FooterTemplate>
        </asp:Repeater>
        <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
        <script src="Scripts/jQuery-1.6.4.min.js"></script>
        <script src="Scripts/jquery.signalR-2.1.1.min.js"></script>
        <script src="signalr/hubs"></script>
        <script type="text/javascript">
            var hub = $.connection.gameHub;
            hub.client.addGame = function (game) {
                $("#BoardGameTblBody").append("<tr><td>" + game.Id + "</td><td>" + game.Name + "</td><td>" + game.Description + "</td><td>" + game.Quantity + "</td><td>" + game.Price + "</td></tr>");
            };
            $.connection.hub.start();
        </script>

And the code behind:

protected void Page_Load(object sender, EventArgs e)
        {
        BoardGameRepeater.DataSource = Application["BoardGameDatabase"];
        BoardGameRepeater.DataBind();
        }

Notice what's happening here. When the server calls context.Clients.All.addGame(game); it's executing the function that's been assigned to hub.client.addGame for every client that is connected to the GameHub. SignalR is taking care of wiring up the events for me, and automatically converting my game object on the server to the game object on the client. And best of all, there's no network traffic back and forth every few seconds, so it's incredibly lightweight.

Advantages:

• Very light on the network traffic
• Easy to develop, but still flexible
• Doesn't send viewstate with the request
• Doesn't poll the server continuously.

Note, you could add a function on the client for editedGame for pushing changed data out to the client easily (same for delete).