Building data driven dashboards with Django

Here at Wildfish, we’ve been working away on django-dashboards to support some of our clients. The aim was to build a package for quickly building data-driven dashboards in a “django like” way, closely coupling with the ORM on other key features of Django and minimising the frontend development required to build a new dashboard. The aim of this post is to provide a quick intro to django-dashboards and how to get started.

Often I find when picking up something new that it helps if I can relate it to a topic I’m interested in, for me that’s often football. I find the stats, predictions, and analysis of games fascinating. With that in mind, we’re going to run over the steps needed to create a new dashboard based on the excellent (and open!) data https://projects.fivethirtyeight.com/soccer-predictions/premier-league/ provides for Club Soccer Football Predictions.

If you're not familiar with this data, not to worry, simply put it ranks football teams. Giving each team an SPI (Soccer Power Index) score which among other factors rates the likelihood of a match result and overall league position by the end of the season. This is where my interest comes in, as I regularly check after a round of matches the likelihood of where my team will finish.

Here is what we will have running by the end of this post

Getting started

You can view the full source code for this post at https://github.com/wildfish/django-dashboards-football-rankings-blog & we’ll be skipping over the pulling and modelling of the data from FiveThirtyEight’s data in this post (it’s actually pretty simple), but you will need to pull a few files as we go, alternatively clone the repo and follow along that way.

In your terminal (linux/mac):

mkdir football-rankings
cd football-rankings

# we use pyenv but feel free to use the virtualenv of your choice
pyenv virtualenv 3.11.0 django-dashboards-football-rankings
pyenv activate django-dashboards-football-rankings

pip install django-dashboards

django-admin startproject demo .
cd demo
django-admin startapp football

You now need to modify a couple of files to setup django-dashboards within your Django project:

# settings.py
# append dashboards and our new app to INSTALLED_APPS

INSTALLED_APPS = [
    ...
    "dashboards",
    "demo.football",
]


# urls.py
# add dashboard.urls as a path 
urlpatterns = [
    ...
    path(
        "dashboard/",
        include("dashboards.urls"),
    ),
]

# apps.py
# import dashboards on AppConfig.ready 

class FootballConfig(AppConfig):
    default_auto_field = "django.db.models.BigAutoField"
    name = "demo.football"

    def ready(self):
        # for registry
        import demo.football.dashboards  # type: ignore # noqa

If you’re following along, now go and grab models.py & the management folder from the source code and place them in your football app. Finally, make and apply the migrations:

python manage.py makemigrations
python manage.py migrate

# pull the data from FiveThirtyEight, not you can rerun this as it deletes existing date
python manage.py import

First Dashboard

Now we have our Django project setup and our data imported we can start creating a dashboard, which will be a Global Rankings dashboard based on the SPIGlobalRank model.

Create a football/dashboards.py adding following:

from dashboards.component import Table
from dashboards.dashboard import Dashboard
from dashboards.registry import registry

from demo.football.serializers import TeamRankTableSerializer


class RankingsDashboard(Dashboard):
    teams = Table(defer=TeamRankTableSerializer, grid_css_classes="span-12")

    class Meta:
        name = "Global Football SPI Rankings"

registry.register(RankingsDashboard)

There are few things to note here:

• The dashboard currently has one component teams which is a Table component.
• table defers which means when the dashboard is loaded the table will be fetched once the component is in viewport not on initial load. This is useful if your dashboard has lots of complex components or data and you want to stagger the loading. Alternatively, use the keyword value to fetch it on the first load.
• TeamRankTableSerializer is passed to defer, more on that later but for now it’s important to note that keywords value & defer can either be a static value, callable, or a class with a serialise function. In this case, we will be leveraging the latter.
• grid_css_classes leverages some inbuilt CSS to say we want this table to fill the full 12-section grid when displayed, ideally you’d roll your own CSS here but this helps you get started.
• Finally, we register the dashboard, this enables the automatic urls/views for this dashboard, similar to the Django admin.

With our dashboard in place, we now need to create our TeamRankTableSerializer, this uses one of the inbuilt serializers within django-dahsboards. Add the following to a football/serlaizers.py file:

from dashboards.component.table import TableSerializer

from demo.football.models import SPIGlobalRank


class TeamRankTableSerializer(TableSerializer):
    class Meta:
        columns = {
            "rank": "Rank #",
            "prev_rank": "Prev #",
            "team__name": "Team",
            "team__league__name": "League",
            "offence": "Offence",
            "defence": "Defence",
            "spi": "SPI",
        }

    def get_queryset(self, *args, **kwargs):
        return SPIGlobalRank.objects.all().select_related("team", "team__league")

If you’re familiar with Django, this hopefully is self explanatory and close to patterns you’ve already seen on other packages.

• TableSerializer allows you to connect your models to a Table component, serialising the data to a format expected within our rendering.
• We are defining a column map of the model field to a table column name.
• Defines a get_queryset of the data to populate the table, including some basic optimisation. You can also point directly to a model within Meta similar to Django admin.

Now we have our first dashboard & component in place, within your terminal start runserver

python manage.py runserver

Now visit http://127.0.0.1:8000/dashboard/football/rankingsdashboard/ in your browser and you will see the following:

Table components in django-dashboard are powered by datatables and there are various config options you can pass down, see the docs for more details. You can also make a BasicTable which is a standard table without datatables and it’s also possible to create serialisers without the ORM.

Adding charts

Table is just one of the components included with django-dashboards, let’s add some Chart components to our dashboard. Update dashboards.py:

from dashboards.component import Chart, Table
from dashboards.dashboard import Dashboard

from demo.football.serializers import (
    BigFiveSPIbyLeagueSerializer,
    OffenceVsDefenceSerializer,
    TeamRankTableSerializer,
    TopSPIbyLeagueSerializer,
)


class RankingsDashboard(Dashboard):
    teams = Table(defer=TeamRankTableSerializer, grid_css_classes="span-12")
    top_spi_by_league = Chart(defer=TopSPIbyLeagueSerializer, grid_css_classes="span-6")
    big_five_spi_by_league = Chart(
        defer=BigFiveSPIbyLeagueSerializer, grid_css_classes="span-6"
    )
    offence_vs_defence = Chart(
        defer=OffenceVsDefenceSerializer, grid_css_classes="span-12"
    )

    class Meta:
        name = "Global Football SPI Rankings"

Here we’ve added 3 more Chart components and now we need to add the serializers to serializers.py:

from typing import Any
from django.db.models import Sum

import pandas as pd
import plotly.express as px
import plotly.graph_objs as go
from dashboards.component.chart import ChartSerializer
from dashboards.component.table import TableSerializer

from demo.football.models import SPIGlobalRank

class TeamRankTableSerializer(TableSerializer):
    …

class DarkChartSerializer(ChartSerializer):
    def apply_layout(self, fig: go.Figure):
        fig = super().apply_layout(fig)
        return fig.update_layout(
            template="plotly_dark",
            plot_bgcolor="rgba(0,0,0,0.05)",
            paper_bgcolor="rgba(0,0,0,0.05)",
        )


class TopSPIbyLeagueSerializer(DarkChartSerializer):
    def get_queryset(self, *args, **kwargs):
        return (
            SPIGlobalRank.objects.values("team__league__name")
            .annotate(total=Sum("spi"))
            .values("team__league__name", "total")
            .order_by("-total")[:10]
        )

    def to_fig(self, data: Any) -> go.Figure:
        fig = px.bar(
            data,
            x=data["team__league__name"],
            y=data["total"],
        )
        return fig

    class Meta:
        title = "Leagues with highest overall SPI"

We’ve now added TopSPIbyLeagueSerializer which is similar to how we defined our Table serializer, the key difference here is that we leverage Plotly express to return a figure. This will result in a bar chart showing the leagues with the highest combined SPI:

• ChartSerializers require a Meta.model or a get_queryset to return the data for this chart. We use get_queryset here to return an annotation of leagues with the total SPI of the teams within the league.
• Here to_fig, turns our data into a bar chart with the league name as the x-axis and total SPI as the y-axis.
• to_fig must return a Plotly figure for our template rendering, however, you could change this to be another library such as highcharts by building your own components/serializers.
• Within ChartSerializer, the data argument is the QuerySet converted to a pandas Dataframe meaning any additional transformation you need to apply to the data outside of the database can be added here.
• We also created DarkChartSerializer, which simply changes the look of the Plotly charts and is to show an example of how your serializers could share common aspects.

Now let’s add the other 2 serializers:

…

BIG_FIVE = [
    "Barclays Premier League",
    "German Bundesliga",
    "Spanish Primera Division",
    "Italy Serie A",
    "French Ligue 1",
]

class BigFiveSPIbyLeagueSerializer(DarkChartSerializer):
    def get_queryset(self, *args, **kwargs):
        return SPIGlobalRank.objects.values(
            "rank", "team__name", "team__league__name", "spi"
        ).filter(team__league__name__in=BIG_FIVE)

    def to_fig(self, data: Any) -> go.Figure:
        data["inverse_rank"] = data["rank"].values[::-1]
        fig = px.scatter(
            data,
            x="spi",
            y="rank",
            color="team__league__name",
            size="inverse_rank",
            hover_data=["team__name"],
        )
        fig["layout"]["yaxis"]["autorange"] = "reversed"
        return fig

    class Meta:
        title = "Big 5 SPI spread"


class OffenceVsDefenceSerializer(DarkChartSerializer):
    def get_queryset(self, *args, **kwargs):
        return SPIGlobalRank.objects.values(
            "rank", "offence", "defence", "spi", "team__name"
        )

    def to_fig(self, data: Any) -> go.Figure:
        data["rank_group"] = pd.cut(data["rank"], 5, labels=False)
        fig = go.Figure()
        fig.add_trace(go.Box(y=data["offence"], x=data["rank_group"], name="Offence"))
        fig.add_trace(go.Box(y=data["defence"], x=data["rank_group"], name="Defence"))
        fig.update_traces(boxpoints="all", jitter=0)
        fig["layout"]["xaxis"]["autorange"] = "reversed"
        return fig

    class Meta:
        title = "Offence vs Defence, teams split into 5 ranks 0 being the best."

These are quite similar to our first Chart in structure and will result in:

• A bubble chart showing the SPI spread across the ‘Big 5’ leagues.
• A box plot showing offence vs defence. This example also leverages Plotly graph objects directly vs Plotly express.
• Both these charts apply a little data transformation within to_fig also.

Now we can start runserver again:

python manage.py runserver

And again visit http://127.0.0.1:8000/dashboard/football/rankingsdashboard/ to see our table and our new charts.

What next

The source code for this blog also contains more dashboards, components and topics you might want to explore:

• Model dashboards - Dashboards that are for a specific object, see how we create a league table from a League object (https://github.com/wildfish/django-dashboards-football-rankings-blog/blob/main/demo/football/dashboards.py#L39) and a Team dashboard to start investigating more granular data.
• A custom component (https://github.com/wildfish/django-dashboards-football-rankings-blog/blob/main/demo/football/components.py#L8) to display team’s form

You can also head over to the docs to see what else you can do with django-dashboards including:

• More in-built components & dependent components
• Python driven layouts
• Async options