Adding National Grid’s DFS Dates to Home Assistant

The National Grid has devised an extensive toolkit to avoid electricity blackouts in the UK during this winter. As part of this, they’ve created a scheme called the Demand Flexibility Service, or DFS for short. The scheme pays a financial incentive to end users who can reduce their electricity usage during times of peak demand. The scheme was trialled earlier in the year with the energy supplier Octopus Energy, one of the more innovative suppliers available in the UK. After the initial trial, the DFS scheme has been widened to allow other energy suppliers to enrol

Several planned test ‘events’ will take place between November 2022 and March 2023. Each event requires the end user to opt in via their energy supplier. The events are split into half-hourly sessions, with participants’ use during each session compared to that of eligible previous days. The difference is credited to the energy supplier which is in turn credited to the end user.

For reference, typical unit prices for energy import are around £0.40 per kWh. The scheme is currently paying energy suppliers £3.00 per kWh saved. Most suppliers are passing on £2.20+ of this to the end user, so the incentive is quite substantial.

Obtaining the Data

The National Grid provide a Data Portal on their website. Through this,both the Live and Test event data is available for consumption. Of particular interest is the Utilisation Report dataset under each event type. These reports detail the energy suppliers who have requested (bid) to participate in each event, and whether their bid was accepted.

It is possible to create a number of sensors in Home Assistant to poll this data. In my sensor YAML below, I have filtered the data to Octopus Energy, and to only include successful bids, as other data is irrelevant. To change this to your own supplier, open the DFS Utilisation Report and Add a Filter for DFS Provider. Enter the name of your supplier as it appears in the report data. Add another filter for Status of Bid and enter Accepted. Press the Submit button to filter the table below. If it all looks correct, you can then press the Query Builder button. This will allow you to copy the URL that you can use to pull this data. Simply change the value for the resource in the below YAML with the URL you’ve just generated.

Home Assistant Sensor YAML

I strongly suggest using a package to keep the sensor yaml together rather than cluttering up your configuration file. The code below should be placed in file called national_grid_dfs.yaml located in the directory /homeassistant/packages/National Grid DFS/. You can of course pull the relevant code out of the below example and directly insert it in your configuration file.

This package will create four sensors, one for the start date and time, and one for end date and time for the latest Test and and Live events. This loops through the data checking for the latest event, and then keeps looking to find the start time of the first session, and the end time of the last session in that event.

national_grid_dfs:
## National Grid Demand Flexibility Service
    sensor:

        # Test Events - Octopus Accepted
        
        # Start Time and Date
      - platform: rest
        resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%22fc466b76-6823-4dd9-8b4a-aab464e0e77b%22%20WHERE%20%22Status%22%20=%20'Accepted'%20AND%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
        scan_interval: 900
        name: National Grid ESO Latest Test DFS Start
        unique_id: national_grid_eso_latest_test_dfs_start
        value_template: >-
            {% set d = value_json['result']['records'][0]['Date'] %}
            {% if '/' in d %}
                        {% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%d/%m/%Y %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') <= ns.start_time -%}
                              {% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% else %}
                        {% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%Y-%m-%d %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') <= ns.start_time -%}
                              {% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% endif %}         
            {{ns.start_time}}

        # End Time and Date
      - platform: rest
        resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%22fc466b76-6823-4dd9-8b4a-aab464e0e77b%22%20WHERE%20%22Status%22%20=%20'Accepted'%20AND%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
        scan_interval: 900
        name: National Grid ESO Latest Test DFS End
        unique_id: national_grid_eso_latest_test_dfs_end
        value_template: >-
            {% set d = value_json['result']['records'][0]['Date'] %}
            {% if '/' in d %}
                        {% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%d/%m/%Y %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') >= ns.end_time -%}
                              {% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% else %}
                        {% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%Y-%m-%d %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') >= ns.end_time -%}
                              {% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% endif %}         
            {{ns.end_time}}


        # Live Events - Octopus Accepted
        
        # Start Time and Date
      - platform: rest
        resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%224e87244e-2479-4e6e-84c2-698dffaca41f%22%20WHERE%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20AND%20%22Status%22%20=%20'Accepted'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
        scan_interval: 900
        name: National Grid ESO Latest Live DFS Start
        unique_id: national_grid_eso_latest_live_dfs_start
        value_template: >-
            {% set d = value_json['result']['records'][0]['Date'] %}
            {% if '/' in d %}
                        {% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%d/%m/%Y %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') <= ns.start_time -%}
                              {% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% else %}
                        {% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%Y-%m-%d %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') <= ns.start_time -%}
                              {% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% endif %}         
            {{ns.start_time}}

        # End Time and Date
      - platform: rest
        resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%224e87244e-2479-4e6e-84c2-698dffaca41f%22%20WHERE%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20AND%20%22Status%22%20=%20'Accepted'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
        scan_interval: 900
        name: National Grid ESO Latest Live DFS End
        unique_id: national_grid_eso_latest_live_dfs_end
        value_template: >-
            {% set d = value_json['result']['records'][0]['Date'] %}
            {% if '/' in d %}
                        {% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%d/%m/%Y %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') >= ns.end_time -%}
                              {% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% else %}
                        {% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%Y-%m-%d %H:%M')) %}
                        {% for record in value_json['result']['records'] %}
                          {% if record['Date'] == d -%}
                            {% if strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') >= ns.end_time -%}
                              {% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') %}
                            {%- endif %}
                          {%- endif %}
                        {% endfor %}
            {% endif %}         
            {{ns.end_time}}     

Using the Sensors

Now that you’ve got the data, its possible to create automations in Home Assistant to turn off any unnecessary devices at the start time of the event, and turn them back on at the end. To do this, I created 4 date-time helpers which trigger such an automation. To update the date-time helper you need to create an automation that runs on a regular basis, or is triggered by the refreshing of the sensors. You can use similar to the below in the action section of an automation. Repeat this for each of the helpers and their corresponding sensors.

      - service: input_datetime.set_datetime
        data:
          datetime: >-
            {{
            strptime(states('sensor.national_grid_eso_latest_live_dfs_start'),
            '%Y-%m-%d %H:%M:%S') }}
        target:
          entity_id: input_datetime.dfs_live_start