Adding National Grid’s DFS Dates to Home Assistant
The National Grid has devised an extensive toolkit to avoid electricity blackouts in the UK during this winter. As part of this, they’ve created a scheme called the Demand Flexibility Service, or DFS for short. The scheme pays a financial incentive to end users who can reduce their electricity usage during times of peak demand. The scheme was trialled earlier in the year with the energy supplier Octopus Energy, one of the more innovative suppliers available in the UK. After the initial trial, the DFS scheme has been widened to allow other energy suppliers to enrol
Several planned test ‘events’ will take place between November 2022 and March 2023. Each event requires the end user to opt in via their energy supplier. The events are split into half-hourly sessions, with participants’ use during each session compared to that of eligible previous days. The difference is credited to the energy supplier which is in turn credited to the end user.
For reference, typical unit prices for energy import are around £0.40 per kWh. The scheme is currently paying energy suppliers £3.00 per kWh saved. Most suppliers are passing on £2.20+ of this to the end user, so the incentive is quite substantial.
Obtaining the Data
The National Grid provide a Data Portal on their website. Through this,both the Live and Test event data is available for consumption. Of particular interest is the Utilisation Report dataset under each event type. These reports detail the energy suppliers who have requested (bid) to participate in each event, and whether their bid was accepted.
It is possible to create a number of sensors in Home Assistant to poll this data. In my sensor YAML below, I have filtered the data to Octopus Energy, and to only include successful bids, as other data is irrelevant. To change this to your own supplier, open the DFS Utilisation Report and Add a Filter for DFS Provider. Enter the name of your supplier as it appears in the report data. Add another filter for Status of Bid and enter Accepted. Press the Submit button to filter the table below. If it all looks correct, you can then press the Query Builder button. This will allow you to copy the URL that you can use to pull this data. Simply change the value for the resource in the below YAML with the URL you’ve just generated.
Home Assistant Sensor YAML
I strongly suggest using a package to keep the sensor yaml together rather than cluttering up your configuration file. The code below should be placed in file called national_grid_dfs.yaml located in the directory /homeassistant/packages/National Grid DFS/. You can of course pull the relevant code out of the below example and directly insert it in your configuration file.
This package will create four sensors, one for the start date and time, and one for end date and time for the latest Test and and Live events. This loops through the data checking for the latest event, and then keeps looking to find the start time of the first session, and the end time of the last session in that event.
national_grid_dfs:
## National Grid Demand Flexibility Service
sensor:
# Test Events - Octopus Accepted
# Start Time and Date
- platform: rest
resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%22fc466b76-6823-4dd9-8b4a-aab464e0e77b%22%20WHERE%20%22Status%22%20=%20'Accepted'%20AND%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
scan_interval: 900
name: National Grid ESO Latest Test DFS Start
unique_id: national_grid_eso_latest_test_dfs_start
value_template: >-
{% set d = value_json['result']['records'][0]['Date'] %}
{% if '/' in d %}
{% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%d/%m/%Y %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') <= ns.start_time -%}
{% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% else %}
{% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%Y-%m-%d %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') <= ns.start_time -%}
{% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% endif %}
{{ns.start_time}}
# End Time and Date
- platform: rest
resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%22fc466b76-6823-4dd9-8b4a-aab464e0e77b%22%20WHERE%20%22Status%22%20=%20'Accepted'%20AND%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
scan_interval: 900
name: National Grid ESO Latest Test DFS End
unique_id: national_grid_eso_latest_test_dfs_end
value_template: >-
{% set d = value_json['result']['records'][0]['Date'] %}
{% if '/' in d %}
{% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%d/%m/%Y %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') >= ns.end_time -%}
{% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% else %}
{% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%Y-%m-%d %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') >= ns.end_time -%}
{% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% endif %}
{{ns.end_time}}
# Live Events - Octopus Accepted
# Start Time and Date
- platform: rest
resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%224e87244e-2479-4e6e-84c2-698dffaca41f%22%20WHERE%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20AND%20%22Status%22%20=%20'Accepted'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
scan_interval: 900
name: National Grid ESO Latest Live DFS Start
unique_id: national_grid_eso_latest_live_dfs_start
value_template: >-
{% set d = value_json['result']['records'][0]['Date'] %}
{% if '/' in d %}
{% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%d/%m/%Y %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') <= ns.start_time -%}
{% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%d/%m/%Y %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% else %}
{% set ns=namespace(start_time = strptime(d+' '+value_json['result']['records'][0]['From (GMT)'],'%Y-%m-%d %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') <= ns.start_time -%}
{% set ns.start_time = strptime(d+' '+record['From (GMT)'],'%Y-%m-%d %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% endif %}
{{ns.start_time}}
# End Time and Date
- platform: rest
resource: "https://api.nationalgrideso.com/api/3/action/datastore_search_sql?sql=SELECT%20COUNT(*)%20OVER%20()%20AS%20_count,%20*%20FROM%20%224e87244e-2479-4e6e-84c2-698dffaca41f%22%20WHERE%20%22DFS%20Provider%22%20=%20'OctopusEnergyLimited'%20AND%20%22Status%22%20=%20'Accepted'%20ORDER%20BY%20%22_id%22%20ASC%20LIMIT%20100"
scan_interval: 900
name: National Grid ESO Latest Live DFS End
unique_id: national_grid_eso_latest_live_dfs_end
value_template: >-
{% set d = value_json['result']['records'][0]['Date'] %}
{% if '/' in d %}
{% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%d/%m/%Y %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') >= ns.end_time -%}
{% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%d/%m/%Y %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% else %}
{% set ns=namespace(end_time = strptime(d+' '+value_json['result']['records'][0]['To (GMT)'],'%Y-%m-%d %H:%M')) %}
{% for record in value_json['result']['records'] %}
{% if record['Date'] == d -%}
{% if strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') >= ns.end_time -%}
{% set ns.end_time = strptime(d+' '+record['To (GMT)'],'%Y-%m-%d %H:%M') %}
{%- endif %}
{%- endif %}
{% endfor %}
{% endif %}
{{ns.end_time}}
Using the Sensors
Now that you’ve got the data, its possible to create automations in Home Assistant to turn off any unnecessary devices at the start time of the event, and turn them back on at the end. To do this, I created 4 date-time helpers which trigger such an automation. To update the date-time helper you need to create an automation that runs on a regular basis, or is triggered by the refreshing of the sensors. You can use similar to the below in the action section of an automation. Repeat this for each of the helpers and their corresponding sensors.
- service: input_datetime.set_datetime
data:
datetime: >-
{{
strptime(states('sensor.national_grid_eso_latest_live_dfs_start'),
'%Y-%m-%d %H:%M:%S') }}
target:
entity_id: input_datetime.dfs_live_start
It certainly does GC – There is an updated version of this article for 23/24 as National Grid have updated the dataset and combined it into a single output. This has been set up to use Home Assistant’s package feature to keep the sensor configuration a little tidier and out of the main configuration.yaml file – follow the link to see how to set it up if you don’t already use them; its really great to keep clutter under control.
Does this still work? Where do I put the YAML, I’ve tried putting it in configuration.yaml but it doesn’t recognise “national_grid_dfs”, is there an integration to go along side this?
I’ve updated the sensor code to reflect that National Grid arbitrarily use two different date formats when uploading / updating their datasets