Skip to content

Plugin

This section contains information on how to update and run the sample plugin included in NOVA. The plugin is used to communicate to external systems, and can be thought of as the translation layer between OpenADR and the customer's APIs.

Plaid Diagram highlighting Plugin

The Plugin is written in C++. By default it uses HTTP POST, but can be updated to use any messaging system the customer's system prefers.

Sample HTTP Plugin

When licensing Plaid, customers get a sample plugin that implements HTTP POST. The fastest way to get started with Plaid is to use the provided sample plugin. Source for the sample plugin can be found in the following path: nova-source-root/pluginimpl/plugin/notifier/http/. Configuration for the plugin can be found in nova-source-root/pluginimpl/plugin/notifier/http/config/config.json.sample.

The config file consists of a series of http paths which will receive a json message when the corresponding OpenADR event occurs. See below for more information.

NOTE (as of 2020/06): the plugin has been updated and the exact fields included in your event messages may vary, please run some test events to confirm your message content. This will be finalized by end of 2020/07.

Plugin Overview

Plaid interacts with an INotifierPlugin plugin to allow custom plugin implementations to respond as necessary to the OpenADR messages without having to implement the OpenADR protocol. NotifierHttpPlugin is an implementation of INotifierPlugin that is supplied with Plaid. NotifierHttpPlugin posts JSON formatted messages for OpenADR events that are then to be handled by the custom plugin code. The following events are posted:

  • on event
  • start event
  • start event interval
  • cancel event
  • delete event
  • end event
  • status
  • start distribute event
  • end distribute event

Reply Format

The sample plugin expects replies to have the following format.

{
  "status": {
    "code": 200,
    "message": "OK"
  }
}

If the status code of the response is not 200, it is detected as an error by the message poster and will be retried. Subsequent messages will not be sent until the message has gone through. As of version 1.2, this restraint has been lifted.

JSON Formatted NotifierHttpPlugin Configuration

The endpoints NotifierHttpPlugin posts to are user defined in a JSON formatted configuration file. A sample configuration file called config.json.sample is supplied with NotifierHttpPlugin. All that is needed is to supply a URL for each of the desired endpoints. A sample configuration file is included with the plugin called config.json.sample.

Here is the JSON schema for the format of the NotifierHttpPlugin configuration file:

{
  "type": "object",
  "properties": {
    "endpoints": {
      "type": "object",
      "properties": {
        "event": { "type": "string" },
        "startEvent": { "type": "string" },
        "startEventInterval": { "type": "string" },
        "cancelEvent": { "type": "string" },
        "deleteEvent": { "type": "string" },
        "endEvent": { "type": "string" },
        "startDistributeEvent": { "type": "string" },
        "completeDistributeEvent": { "type": "string" },
      }
    },
    "deadLetterPollInSeconds": { "type": "number" },
    "instanceId": { "type": "string" },
    "vtnId": { "type": "string" },
    "venId": { "type": "string" }
  }
}

And example configuration file is supplied with the plugin.

JSON Formatted Messages

This section describes the format of the JSON messages sent to the endpoints.

OnEvent, OnEventStart

OnEvent is sent when a new event is created on the VTN and contains the details of the event.

OnEventStart is created at the start of the event. OnEventStartInterval is created at the start of each interval in the event (there are a minimum of one interval per event, and the intervals must be executed sequentially for the full length of the event). To avoid redundancy, we suggest implementing the OnEventStartInterval callback over the OnEventStart callback. For more information on implementing events, please read the event implementation page.

{
  "type": "object",
  "properties": {
    "eventId": { "type": "string" },
    "notifierEventMessage": { "type": "string" },
    "dtStartTimet": { "type": "integer" },
    "durationInSeconds": { "type": "integer" },
    "signalName": { "type": "string" },
    "signalType": { "type": "string" },
    "units": {
      "type": "object",
      "properties": {
        "description": { "type": "string" },
        "units": { "type": "string" },
        "siScaleCode": { "type": "string" }
      }
    },
    "payload": { "type": "number" },
    "status": { "type": "string" },
    "pluginApiVersion": { "type": "string" },
    "pluginVersion": { "type": "string" },
    "venId": { "type": "string" },
    "vtnId": { "type": "string" },
    "instanceId": { "type": "string" },
    "targets": {
      "type": "object",
      "properties": {
        "venId": { "type": "string" },
        "partyId": { "type": "string" },
        "groupIds": {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      },
      "resourceIds": {
        "type": "array",
        "items": {
          "type": "string"
        }
      }
    },
  }
}

An example payload:

{
  "eventId": "Event110916_151721_348_0",
  "notifierEventMessage": "OnEvent | StartEvent",
  "dtStartTimet": 1478722760,
  "durationInSeconds": 180,
  "signalName": "simple",
  "signalType": "level",
  "payload": 1,
  "status": "active",
  "pluginApiVersion": "1.0",
  "pluginVersion": "2.1",
  "venId": "TH_VEN",
  "vtnId": "Utilityco_VTN_1",
  "instanceId": "test",
  "targets": {
    "groupIds": [
      "22ad132d-043d-476f-aa79-ca2c8e63c13e",
      "2ba4a943-e3f2-4b2b-b406-4768973d5fe6"
    ],
    "partyId": "Party ID",
    "resourceIds": [
      "Resource2",
      "Resource1"
    ],
    "venId": "TH_VEN"
  }
}

Start Event Interval

The start event interval message is formatted the same as start event but with extra fields for interval start time and interval duration. There are no end event interval messages because each start event interval delineates the end of the previous event interval.

{
  "type": "object",
  "properties": {
    "eventId": { "type": "string" },
    "notifierEventMessage": { "type": "string" },
    "dtStartTimet": { "type": "integer" },
    "durationInSeconds": { "type": "integer" },
    "dtIntervalStartTimet": { "type": "integer" },
    "intervalDurationInSeconds": { "type": "integer" },
    "signalName": { "type": "string" },
    "signalType": { "type": "string" },
    "units": {
      "type": "object",
      "properties": {
        "description": { "type": "string" },
        "units": { "type": "string" },
        "siScaleCode": { "type": "string" }
      }
    },
    "payload": { "type": "number" },
    "status": { "type": "string" },
    "pluginApiVersion": { "type": "string" },
    "pluginVersion": { "type": "string" },
    "venId": { "type": "string" },
    "vtnId": { "type": "string" },
    "instanceId": { "type": "string" },
    "targets": {
      "type": "object",
      "properties": {
        "venId": { "type": "string" },
        "partyId": { "type": "string" },
        "groupIds": {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      },
      "resourceIds": {
        "type": "array",
        "items": {
          "type": "string"
        }
      }
    }
  }
}

An example payload:

{
  "eventId": "Event110916_151721_348_0",
  "notifierEventMessage": "StartEventInterval",
  "dtStartTimet": 1478722760,
  "durationInSeconds": 180,
  "dtIntervalStartTimet": 1478722760,
  "intervalDurationInSeconds": 90,
  "signalName": "simple",
  "signalType": "level",
  "payload": 1,
  "status": "active",
  "pluginApiVersion": "1.0",
  "pluginVersion": "2.1",
  "venId": "TH_VEN",
  "vtnId": "Utilityco_VTN_1",
  "instanceId": "test",
  "targets": {
    "groupIds": [
      "22ad132d-043d-476f-aa79-ca2c8e63c13e",
      "2ba4a943-e3f2-4b2b-b406-4768973d5fe6"
    ],
    "partyId": "Party ID",
    "resourceIds": [
      "Resource2",
      "Resource1"
    ],
    "venId": "TH_VEN"
  }
}

Cancel Event, Delete Event, End Event

The cancel, delete, and end events have the same message format:

{
  "type": "object",
  "properties": {
    "eventId": { "type": "string" },
    "notifierEventMessage": { "type": "string" },
    "pluginApiVersion": { "type": "string" },
    "pluginVersion": { "type": "string" },
    "venId": { "type": "string" },
    "vtnId": { "type": "string" },
    "instanceId": { "type": "string" },
  }
}

And an example payload:

{
  "eventId": "Event110916_151721_348_0",
  "notifierEventMessage": "CompleteEvent",
  "pluginApiVersion": "1.0",
  "pluginVersion": "2.1",
  "venId": "TH_VEN",
  "vtnId": "Utilityco_VTN_1",
  "instanceId": "test",
}

Status

The status message has the following format

{
  "type": "object",
  "properties": {
    "notifierEventMessage": { "type": "string" },
    "pluginApiVersion": { "type": "string" },
    "pluginVersion": { "type": "string" }
  }
}

And an example payload:

{
  "notifierEventMessage": "Status",
  "pluginApiVersion": "1.0",
  "pluginVersion": "2.1",
  "venId": "TH_VEN",
  "vtnId": "Utilityco_VTN_1",
  "instanceId": "test",
}

Custom Messages

Plaid allows one custom namespace to be registered by the plugin in the NovaNotifierHttpPlugin::Initialize function. When Plaid receives a message for this namespace in the http api, the message is routed to the NovaNotifierHttpPlugin::OnRegisteredMessage function in the plugin for user code to interpret the message contents. The incoming message to Plaid is as follows:

{
  "type": "object",
  "properties":{
    "namespace": { "type": "string" },
    "parameters": {
      "type": "object",
      "properties":{

      }
    }
  }
}

And an example payload:

{
  "namespace": "nova.custom.namespace",
  "parameters": {
    "function": "getDeadLetterQueue"
  }
}

The example above is already implemented in the plugin and will return a json array of messages that expired before they were successfully posted. If a message is posted to Plaid that has just the namespace, the example plugin will send a list of events.

Start and End Distribute Event

startDistributeEvent and endDistributeEvent endpoints are POSTed to when an event is being distributed and when it has finished distribution. There is no body in these messages.

Message Field Description

The following list describes the fields that are found in the JSON messages posted for INotifierPlugin consumption:

  • eventId - unique identification of the event created by the VTN
  • notifierEventMessage - set to OnNewEvent, OnStartEvent, OnStartEventInterval, OnUpdateEvent, OnCancelEvent, OnDeleteEvent, or OnCompleteEvent - updated on Dec 12, 2019: updated name from notifierEventType, added prefix On
  • dtStartTimet - seconds from epoch (January 1, 1970 00:00:00 GMT) of the start time of the event
  • durationInSeconds - how long the event lasts in seconds (this spans all intervals of the event)
  • dtIntervalStartTimet - seconds from epoch (January 1, 1970 00:00:00 GMT) of the start time of the event interval
  • intervalDurationInSeconds - how long the event interval lasts in seconds
  • signalName - valid values are defined by the OpenADR protocol and listed in the Signal Names and types section
  • signalType - valid values are defined by the OpenADR protocol and listed in the Signal Names and types section
  • payload - value or quantity of interval, use the signal name and signal type to interpret the meaning of the payload
  • status - status of a new event, valid values are far, near, active, completed, and cancelled.
  • pluginApiVersion - version number of the INotifierPlugin API used to build the plugin, formatted as MAJOR_VERSION.MINOR_VERSION, where MAJOR_VERSION and MINOR_VERSION are integers.
  • pluginVersion - version number of the NotifierPlugin implementation. This number changes independently of the pluginApiVersion at the discretion of the plugin vendor.
  • VenId - ID of the VEN that the request is coming through, for reference if all requests are ultimately aggregated in the customer system.
  • VtnId - ID of the VTN that the request is coming from.
  • instanceId - ID of nova instance, as entered in the plugin config file.
  • targets - the targeted components for the event
  • venId - ID of the VENs to be targeted with the event
  • partyId - ID assigned to a VEN for a given market context
  • groupIds - list of IDs for targeted groups (group IDs and groupings are agreed upon out of band with the VTN provider)
  • resourceIds - list of ids for resources controlled by the VEN (resource ID mappings to local resources are determined out of band)

Creating a plugin

It is possible to create a plugin from scratch. Documentation for this is not yet complete.

A pointer to an IVENManager is passed to the plugin's initialize function. This object shouldn't be needed under most circumstances as messages sent through the http API (see the HTTP API section) are forwarded to the VENManager. This object is offered as a convenience in the event that NOVA lacks functionality required for integration.

FUNCTIONS ON THE IVENManager OJBECT SHOULD ONLY BE CALLED WHEN INITIATED FROM THE HTTP API. THEY SHOULD NOT BE CALLED FROM ANY OF THE CALLBACKS.

The function OnRegisteredMessage in the sample plugin demonstrates calling the forEach event function on IVENManager.

Error Handling Tips

Plaid executes as a series of jobs which follow two simple rules:

  1. Only one job may execute at a time
  2. If a job fails, retry it at some interval

Each job is a series of steps. If a failure is detected in any step, the entire job will be retried. Failures mainly occur at the boundaries: when communicating to external systems. There are two external systems involved: the VTN and the customer system. Jobs that fail when communicating to the customer system will be retried at whatever retry interval is set in the config file. Jobs that fail when communicating to the VTN use an exponential back-off algorithm to determine when the next retry will occur.

Retries in a micro-service are an essential component of proper operation. Even when everything is up and running as expected, unexpected failures can occur at the network layer. Rebooting systems and services can also cause issues. Failures which occur from these and other circumstances are mitigated with retries.

Retries are most useful for coping with short outages. When long outages occur, the best strategy may be to restart Plaid. It is always safe to restart.

The following sections do not cover the intention of the plugin callbacks. See the header file INovaNotifierPlugin.h for an up-to-date description of the callbacks. Instead, the following sections provide tips for handling error conditions.

Idempotent Functions

Any function that can be retried must be designed to be idempotent. A function is idempotent if it can be executed multiple times with the same information and the system is left in a correct state.

Let's take a customer order system as an example. If the same exact order is charged to a customer account multiple times (e.g. as defined by a unique orderID), the customer should only be charged once.

When failures occur, Plaid may call callback functions with the same information. The callback function and any functions called on the external system must be idempotent.

Error Handling in the Plugin

Plaid automatically handles retries to the VTN and may also handle retries to the external system. This section discusses when and how this retry mechanism can be used.

The custom plugin communicates to the customer system through a library which implements some protocol such as http, message queues, database, etc. When a failure occurs, the client code detects the failure in 1 of 3 ways:

  1. An error code returned from a function call
  2. An exception thrown from a function call
  3. A negative response in the message from the external system

Assuming the micro-service code and API code on the external system are correct, all 3 errors are an indication that the system is temporarily down and the message should be retried at some point in the future. The 3rd error type might also indicate an error in the micro-service or the external system so it's good practice to log the 3rd error type and send a notification.

Error handling in the plugin can be done in two ways:

  1. Catch the errors and handle retries in the plugin. The sample plugin does this. This strategy works OK for event handling, but does not work for report handling.
  2. Don't catch exceptions and let Plaid do the retries. If an error is detected, throw a std::runtime_error("With a useful message") and let Plaid retry the job.

Plaid detects failures through C++ exceptions. As long as the exception can be caught with std::exception, Plaid will catch the error and retry the job at whatever interval is specified in the config. When an exception occurs, Plaid calls the OnException callback with details of the job that failed and the exception message.

You are not required to choose one strategy over the other. Choose whichever strategy works best for the given callback.

Some jobs communicate to both the VTN and to the customer system. In these instances, failures in the customer system affect what data is sent to the VTN. The following discussions on Event and Report handling in the plugin offer options for handling failures specific to those callbacks.

These discussions assume some familiarity of the callbacks. Before digging into the details below, it will help to read through and start implementing the callbacks.

Events

The plugin callbacks OnDistributeEventStart, OnDistributeEventComplete, OnEvent, and OnEventCancel all occur while Plaid is collecting data that will be sent to the VTN. These callbacks allow the plugin to opt in or out of events and that information is relayed back to the VTN. If an error is thrown from one of these functions, Plaid won't have the information needed to respond to the VTN and will delay sending the message to the VTN.

Here are some options for handling errors in these functions:

  1. Catch errors and default the opt status: The default opt status should probably be optOut to the let the VTN know the VEN is not available for control. Defaulting to optIn is also an option. Under this strategy, the external system would presumably not have event information stored in the customer system so a strategy is needed to sync the event information when the external system comes back online. Options to sync event information include restarting Plaid and writing a custom HTTP API function to retrieve the information. If the opt status of an event needs to be changed, the external system will also need to make calls to the http API function Created Event and/or Opt Event. If Plaid is restarted, the opt status will automatically be updated since Plaid will retrieve the list of events and redo the event callbacks.
  2. Throw errors: This is the easier strategy to implement. Plaid will keep retrying event processing and when the job completes successfully, Plaid and the external system will be in sync so there's no need for a strategy to resync event information once the external system is available again.

The other OnEvent* callbacks occur between Plaid and the plugin only - the VTN does not receive any communication from these callbacks. Since the VTN isn't involved, the plugin can handle retries (as the sample plugin does). The plugin can also detect problems and throw errors. Throwing errors is the easier strategy.

Reports

The callbacks OnPeriodicReportStart and OnPeriodicReportComplete do not properly handle exceptions. If an error occurs while processing these callbacks, Plaid must be restarted to correct the error. Failures in this callback should be extremely rare. We are working on a fix.

The OnGeneratePeriodicReport callback does properly handle exceptions. The plugin should track what the last upload date was for each reportRequestID and use that date to "widen" the report window when an error occurs. The sample plugin tracks this information in the QueryIntervals object. The only option for handling retries with this callback is to throw errors: retries cannot be handled within the plugin.

Registration

The registration process is a single job but it hits many callbacks for events and reports. If a failure occurs anywhere in the process, the entire job is restarted.

Implementing for Multiple Utilities

Each running instance of Plaid connects to a single VTN. Therefore, to work with multiple utilities, each utility require it's own instance of Plaid.

The Plaid plugin should be designed to work with any utility though some options in the config file will change (such as vtn url) for each utility. This is true whether Plaid is running in the cloud or on device. As much as possible, control logic should be implemented in the external system with pluggable control logic.

When running multiple instances of Plaid on the same machine, each instance will need it's own log file and listening port (if the HTTP API is enabled).

Plaid can be executed from the same location in two ways:

  1. Use different config files for each instance
  2. Use the same config file with different command line parameters to override the parameters that differ.