Decentralized accounting

Accounting flow

The following is the general outline of the whole accounting procedure flow. This protocol contains both on-chain and off-chain steps and is inspired by the state channels.

1. An agreement between a Publisher and a Subscriber is made and registered on Chain as a service. The service requires a certain amount of tokens locked by Subscriber (referred to as the credit) for the stream delivery payments. The credit is consumed when a particular amount of data is delivered to one of the Clients authorized by that Subscriber.
2. Clients connect to Brokers and request a delivery of data. The layer of Brokers is transparent to Clients, which means that any Broker Node is able to deliver requested streams to the Clients.
3. Brokers verify the request and check if a proper agreement is made and credit is available. When both checks pass, then Brokers start delivering data to the Client. If the Publisher is not yet delivering data to the network then the Broker requests that stream from the Publisher by sending a StartRequest message. Brokers are responsible for monitoring the consumption of the Subscriber’s credit.
4. Clients must confirm reception of data by sending a Proof of Consumption (PoC) to the Broker. When a Subscriber does not send or Broker does not receive a PoC, then Broker should terminate the connection with such Client and stop delivering data to it.
5. By connecting to the public accounting data stream, Brokers verify correctness of the PoC. This accounting data stream is called state channel and is intended for the cross-Broker synchronization, so that all Broker nodes that are serving the same subject for the same Client have access to the same accounting metadata. If the PoC is incorrect, then no more data must be sent to the Client. If the Proof is correct, then it must be retained by the Broker with the purpose of building Proof of Delivery.
6. Brokers construct a Proof of Delivery (PoD) out of the set of Proofs of Consumption, and submit it on-chain.
7. Chain randomly assigns a set of Observers to validate this PoD. Observers are monitoring the chain, hence they know when they are being assigned.
8. The assigned Observers obtain accounting data required for checking the PoD by connecting to the public off-chain data stream (state channel).
9. Observers verify correctness of the Proof of Delivery by reconstructing it. They check if the PoD matches the one submitted on-chain.
10. If PoDs match, Observers attest via on-chain message, if not, Observers raise an on-chain challenge.
11. When a required majority of Observers attest to PoD and no challenges are raised, then the PoD is valid. Chain executes payments. If challenges are raised, they need to be resolved, then the chain penalizes accordingly, either a Broker for submitting a faulty PoD, or an Observer for submitting a faulty challenge.

Figure: Decentralized accounting flows

Accounting header

After receiving the first StartRequest, a Publisher starts sending a stream of data to the Broker network. The stream is effectively a set of messages.

The message-blob in the following context consists of multiple data messages batched together. Each message-blob is prefixed with a header that identifies the Publisher and adds contextual information like the global counter of the message-blobs in the stream and the global length of the message stream. The header must be signed by the Publisher.

Header {
   ServiceId <identifier>,
   StreamCounter <number>,
   StreamLength <number>,
   MessageHash <hash>,
}

The accounting headers are published in the off-chain data stream, separate from the primary data stream which is being accounted for.

Proof of Consumption

The Proof of Consumption is generated by the Client (or more likely the code written by the Subscriber) as an acknowledgement of reception of a message delivered by a Broker (or a set of Brokers) sent by Publisher. The proof achieves binding properties between ClientId, SubscriberId, BrokerId(s), ServiceId, and received message properties (length, hash, counter).

Proof_Of_Consumption {
   Header <Signed[Header||Broker_Signatures]>
   Consumption_Token {
      SubscriberId,
      ClientId,
      ServiceId,
      Value,
      Nonce,
   }<Signed_by_SubscriberId>
}<Signed_by_ClientId>

Proof of Delivery

The Proof of Delivery (PoD) is generated by a Broker as an attestation of the work the Broker has  done. PoD proves that a particular and continuous set of messages was delivered by the Broker (or set of Brokers) to a particular Client.

Construction

The construction of the Proof of Delivery is straightforward. It is built with the following elements:

• First: a signed Proof of Consumption related with delivery of a particular stream to a particular Client, which is the first in the stream.
• Last: a signed Proof of Consumption related with delivery of a particular stream to a particular Client, which is the last in the stream.
• Proof: a Merkle Tree Proof based on a set of Proof of Consumptions associated with a particular stream and particular Client.
• Signature: a signature authenticating the sender of the Proof of Delivery.

Proof_Of_Delivery {
   First <Signed Proof_of_Consumption>,
   Last <Signed Proof_of_Consumption>,
   Proof <Merkle Tree Proof>,
   Signature <Broker Signature>
}

Generation

In order to generate a correct PoD, a Broker must take a continuous set of PoCs, which means that each Proof of Consumption must increment the Header.StreamCounter by one. The set of PoCs must be coherent, which means that it must hold a binding property for all proofs included in the set.

Verification

The verification of the PoD is successful if the signatures are correct and included PoCs are consistent with the service registered on-chain.

Challenging

A Proof of Delivery can be challenged by Observers, who are responsible for monitoring an agreement dedicated subject where all Proofs of Consumption are posted, collecting them, and validating Proofs of Delivery submitted on chain.

Invalid PoD means that the proof is not based on a continuous set of Proofs of Consumption. If an invalid PoD is submitted on-chain, then the Observer can challenge it, asking for providing a Merkle Tree Proof of Inclusion of a missing Proof of Consumption.

Then the on-chain verification will indicate that the Merkle proof was badly constructed and the submitter of the falsified Proof of Delivery is slashed and the Observer who submitted the Challenge is rewarded. Additionally, the credit of all Observers who approved of that Proof are slashed.

Attestation

Challenging is assumed to be a relatively rare event, since the discouraging penalization is in place.

The main incentivisation mechanism for Observers is a share of Network Fee, earned via signing the submitted PoDs in order to make these proofs eligible for the final accounting round that results in reward payments.

The same mechanism also encourages Brokers to make all collected Proofs of Consumption available, rendering the potential Brokers’ attempts to block Observers from reconstructing PoDs futile: not enough attestations means no reward.

Processing

The Proof of Delivery is processed by an on-chain module, which executes payments after a successful verification and after a challenge delay (without any successful challenge). The payment is calculated based on the difference between the message descriptors of the Header of the Last and the First Proofs of Consumption.

The payment is calculated as follows:

PublisherPayment = (Last.Header.StreamLength - First.Header.StreamLength + 1) * Last.Header.Service.ServiceFeeRate

NetworkReward = (Last.Header.StreamLength - First.Header.StreamLength + 1) * NetworkFee

BrokersReward = NetworkReward * BrokerObserverSplitRatio

ObserversReward = NetworkReward * (1 - BrokerObserverSplitRatio)

Here, both BrokersReward and ObserversReward are referring to a reward paid for all Brokers and Observers, which took part in the Proof of Delivery submission. Which means that the exact amount of the reward to each party will be a portion of that sum (see calculation example here).