Hello everyone,
In this thread, I want to propose a new LIP for the roadmap objective “Introduce universal serialization method”. This proposal defines how the generic serialization algorithm will be applied to accounts and specify the appropriate JSON schema.
Looking forward to your feedback.
Here is the complete LIP draft:
LIP: <LIP number>
Title: Define schema and use generic serialization for account state
Author: Alessandro Ricottone <alessandro.ricottone@lightcurve.io>
Type: Standards Track
Created: <YYYY-MM-DD>
Updated: <YYYY-MM-DD>
Requires: 00xx "A generic, deterministic and size efficient serialization method"
Abstract
This LIP defines how the generic serialization algorithm will be applied to accounts by specifying the appropriate JSON schema. We specify how the different address format introduced in LIP 0018 will be handled, to ensure that all of the account states are consistent across all Lisk nodes.
Copyright
This LIP is licensed under the Creative Commons Zero 1.0 Universal.
Motivation
As part of the “Introduce decentralized re-genesis” roadmap objective, all accounts will be serialized and inserted in a Merkle tree to calculate the Merkle root that will be included in the Regenesis Transaction. For this reason, it is fundamental to uniquely specify how accounts are serialized in order to achieve a consistent state across all Lisk nodes.
Furthermore, having a standard way of serializing the accounts state is beneficial in other parts of the Lisk protocol:
- A standard account serialization system can help improve storage efficiency.
- Developers using the Lisk-SDK will benefit from a flexible and expandable serialization method.
LIP 00xx “A generic, deterministic and size efficient serialization method” defines the general serialization method and how JSON schemas are used to serialize data in the Lisk protocol.
In this LIP, we specify the JSON schema used to serialize an account.
Rationale
We define a root schema account to serialize accounts. This schema contains all properties related to transaction validation that are relevant regardless of the transaction type. Properties that are specific to a certain transaction type are instead stored in the account asset. The account asset is serialized according to the asset schema, here defined for the Lisk mainchain.
Handling Legacy Addresses
After LIP 0018 “Use base32 encoding of long hash of public key plus checksum for address” is implemented, the address format in the Lisk protocol will change. In particular, the current 8 bytes addresses will be replaced by 20 bytes addresses. Accounts that have issued no transactions at the time of the address conversion will have no public key set, and as such the two address formats can not be linked (and the old format can not be updated). In this case, the address property will contain an address in the old format and users will have to move their balance from an old to a new account by issuing a Reclaim Transaction from the new account. If the public key is present instead, the address property will be updated to the new format.
Specification
A schematic of the account-serialization structure. Objects are indicated in bold and their properties are included in a box. Arrays are indicated with the items type followed by square brackets, and the attached boxes represent the schema of their items.
account Schema
Accounts are serialized and deserialized according to the account schema. The account schema contains the following properties:
-
address: The account address. This can either be: -
balance: The account balance in Beddows. -
publicKey: The account public key that generated theaddress. Notice that this property has a non-empty value if and only if the account issued at least one transaction included in the blockchain. This value is either 0 bytes or 32 bytes long. -
keys: Accounts that issued a Multisignature Transaction store the transaction-signing rules in this property. -
nonce: The current nonce, representing the total number of transactions sent from the account. -
asset: The account asset stores extra information related to transaction assets. In this LIP we specify the asset schema for the Lisk mainchain.
account = {
"type": "object",
"properties": {
"address": {
"dataType": "bytes",
"fieldNumber": 1
},
"balance": {
"dataType": "uint64",
"fieldNumber": 2
},
"publicKey": {
"dataType": "bytes",
"fieldNumber": 3
},
"nonce": {
"dataType": "uint64",
"fieldNumber": 4
},
"keys": {
...keysObject,
"fieldNumber": 5
},
"asset": {
...accountAsset,
"fieldNumber": 6
}
},
"required": [
"address",
"balance",
"publicKey",
"nonce",
"keys",
"asset"
]
}
Here, the ... notation, borrowed from JavaScript ES6 data destructuring, indicates that the corresponding schema should be inserted in place, and it is just used for notational convenience.
keysObject Schema
This property contains the public keys necessary to sign a transaction from the account as defined by LIP 0017.
The schema keysObject contains 3 properties:
-
numberOfSignatures: The number of private keys that must sign a transaction. This value is greater than 0 only if the account issued a Multisignature Registration Transaction. -
mandatoryKeys: An array of public keys in lexicographical order. The corresponding private keys necessarily have to sign the transaction. A valid public key is 32 bytes long. -
optionalKeys: An array of public keys in lexicographical order. The number of corresponding private keys that have to sign the transaction equalsnumberOfSignaturesminus the size ofmandatoryKeys. A valid public key is 32 bytes long.
keysObject = {
"type": "object",
"properties": {
"numberOfSignatures": {
"dataType": "uint32",
"fieldNumber": 1
},
"mandatoryKeys": {
"type": "array",
"items": {
"dataType": "bytes"
},
"fieldNumber": 2
},
"optionalKeys": {
"type": "array",
"items": {
"dataType": "bytes"
},
"fieldNumber": 3
}
},
"required": [
"numberOfSignatures",
"mandatoryKeys",
"optionalKeys"
]
}
accountAsset Schema for the Lisk Mainchain
The asset property stores all information relevant to the protocol of the blockchain the accounts belong to. SDK developers can include custom properties of their sidechains by modifying this schema.
The accountAsset schema for the Lisk mainchain contains the following properties:
-
delegate: This property contains information about forging functionalities of an account registered as a delegate. -
votes: An array storing all votes cast by the account using a Vote Transaction, ordered lexicographically bydelegateAddress. -
unlocking: An array storing all votes withdrawn by the account using a Vote Transaction and the corresponding block height at which the transaction was included in the blockchain, ordered lexicographically by(delegateAddress, unvoteHeight, amount).
accountAsset = {
"type": "object",
"properties": {
"delegate": {
...delegateObject,
"fieldNumber": 1
},
"votes": {
"type": "array",
"items": {
...vote
},
"fieldNumber": 2
},
"unlocking": {
"type": "array",
"items": {
...unlockingObject
},
"fieldNumber": 3
}
},
"required": [
"delegate",
"votes",
"unlocking"
]
}
delegateObject Schema
The delegateObject schema contains the following properties:
-
username: A string representing the delegate username, with a minimum length of 1 character and a maximum length of 20 characters. -
pomHeights: An array containing the heights (in ascending order) at which a Proof-of-Misbehaviour Transaction regarding the delegate has been included in the blockchain. -
consecutiveMissedBlocks: Number of consecutive blocks missed by the delegate, used to determine if the delegate has to be banned. -
lastForgedHeight: The height of the last block forged by the delegate, used to determine if the delegate has to be banned. -
isBanned: A boolean, true if the delegate has been banned. -
totalVotesReceived: Total amount of votes for the delegate. This value is used to calculate the delegate weight.
delegateObject = {
"type": "object",
"properties": {
"username": {
"dataType": "string",
"fieldNumber": 1
},
"pomHeights": {
"type": "array",
"items": {
"dataType": "uint32"
},
"fieldNumber": 2
},
"consecutiveMissedBlocks": {
"dataType": "uint32",
"fieldNumber": 3
},
"lastForgedHeight": {
"dataType": "uint32",
"fieldNumber": 4
},
"isBanned": {
"dataType": "boolean",
"fieldNumber": 5
},
"totalVotesReceived": {
"dataType": "uint64",
"fieldNumber": 6
},
},
"required": [
"username",
"pomHeights",
"consecutiveMissedBlocks",
"lastForgedHeight",
"isBanned",
"totalVotesReceived"
]
}
vote Schema
The vote schema contains the following properties:
-
delegateAddress: The address of the voted delegate. Notice that this address is always in the new format, and therefore it is 20 bytes long. -
amount: The amount of Beddows voted for the delegate
vote = {
"type": "object",
"properties": {
"delegateAddress": {
"dataType": "bytes",
"fieldNumber": 1
},
"amount": {
"dataType": "uint64",
"fieldNumber": 2
}
},
"required": [
"delegateAddress",
"amount"
]
}
unlockingObject Schema
The unlockingObject schema contains the following properties:
-
delegateAddress: The address of the unvoted delegate. Notice that this address is always in the new format, and therefore it is 20 bytes long. -
amount: The amount of Beddows withdrawn from the vote amount. -
unvoteHeight: The height at which the transaction to unvote the delegate has been included in the blockchain.
unlockingObject = {
"type": "object",
"properties": {
"delegateAddress": {
"dataType": "bytes",
"fieldNumber": 1
},
"amount": {
"dataType": "uint64",
"fieldNumber": 2
},
"unvoteHeight": {
"dataType": "uint32",
"fieldNumber": 3
}
},
"required": [
"delegateAddress",
"amount",
"unvoteHeight"
]
}
Backwards Compatibility
This proposal does not introduce any forks in the network, as it only defines the JSON schema used to serialize an account in the Lisk protocol.
Appendix: Serialization Example
accountData = {
"address": e11a11364738225813f86ea85214400e5db08d6e,
"balance": 10n,
"publicKey": 0fd3c50a6d3bd17ea806c0566cf6cf10f6e3697d9bda1820b00cb14746bcccef,
"nonce": 5n,
"keys": {
"numberOfSignatures": 2,
"mandatoryKeys":
[ c8b8fbe474a2b63ccb9744a409569b0a465ee1803f80435aec1c5e7fc2d4ee18,
6115424fec0ce9c3bac5a81b5c782827d1f956fb95f1ccfa36c566d04e4d7267 ],
"optionalKeys": []
},
"asset": {
"delegate": {
"username": "Catullo",
"pomHeights": [ 85 ],
"consecutiveMissedBlocks": 32,
"lastForgedHeight": 64,
"isBanned": false,
"totalVotesReceived": 300000000n
},
"votes":
[ { "delegateAddress":
cd32c73e9851c7137980063b8af64aa5a31651f8dcad258b682d2ddf091029e4,
"amount": 100000000n },
{ "delegateAddress":
9d86ad24a3f030e5522b6598115bb4d70c1692c9d8995ddfccb377379a2d86c6,
"amount": 250000000n } ],
"unlocking":
[ { "delegateAddress":
655e665765e3c42712d9a425b5b720d10457a5e45de0d4420e7c53ad73b02ef5,
"amount": 400000000n,
"unvoteHeight": 128 } ]
}
}
accountMsg = {
0a14: e11a11364738225813f86ea85214400e5db08d6e,
10: 0a,
1a20: 0fd3c50a6d3bd17ea806c0566cf6cf10f6e3697d9bda1820b00cb14746bcccef,
20: 05,
2a46: {
08: 02,
(12):
[ 1220 c8b8fbe474a2b63ccb9744a409569b0a465ee1803f80435aec1c5e7fc2d4ee18,
1220 6115424fec0ce9c3bac5a81b5c782827d1f956fb95f1ccfa36c566d04e4d7267 ],
optionalKeys: []
},
329901: {
0a18: {
0a07: 436174756c6c6f,
(1201): 1201 55,
18: 20,
20: 40,
28: 00,
30: 80c6868f01
},
(12):
[ { 1227 0a20:
cd32c73e9851c7137980063b8af64aa5a31651f8dcad258b682d2ddf091029e4,
10: 80c2d72f },
{1227 0a20:
9d86ad24a3f030e5522b6598115bb4d70c1692c9d8995ddfccb377379a2d86c6,
10: 80e59a77 } ],
(1a):
[ { 1a2b 0a20:
655e665765e3c42712d9a425b5b720d10457a5e45de0d4420e7c53ad73b02ef5,
10: 8088debe01,
18: 8001 } ]
}
}
binaryMsg [307 bytes] = 0a14e11a11364738225813f86ea85214400e5db08d6e100a1a200fd3c50a6d3bd17ea806c0566cf6cf10f6e3697d9bda1820b00cb14746bcccef20052a4608021220c8b8fbe474a2b63ccb9744a409569b0a465ee1803f80435aec1c5e7fc2d4ee1812206115424fec0ce9c3bac5a81b5c782827d1f956fb95f1ccfa36c566d04e4d72673299010a180a07436174756c6c6f1201551820204028003080c6868f0112270a20cd32c73e9851c7137980063b8af64aa5a31651f8dcad258b682d2ddf091029e41080c2d72f12270a209d86ad24a3f030e5522b6598115bb4d70c1692c9d8995ddfccb377379a2d86c61080e59a771a2b0a20655e665765e3c42712d9a425b5b720d10457a5e45de0d4420e7c53ad73b02ef5108088debe01188001