Replace static fee system by dynamic fee system

Dear Lisk community,

With this email I want to kickstart the discussion and gather your feedback for a LIP regarding the roadmap objective "Replace static fees with dynamic fee system”. You can find a short summary of the LIP and the full proposal in the attachment below.

Abstract

This LIP proposes a dynamic fee system as a free fee market where the fixed fee for all transaction types will be removed and a minimum fee will be defined for every transaction. Any transactions received with a fee below that minimum fee will be rejected. For each transaction, it will be up to the issuer to choose an appropriate transaction fee depending on the network load, which has to be at least the minimum transaction fee.

Note that the values given in this proposal for the three fee constants may vary before the proposal is finalised depending on the feedback gathered on this mail thread.

lip-replace_static_fee_system_by_dynamic_fee_system.md (38.7 KB)

···

I like the idea of dynamic fees very much and I think the proposed system is dynamic in the sense of that it can adapt to different network situations/congestion. But I see with it the same issue as we have with the current system: The final cost of a transaction in fiat (for example USD) is in a fix relation to the price of LSK and not dynamic at all. This means that when the LSK price increases in the future (let's say to $100 per LSK, to be on the conservative side), transactions will cost 100 times more than they do now, also with the proposed dynamic fees system. We then will be in the exact same situation as we are now with our current fee system. We will need an additional hard fork to lower the fees again. We should try to avoid that and make the fee system not only dynamic in relation to the network situation but also in relation to the LSK price.

The issue with the proposed system is that minFeePerByte is set to a more or less arbitrary value. The resulting minimum fee for the transactions might be good in the current situation, with current LSK prices. Spam might be prevented in the current situation. But as soon as the price of LSK goes up or down, transactions will cost (much) more or less (measured in fiat), which is not desirable, in my opinion. Even if it still costs very low fees, why should the same transaction cost 100 times more in the future?

How to fix that issue? My proposal is to make minFeePerByte adjustable every round, instead of using a fixed value. This would allow to adapt minFeePerByte to different market situations to keep transactions at a similar and reasonable price level.

But how to adapt minFeePerByte? In a decentralized system you can't peg such a variable directly to values which are offered by centralized third parties. This means we can't just peg minFeePerByte to the LSK price. Where to get that LSK price? Is it correct? What if that system is down or manipulated? We need a decentralized way to do this.

My proposal how to make the fee system truly dynamic, also in relation to fiat values looks as follows:
minFeePerByte should be adjustable and not just a more or less arbitrary defined value. It will be set in a decentralized way by the forging delegates, using a kind of a voting mechanism. The exact functionality of this voting mechanism is still to be defined, I would suggest something similar to the following:
We add a new transaction type, called 'feeVote', which is valid only from an active delegate account. This transaction contains the value of minFeePerByte the sending delegate would like to have from now on. When the market situation changes, delegates can transmit such feeVote transactions and propose with them a new value for minFeePerByte. Every round, the system takes the newest 57 feeVote transactions (only the newest one if a delegate sent multiple), removes the highest 3 and the lowest 3 (because they might be outliers), calculates the average of the resulting 51 feeVote transactions and takes this value as the new minFeePerByte. This is basically a floating moving average of the last 51 votes.

How to prevent that delegates set artificially high fee values to earn more from fees?

• top and bottom 3 values are excluded from calculation, that prevents that a few delegates can push the calculated fee up or down.
• we could say that only feeVote values are valid that are within 20% of the last value. This would allow only small changes, no big jumps.
• we could even penalize delegates who send too high/low feeVote transactions.
• delegates have an economic interest that Lisk is perceived as a top blockchain, which increases the value of LSK. Delegates won't hurt Lisk deliberately. This lowers the incentive to manipulate fee prices.
• delegates earn much more from block rewards than from fees, even if fees would be very high. This lowers the incentive to set artificially high fees.
• finally, the proposed feeVote transactions from the delegates are public, this means that voters can unvote delegates who propose fake fee values. This increases the incentive to act honestly.

TLDR: delegates vote in a decentralized way on the value of minFeePerByte. This allows the system to adapt to changing market situations (increasing/decreasing LSK price) in a truly decentralized dpos-style way, to keep fees at reasonable levels.

cc001

Dear cc001,

Thank you very much for your response and feedback. I will try to answer it going from the general idea and implications for the proposed minFeePerByte constant to finish with comments specific to your proposal.

First, let’s put the numbers of this proposal into perspective: Let’s take the current market situation (the price of 1LSK is approx 1.25$) and assume the LIP is implemented with the constants defined in it. This means that, for the two most common transaction types, the minimum price for a transaction is:

  • Balance transfer minimum fee = 0.00125LSK = 0.0015$
  • Vote minimum fee = 0.02262LSK = 0.02825$

As you can see, the LSK token price would need to be more than 100x higher for the minimum fee per byte defined in the current proposal to be a concern (which is more or less the case you considered in your reply).

I totally see that in the future, when the Blockchain Application Platform is completely enabled and a full ecosystem of sidechains is running, several protocol constants will be adjusted. That is why there is a note in this proposal commenting this possibility. But it is not going to be only a question of decreasing minFeePerByte but other critical constants that for now we want to keep comprehensively modest. I expect that with the incoming protocol improvements (block transmission and verification efficiency, dencetralized re-genesis, etc) we will be able to introduce more ambitious constants. For example, the block size limit (that we proposed to set to 15 KB in https://github.com/LiskHQ/lips/blob/master/proposals/lip-0002.md) is an important parameter to consider in order to have a scalable and trust-less bond/communication between the Lisk mainchain and sidechains. However, I do not see the fact that these changes will introduce a hard-fork as such a big problem, at least not before Blockchain Application Platform is completely released. Bear in mind that the incoming roadmap milestones will imply hard-forks by themselves and they may also implement constants changes if there is a necessity.

Summarizing, I would say that the proposed value for the minimum fee per byte constant (minFeePerByte= 10^-5LSK) is an upper bound and it may/should lower down with the maturity and growth (in usage) of the network.

Let’s now focus on the proposal of your reply:

Definitely the idea of having a dynamic minFeePerByte value is interesting and in general desirable. I think it is complex enough to deserve a separate proposal with its complete research (define a new transaction type, define the voting process for the new value of minFeePerByte, etc), and if accepted, it could be implemented together with this proposal or for a later milestone in the roadmap (it only impacts minFeePerByte concept so they would not conflict). However, before any of that, I would like to get some ideas clear from your proposal:

In your response, it seems you did not consider the fact that in this LIP, the proposal is to burn the minimum part of the fee in every transaction (trs.minFee in the document). This means that delegates only get fees for transactions paying a fee over trs.minFee. If minFeePerByte is adjustable, in general, delegates would not have much incentive in changing its value, whatever it is: If blocks are not full, users can get their transaction confirmed without problem paying the minimum fee. Then delegates do not get any transaction fees so they do not care about minFeePerByte value. If the network is actively used (blocks are usually full), users are going to pay fees over the trs.minFee for their transaction to get it confirmed (faster). But the extra fee they pay (to the delegates) is not going to be (directly) correlated to minFeePerByte. It is going to be mainly correlated to the block space available and block space demand (see Bitcoin case where in theory their minFeePerByte = 0). By the way, I said “mainly correlated” because one can argue that delegates are incentivised to set a high minFeePerByte since it implies less inflation and hence a small profit for every token holder. But that also implies lowering down the demand for block space from the users (less transactions).

So I can conclude that in general, if minFeePerByte is adjustable, delegates will offer lowest possible minimum fees in exchange of votes from the community (they do not have any special incentive to do the opposite), something that can harm the network if it is not mature enough to cope with the load.

Also, you did not mention anything about delegateFee and dappFee. Would you propose the same idea for these two constants too? I suspect it may be problematic to let the delegates to agree on what (minimum) fee one needs to pay to register as a delegate. In the same way, dappFee parameter will in the future define the first entry level to start with a sidechain project so it may make sense to have this one set to a value to increase adoption and accessibility to the Lisk environment for Dapp developers.

I am looking forward to your reply, specially if you think I made a wrong assumption from your ideas.

Best,
Iker

···

Message: 1
Date: Wed, 13 Feb 2019 21:34:06 +0100
From: cc001 cc001.bct@gmail.com
To: lips@lists.lisk.io
Subject: [LIPS] Replace static fee system by dynamic fee system
Message-ID: 7f42eb69-5ddc-7be3-1be5-e0e280785e51@gmail.com
Content-Type: text/plain; charset=utf-8; format=flowed

I like the idea of dynamic fees very much and I think the proposed
system is dynamic in the sense of that it can adapt to different network
situations/congestion. But I see with it the same issue as we have with
the current system: The final cost of a transaction in fiat (for example
USD) is in a fix relation to the price of LSK and not dynamic at all.
This means that when the LSK price increases in the future (let’s say to
$100 per LSK, to be on the conservative side), transactions will cost
100 times more than they do now, also with the proposed dynamic fees
system. We then will be in the exact same situation as we are now with
our current fee system. We will need an additional hard fork to lower
the fees again. We should try to avoid that and make the fee system not
only dynamic in relation to the network situation but also in relation
to the LSK price.

The issue with the proposed system is that minFeePerByte is set to a
more or less arbitrary value. The resulting minimum fee for the
transactions might be good in the current situation, with current LSK
prices. Spam might be prevented in the current situation. But as soon as
the price of LSK goes up or down, transactions will cost (much) more or
less (measured in fiat), which is not desirable, in my opinion. Even if
it still costs very low fees, why should the same transaction cost 100
times more in the future?

How to fix that issue? My proposal is to make minFeePerByte adjustable
every round, instead of using a fixed value. This would allow to adapt
minFeePerByte to different market situations to keep transactions at a
similar and reasonable price level.

But how to adapt minFeePerByte? In a decentralized system you can’t peg
such a variable directly to values which are offered by centralized
third parties. This means we can’t just peg minFeePerByte to the LSK
price. Where to get that LSK price? Is it correct? What if that system
is down or manipulated? We need a decentralized way to do this.

My proposal how to make the fee system truly dynamic, also in relation
to fiat values looks as follows:
minFeePerByte should be adjustable and not just a more or less arbitrary
defined value. It will be set in a decentralized way by the forging
delegates, using a kind of a voting mechanism. The exact functionality
of this voting mechanism is still to be defined, I would suggest
something similar to the following:
We add a new transaction type, called ‘feeVote’, which is valid only
from an active delegate account. This transaction contains the value of
minFeePerByte the sending delegate would like to have from now on. When
the market situation changes, delegates can transmit such feeVote
transactions and propose with them a new value for minFeePerByte. Every
round, the system takes the newest 57 feeVote transactions (only the
newest one if a delegate sent multiple), removes the highest 3 and the
lowest 3 (because they might be outliers), calculates the average of the
resulting 51 feeVote transactions and takes this value as the new
minFeePerByte. This is basically a floating moving average of the last
51 votes.

How to prevent that delegates set artificially high fee values to earn
more from fees?

? top and bottom 3 values are excluded from calculation, that prevents
that a few delegates can push the calculated fee up or down.
? we could say that only feeVote values are valid that are within 20% of
the last value. This would allow only small changes, no big jumps.
? we could even penalize delegates who send too high/low feeVote
transactions.
? delegates have an economic interest that Lisk is perceived as a top
blockchain, which increases the value of LSK. Delegates won’t hurt Lisk
deliberately. This lowers the incentive to manipulate fee prices.
? delegates earn much more from block rewards than from fees, even if
fees would be very high. This lowers the incentive to set artificially
high fees.
? finally, the proposed feeVote transactions from the delegates are
public, this means that voters can unvote delegates who propose fake fee
values. This increases the incentive to act honestly.

TLDR: delegates vote in a decentralized way on the value of
minFeePerByte. This allows the system to adapt to changing market
situations (increasing/decreasing LSK price) in a truly decentralized
dpos-style way, to keep fees at reasonable levels.

cc001

Hi Iker

Thanks for your response.

I agree that the new fees are pretty low and even a 100 times increase in relation to fiat money (or BTC) is still tolerable. And I also see the point that the current roadmap contains a lot of hardforks anyways where such adaptations of different constants can happen. It just bothers me that we get a complete new dynamic fee system, but we are still kind of static in relation to fiat costs of those transactions. On the other side, I think all (or most?) blockchains work this way, even bitcoins transactions fees only adapt to network status, but not to the price of the token.

You are right about the burning of fees. I saw in the proposal that fees are burnt, but I didn't realize that the minFee is burned and therefore different incentives apply to the whole system. I would have to think about that again.

Overall I like your proposed solution for a dynamic fee system. It still bothers me that it's kind of hard coded in relation to fiat money though. But I don't have a simple solution how to fix that. My suggestion of a decentralized voting mechanism seems to not work properly with this burning-part (which is needed to prevent the self-mining-own-transactions issue). I still think that such kind of voting mechanism is pretty elegant to change/decide on things in a decentralized way, it might be difficult to set working incentives though.

Regards

···

On 15.02.19 15:05, Iker Alustiza wrote:

Dear cc001,

Thank you very much for your response and feedback. I will try to answer it going from the general idea and implications for the proposed `minFeePerByte` constant to finish with comments specific to your proposal.

First, let’s put the numbers of this proposal into perspective: Let’s take the current market situation (the price of 1LSK is approx 1.25$) and assume the LIP is implemented with the constants defined in it. This means that, for the two most common transaction types, the minimum price for a transaction is:

  * Balance transfer minimum fee = 0.00125LSK = 0.0015$
  * Vote minimum fee = 0.02262LSK = 0.02825$

As you can see, the LSK token price would need to be more than 100x higher for the minimum fee per byte defined in the current proposal to be a concern (which is more or less the case you considered in your reply).

I totally see that in the future, when the Blockchain Application Platform is completely enabled and a full ecosystem of sidechains is running, several protocol constants will be adjusted. That is why there is a note in this proposal commenting this possibility. But it is not going to be only a question of decreasing `minFeePerByte` but other critical constants that for now we want to keep comprehensively modest. I expect that with the incoming protocol improvements (block transmission and verification efficiency, dencetralized re-genesis, etc) we will be able to introduce more ambitious constants. For example, the block size limit (that we proposed to set to 15 KB in https://github.com/LiskHQ/lips/blob/master/proposals/lip-0002.md) is an important parameter to consider in order to have a scalable and trust-less bond/communication between the Lisk mainchain and sidechains. However, I do not see the fact that these changes will introduce a hard-fork as such a big problem, at least not before Blockchain Application Platform is completely released. Bear in mind that the incoming roadmap milestones will imply hard-forks by themselves and they may also implement constants changes if there is a necessity.

Summarizing, I would say that the proposed value for the minimum fee per byte constant (`minFeePerByte`= 10^-5LSK) is an upper bound and it may/should lower down with the maturity and growth (in usage) of the network.

Let’s now focus on the proposal of your reply:
Definitely the idea of having a dynamic `minFeePerByte` value is interesting and in general desirable. I think it is complex enough to deserve a separate proposal with its complete research (define a new transaction type, define the voting process for the new value of `minFeePerByte`, etc), and if accepted, it could be implemented together with this proposal or for a later milestone in the roadmap (it only impacts `minFeePerByte` concept so they would not conflict). However, before any of that, I would like to get some ideas clear from your proposal:

In your response, it seems you did not consider the fact that in this LIP, the proposal is to burn the minimum part of the fee in every transaction (`trs.minFee` in the document). This means that delegates only get fees for transactions paying a fee over `trs.minFee`. If `minFeePerByte` is adjustable, in general, delegates would not have much incentive in changing its value, whatever it is: If blocks are not full, users can get their transaction confirmed without problem paying the minimum fee. Then delegates do not get any transaction fees so they do not care about `minFeePerByte` value. If the network is actively used (blocks are usually full), users are going to pay fees over the `trs.minFee` for their transaction to get it confirmed (faster). But the extra fee they pay (to the delegates) is not going to be (directly) correlated to `minFeePerByte`. It is going to be mainly correlated to the block space available and block space demand (see Bitcoin case where in theory their `minFeePerByte` = 0). By the way, I said “mainly correlated” because one can argue that delegates are incentivised to set a high `minFeePerByte` since it implies less inflation and hence a small profit for every token holder. But that also implies lowering down the demand for block space from the users (less transactions).

So I can conclude that in general, if `minFeePerByte` is adjustable, delegates will offer lowest possible minimum fees in exchange of votes from the community (they do not have any special incentive to do the opposite), something that can harm the network if it is not mature enough to cope with the load.

Also, you did not mention anything about `delegateFee` and `dappFee`. Would you propose the same idea for these two constants too? I suspect it may be problematic to let the delegates to agree on what (minimum) fee one needs to pay to register as a delegate. In the same way, `dappFee` parameter will in the future define the first entry level to start with a sidechain project so it may make sense to have this one set to a value to increase adoption and accessibility to the Lisk environment for Dapp developers.

I am looking forward to your reply, specially if you think I made a wrong assumption from your ideas.

Best,
Iker

----------------------------------------------------------------------

Message: 1
Date: Wed, 13 Feb 2019 21:34:06 +0100
From: cc001 <cc001.bct@gmail.com <mailto:cc001.bct@gmail.com>>
To: lips@lists.lisk.io <mailto:lips@lists.lisk.io>
Subject: [LIPS] Replace static fee system by dynamic fee system
Message-ID: <7f42eb69-5ddc-7be3-1be5-e0e280785e51@gmail.com <mailto:7f42eb69-5ddc-7be3-1be5-e0e280785e51@gmail.com>>
Content-Type: text/plain; charset=utf-8; format=flowed

I like the idea of dynamic fees very much and I think the proposed
system is dynamic in the sense of that it can adapt to different network
situations/congestion. But I see with it the same issue as we have with
the current system: The final cost of a transaction in fiat (for example
USD) is in a fix relation to the price of LSK and not dynamic at all.
This means that when the LSK price increases in the future (let's say to
$100 per LSK, to be on the conservative side), transactions will cost
100 times more than they do now, also with the proposed dynamic fees
system. We then will be in the exact same situation as we are now with
our current fee system. We will need an additional hard fork to lower
the fees again. We should try to avoid that and make the fee system not
only dynamic in relation to the network situation but also in relation
to the LSK price.

The issue with the proposed system is that minFeePerByte is set to a
more or less arbitrary value. The resulting minimum fee for the
transactions might be good in the current situation, with current LSK
prices. Spam might be prevented in the current situation. But as soon as
the price of LSK goes up or down, transactions will cost (much) more or
less (measured in fiat), which is not desirable, in my opinion. Even if
it still costs very low fees, why should the same transaction cost 100
times more in the future?

How to fix that issue? My proposal is to make minFeePerByte adjustable
every round, instead of using a fixed value. This would allow to adapt
minFeePerByte to different market situations to keep transactions at a
similar and reasonable price level.

But how to adapt minFeePerByte? In a decentralized system you can't peg
such a variable directly to values which are offered by centralized
third parties. This means we can't just peg minFeePerByte to the LSK
price. Where to get that LSK price? Is it correct? What if that system
is down or manipulated? We need a decentralized way to do this.

My proposal how to make the fee system truly dynamic, also in relation
to fiat values looks as follows:
minFeePerByte should be adjustable and not just a more or less arbitrary
defined value. It will be set in a decentralized way by the forging
delegates, using a kind of a voting mechanism. The exact functionality
of this voting mechanism is still to be defined, I would suggest
something similar to the following:
We add a new transaction type, called 'feeVote', which is valid only
from an active delegate account. This transaction contains the value of
minFeePerByte the sending delegate would like to have from now on. When
the market situation changes, delegates can transmit such feeVote
transactions and propose with them a new value for minFeePerByte. Every
round, the system takes the newest 57 feeVote transactions (only the
newest one if a delegate sent multiple), removes the highest 3 and the
lowest 3 (because they might be outliers), calculates the average of the
resulting 51 feeVote transactions and takes this value as the new
minFeePerByte. This is basically a floating moving average of the last
51 votes.

How to prevent that delegates set artificially high fee values to earn
more from fees?

? top and bottom 3 values are excluded from calculation, that prevents
that a few delegates can push the calculated fee up or down.
? we could say that only feeVote values are valid that are within 20% of
the last value. This would allow only small changes, no big jumps.
? we could even penalize delegates who send too high/low feeVote
transactions.
? delegates have an economic interest that Lisk is perceived as a top
blockchain, which increases the value of LSK. Delegates won't hurt Lisk
deliberately. This lowers the incentive to manipulate fee prices.
? delegates earn much more from block rewards than from fees, even if
fees would be very high. This lowers the incentive to set artificially
high fees.
? finally, the proposed feeVote transactions from the delegates are
public, this means that voters can unvote delegates who propose fake fee
values. This increases the incentive to act honestly.

TLDR: delegates vote in a decentralized way on the value of
minFeePerByte. This allows the system to adapt to changing market
situations (increasing/decreasing LSK price) in a truly decentralized
dpos-style way, to keep fees at reasonable levels.

cc001

Feature request: Implement max transaction fee during the “Network Economics” phase.

Proposal:

Looking at a recent event where a transaction was made on another blockchain with a fee larger than was intended by the user. And seeing how future dynamic fees in lisk will allow the user to set their own fee (above the minimum required). I recommend implementing a maximum tx fee of for instance 2 lisk for type 0 transactions. This could be part of the protocol “Replace static fees with dynamic fee system”.

Examples of fees gone wrong:

On Feb-19-2019 a transaction was made on the Ethereum blockchain with a fee of 2,100 Ether. Many times higher than required. This resulted in the loss of $308,679 for the user.

In June 2018, Ripple implemented a maximum transaction fee after a user mistakenly burned 100000 XRP in tx fees.

Solution:

Check if the fee for a type 0 transaction is <2 lsk, otherwise prevent the tx from being broadcasted.

Regards,

Jacco

Dear Jacco,

Thank you very much for the reply and the interest in my proposal.

My consideration for this issue is similar to what the GitHub user “stellardynamic” replied on https://github.com/LiskHQ/lisk/issues/2915. I strongly believe that this should not be a change at protocol level. It will add additional complexity for an edge case and it may hinder future use cases of the Lisk platform which may require a hard fork.

Hence, I think this is something that must be appropriately enforced locally in the user interfaces. As said in the mentioned GitHub link, Commander and Elements should include warnings for this situation and Lisk Hub and Lisk Mobile should make proper use of them. Therefore, I will take your feedback into consideration and add short remarks about this issue in the “Wallets” subsection of “Rationale” and “Specification” sections of the proposal.

Also, as you said in your solution, it would be straightforward for a front-end developer implementing a new wallet to add a check for the fee of a certain transaction type (i.e., trs.fee < 2 LSK), and prevent the transaction from being broadcasted if the check fails.

Best,

Iker

1 Like

I just created the pull request for this proposal on GitHub: https://github.com/LiskHQ/lips/pull/13

The changes from the version uploaded here are:

  • Minor format fixes.
  • Fix the size of Delegate Registration transaction in Appendix.
  • Add Include fee property in transaction JSON object subsection at Specification section.
  • Change ‘trs.fee’ property format to big-endian.
  • Include comment at Wallets subsection of Rationale section to address feedback from the discussion above.
  • Require LIP-0009 to avoid any kind of transaction replay attack from a previous protocol version

The pull request was just merged and the LIP is now drafted: