New idea to mine blockchain

1

Hello everybody,

I know this is not really IPFS-related, but I wanted to share with you an idea to mitigate the electric consumption of the Proof of Work mechanism.

But before that, a short reminder: When a miner wants to mine a new block in the blockchain, he uses bruteforce and tries a large quantity of nonces. A nonce is a number that we add to the blockā€™s data during the process of mining. The perfect one is the one that makes the hash of the block begin with a fixed number of zeros. The first miner who finds it can add to himself a reward in the next block of the blockchain.

Ok but what can be optimized here?

Well, the miners do not work hand in hand, and what it implies is that 2 different miners can try for the same nonce separately, resulting in a waste of electric power. I assume that, even if the nonces are chosen randomly, 2 miners trying for the same nonce happens rather often, given the enormous amount of data all the miners of the network process each second.

Ok, now to explain my idea, letā€™s simplify a little bit. Imagine that there are only 10 miners in the network, identified by a letter ranging from A to J. If we want to be sure that 2 different miners can never try for the same nonce, we can make the miner A try all the nonces ending with ā€˜0ā€™, the miner B try all the nonces ending with ā€˜1ā€™, the miner C all the nonces ending with ā€˜2ā€™, and so on until miner J.

But how can we be sure that the miners follow this separation when mining?

What we can do is use the peerID (or in this example, their letter) of the miners to determine wich nonces they are allowed to mine. Letā€™s consider the function getMiningOffset(peerID)ā†’offset. As you can see, it takes in argument the peerID of the miner and returns a positive integer between 0 and 9, the offset. The allowed nonces for the given peerID are all the nonces matching offset modulo 10.

And if a miner manages to find the correct nonce but it isnā€™t one of his allowed ones, then his reward is denied by the network, so he has no incentive whatsoever to mine where he is not allowed to.

But what if a new miner arrives in the network?

Indeed there are issues with this naive approach, and adding a new miner is not the only one. It is practically impossible to have a perfectly bijective function with such small codomain.

To circumvent that, the getMiningOffset function can return values in a very large set (codomain), for example all the positive integers below 137 438 953 472. Why this number? Well, firstly because it is very big and thus it is very unlikely that 2 different peerIDs map to the same offset (collision risk), but also because it is a power of 2, so doing the binary addition is very quick to do. So for instance, the allowed nonces for miner A would be all the nonces matching getMiningOffset('A') modulo 137 438 953 472.

So what do you think of this idea? Maybe I am completely mistaken and 2 miners trying the same nonce is very uncommon. Anyway thank you for reading!

2 Likes
2

Sounds like youā€™ve just described a mining pool. Problem is trusting a centralized source to not rob you of the reward earned by a miner in the pool. I canā€™t think of how you could do that without a central source since you canā€™t trust individual miners to follow the rules either. I canā€™t think of any solution that isnā€™t implemented at the bitcoin mining algorithm level.

3

@CyberVenus
My idea is not similar to a mining pool. I mean, yes, the miners in a mining pool try to share the universe of nonces evenly between themselves, like I said here. But in a mining pool, there is only one miner that inserts the block and gets the reward. The other miners have to trust him that he will share it with them. This single miner can be viewed as a central source, but there is no such thing in my idea. The miners that donā€™t find the correct nonce donā€™t get any money, unlike in a mining pool. The reason for the miners to follow the rules are because of the algorithm: if a miner gives a nonce that is not allowed for his peerID, then the nodes of the network must deny him is reward. Having a standard (in this case a common algorithm for all the peers of the network) is different from having a central source.

4

Well, yes. Thatā€™s what I meant by I couldnā€™t think of a solution that doesnā€™t require modifications to bitcoin itself. Seeing as how this isnā€™t really a bitcoin specific forum I had assumed you were looking for something that didnā€™t involve that.

5

@CyberVenus Well if you want to fix a problem of Bitcoin (or any other cryptocurrency for that matter), you have no alternative but to make modifications to it. I know this is not the best forum to post something like that and Iā€™m sorry, but I thought it might interest people here as we are all dweb enthusiasts.
On another note, you might want to check my last message that I have edited quite significantly.

6

Yes. I agree with your proposal. I didnā€™t mean to imply you should get out or anything, just that I personally have no where near the knowledge required to actually make the necessary modifications to implement your idea in a bitcoin fork. The only problem I see with your idea is the illusion of fairness. It does sound like your proposal would rely on the blocks being well distributed amongst each miner, chance-wise, in some way so that a slow miner didnā€™t just never get anything anyways in the case that luck just didnā€™t work out their way.

7

Again, I can point out flaws all day, but I have no solution at present. I like the idea, but aside from making a profit off of mining, there doesnā€™t seem to be a significant point in the proposal as far as like securing the blockchain. For security Iā€™d say the best thing to do is just ensure the majority of the community stays involved and communicating so that if a problem begins to arise action can be taken to split the blockchain or to collectively ban a miner. (Ban from communication with other miners, which could restrict their ability to verify blocks.)

8

@CyberVenus

Iā€™m not asking anyone to implement it. I just want to talk about it to know if itā€™s really useful or not or if it has problems that I have failed to anticipate.

If you are a slow miner and the correct nonce is not one of your allowed ones, then too bad for you but you will maybe be more lucky for the next block. It is not less fair than the current PoW system where nonces are chosen randomly. And besides, I keep saying ā€˜the correct nonceā€™ but in fact there is an infinity of them. Each miner has at least one correct nonce in his allowed nonces.

The only way to improve the blockchain is not to improve security. The 2 main issues I see right now are the increasing size of the blockchain (200gb+ as of 2019) and the electric consumption of miners, that I try to tackle here.

1 Like
9

hi,

cool discussion ! and i have a question :

what is different your approach with proof of stake https://en.wikipedia.org/wiki/Proof_of_stake ?

regards

10

@josselinchevalay
I donā€™t understand PoS as much as I understand PoW, but from what I can tell, to validate a new block in a PoS context there is only one validator (the counterpart of miner in PoW). My idea is just an upgrade to PoW so that there can never be 2 miners doing the same cryptographic calculation to mine a new block and thus wasting electricity uselessly. So the first difference is that in PoS only one person ā€˜minesā€™ a new block whereas in my idea every miner works ā€˜hand in handā€™ to mine the new block. Secondly there is no stake in my idea, if the miners approve a fraudulent transaction, the network just rejects it, whereas in PoS the validator loses money if it happens. Thirdly, the cryptographic complexity of the calculation that the validator has to do in PoS is far lesser than in my idea (and in PoW in general) because if it was the case, a single validator would have to do the work that thousands and thousands of miners do in PoW in 10 minutes in order to validate a new block.

11

@JacopoStanchi The main issue with your idea is that every node out there is trying to solve a different equation. Simplified, you are trying to calculate a result below a specific value for the following

Hash(transaction blocks + my wallet id + nonce)

As each node has a unique wallet id, the required values of the nonce will be different for each node so making sure that they donā€™t calculate the same nonces could actually result in blocks not being found.

This algorithm could only be used in mining pools (where it probably already is) because the way they work is that you use the wallet id of the pool, and then trust that to pay you a ā€œshareā€ of what has been mined over a period of time

12

I think there might be confusion about some facts:

  1. As @MattewSteeples pointed out, you have to mine on the same wallet ID to ā€œshare the loadā€
  2. Exploring a given walletID ā€œnonce spaceā€ is not better (or worse) than exploring several individual ā€œnonce spaceā€. More on that below.
  3. The difficulty (aka the number of leading zero of the output hash you should have to get a reward) is dynamically adjusted so that blocks are mined every 10 min (on average)

(1) tells us that to cooperate, you have to trust the wallet iD of the group. Whoever have control over it have power. If you get rewarded only when you find, you have no interest in cooperating (see below) but have the risk of being robbed by the pool leader (mathematically itā€™s a pure loss). If you share the reward, you have the same amount on average, but the income is much more regular. Thatā€™s why mining pools exist in the first place, at the cost of having to trust the pool leader.

(2) For a given block, any pair (walletID, nonce) have the same chance of giving a winning hash and give you a reward. This is counter-intuitive but important to understand: Having explored 10^20 nonces for your wallet ID without having found a winning nonce DOES NOT increase your chances of finding a good one in the next N tries. Exactly as loosing 10 times at the roulette wheel in a casino DOES NOT increase (or decrease) your chances of winning the next round.

(3) but all that is not really important: the network will adjust its difficulty so that a winning (walletID, nonce) is found every 10 min. Wether its because of dumb uncooperative nodes doing the same computation again and again, or a fully cooperative network all computing on the same wallet ID and never trying the same nonce twice (will never happen, but you get the point).

No matter what, the energy spent for a block (network-wide) will be 10min * electrical power of ALL the mining nodesā€¦ Whatever their strategy. Having a good strategy only increase your cake share at the expense of the others. No effect on electrical consumption.

So. If you want to optimize electrical consumption, you only have two options: reduce the number of nodes (nothing you can do about that) or reduce the energy required for computing one hash (via new hardware, etc). :confused:

13

@MatthewSteeples

I agree that the miners try to solve different equations and the nonce is different for each miner, but I donā€™t understand why you say that it can result in blocks not being found. With this mechanism, more nonces are being tried each minute than with current PoW. Besides, if there was only one correct nonce, then maybe the set of allowed nonces of all the miners of the network may not contain this nonce. But thatā€™s not the case, there is an infinity of correct nonces and each miner has an infinity of them in their allowed nonces. So in my opinion blocks should always be found eventually.

Your formula is not quite right. It should be more like hash({previous block hash, block data, getOffset(wallet id) + n*2^37})
Come to think of it but instead of using a getOffset function we can just do wallet id + n*total number of possible wallet ids.

But in my idea the miners who didnā€™t find the nonce donā€™t get a share of the money, whereas in a pool they do.

14

@Akita
Iā€™m also adressing the addendums you made at the end of your message.

Miners in my idea donā€™t share the load. If they donā€™t cooperate (i.e. try unallowed nonces) their reward is denied by the network.

Iā€™m not arguing that, but the use of partioning into wallet ids nonce spaces is to not have overlap between the sets of nonces tried by 2 miners. Iā€™m not saying that we have a better chance of finding the correct nonce at each test, but that the rate of different nonces tried in one minute by all the miners is higher.

Why the limit of one block every 10 minutes? It is an arbitrary figure, each cryptocurrency has a different limit. The reason is to not create new blocks too frequently and thus saturate the blockchain. So we have to change the complexity of mining accordingly to match the 10 minutes of mining for Bitcoin, so that enough transactions can be added to the pending list during mining.
But with my idea, for the same complexity, you find a new block in less time. What it means is, for a given complexity, instead of finding a new block in 10 minutes, you find one in 5 minutes. What you can do is either increase the complexity so that it becomes 10 minutes again OR miners stop to consume electricity for 5 minutes before the new block is opened. Either you have more security OR less power wasted.

15

Reduction of calculations in order to cut power consumption is certainly an honorable means to an end but not the only means. If environmentally detrimental power consumption is the problem to be solved, then it should also be solved by environmentally benign power generation. Power your mining operation with alternative sources to fossil fuels.

16

You are right, I was referring to the Bitcoin Blockchain as an example. Sorry for not having made it clear. I thought you wanted to find a new way to mine existing blockchain, not designing a new one.

Miners wonā€™t stop for five minutes. They will start processing the new block, try to find the next one (trying nonces in their own nonce-space, of course) and submit it as soon as they can (with transactions). You have actually 2 pathes: more security (at the expense of throughput), more throughtput (at the expense of security). If you have a dynamic difficulty, then it will go back to that value, no matter what. If you donā€™t, blocks gets mined faster and faster with technical evolution, and you have a problem of centralisation/security very soon.

NB: You could argue that network would deny publishing too soon this block N (before 10 min), but your would be shifting the competing from the better hash rate to a better network access. Miners will precompute block N+1 and publish it at the beginning of the cycle. The one with better access (more open connections with other nodes, more bandwidth) will have it sent and accepted by a lot of nodes and have better chance of it being mined upon for round N+2 and be selected permanently with the rule of the longer chain. Network access is even more unequal than CPU/GPU/ASIC, so youā€™re actually decreasing security by discouraging nodes with high latency because them finding early is not an advantage anymore.

NB:

Thatā€™s the contrary. We change the difficulty so that not too many transactions per day are validated when processing power rise, so that the blockchain (the list of transactions, not the network) doesnā€™t grow too fast. Why do we want that? For decentralisation of validation: every crapy mobile app is able to catch up the validation of every transaction onchain, even if unable to mine, but thatā€™s less of a problem: you can trust that the records are healthy without mining.

17

There arenā€™t an infinite number of nonces, correct or otherwise. In Bitcoin (for example) itā€™s a 32-bit field.

The problem is that there is no ā€œoverlap of noncesā€ as far as the calculation is concerned. The nonce is part of the calculation, as is the nodes own wallet id. For example, you could calculate using a nonce of 10 and not get an acceptable value. I could calculate with a nonce of 10 and, because my wallet id is different, my value would be accepted.

18

Yes I have heard about miners going to Iceland to exploit geothermic energy. But given that most miners are based in China and that China has a power shortage right now, I canā€™t bring myself to think that most of the energy used on mining today is clean. Besides, even the clean energy could be used for a better purpose. Thatā€™s a really interesting subject but I admit that I donā€™t have enough knowledge on this.

No, the current block would be published as soon as it has been found and the miner would get their reward, however the next block would be opened at 10 minutes. Miners wouldnā€™t be able to precompute it because they would need to wait until 9 minutes and 59 seconds to be sure that every transaction in the block has been broadcasted to the network. Nonetheless I admit that this idea is very experimental, and I suspect that it may facilitate a 51% attack. But if it is really a bad idea, we can still use the other option which is increase the complexity of mining, and thus increase the security of the blockchain (failing a reduction of miningā€™s electrical consumption).

19

Yes sorry that was a mistake of me. I know there is a finite amount of nonces, what I meant is that all miners will ā€œstatisticallyā€ always have a correct nonce in their ā€œallowed nonce spaceā€. So the blocks will always be found eventually.
But 32 bits seems strange. It only amounts to 4 billion possible values. Even my PC with a 3GHz processor can compute this much values in a relatively short time.

What I mean by overlap is all the nonces tried by 2 different miners, and Iā€™m pretty sure there are overlaps with mining currently. But as the nonces are chosen randomly to minimize such overlaps I cannot assess how big they are. If the overlap is negligible, my idea is completely useless.
The correct nonce doesnā€™t depend on the wallet id. If you get an acceptable value with a nonce of 10, then I also get an acceptable value with a nonce of 10. The difference is who is able to claim the reward. If miner A finds an acceptable value with a nonce that ā€˜belongsā€™ to miner B, then he gets nothing and B gets the reward even though he did nothing.

20

They donā€™t need every transaction, they need enough to fill the block or at least enough to make it worth it to begin 5 min earlier. On a very saturated blockchain like bitcoin where even big juicy transactions are in the queue for a long time, you always have some to process. And even if the queue is empty (which basically never happens) or you donā€™t want to wait for it to fill up, nothing stops you from mining an empty block (!) You wonā€™t have transaction fees, but you get the reward, so itā€™s still worth not waiting to start finding a good nonce. You would still have to wait to publish it, but you can pretend you just started to compute, and it was the very first nonce you tried (pretty lucky, right? :sunglasses:)

In that case, you are back to square one about electrical consumption, network speed, etc, etc. the only difference is that the miners didnā€™t tried the same nonces, but it wonā€™t make much of a difference, would it? (Unless the crypto puzzle is useful by itself. Some crypto-currencies tried that, and you would compute to simulate protein folding, simulating physics, etc. There, the blockchaincouldnā€™t care less what you compute, but humans do and society benefits from miners never testing real-world hypotesis twice. These coins never reached the critical size, unfortunatelyā€¦ :confused: )

An additional note: for miner to explore the same nonce space (but different part of it), you will have to remove the wallet ID from the equation. You can change the consensus rule from ā€œhash(previous block hash, wallet ID, nonce) must have N leading zeroā€ to ā€œwalletID.sign(hash(previous block hash, nonce) with N leading zeros)ā€, so that miners explore different parts of the same space, but you can still verify who is the winner thanks to their signature.