Contact Us

Contact Us



Pleas confirm by checkbox


Uncategorized

Hacking ethereum to inject our own consensus alogrithm Part-2

Author_img
By Hemant Sachdeva August 17, 2018

This is the second blog post for hacking Ethereum. If you have not gone through the first blog post please click here and have your local environment set up. The main entry point for Ethereum is /go-ethereum/cmd/geth/main.go

Open up main.go and look at ‘main’ method, which is the entry point whenever we execute ./geth binary.  This function basically executes the application with whatever command parameter we have supplied to it. As we know in ‘go’ there is specific function ‘init’ which gets executed before calling the module, let’s see what’s inside in initialisation code. The important line to notice is:

 

app.Action = geth

 

So default action if we don’t supply any via command line parameter is geth. ‘geth’ is a function inside main.go. This function is actually starting up all background services for a node.

 

func geth(ctx *cli.Context) error {

node := makeFullNode(ctx)

startNode(ctx, node)

node.Wait()

return nil

}

 

Let further analyse the function ‘makeFullNode(ctx)’. Go to its implementation, we see at 2nd line  “utils.RegisterEthService(stack, &cfg.Eth)”

It is registering “eth” service using utils. As we are running full node following is the code which actually gets executed:

 

func RegisterEthService(stack *node.Node, cfg *eth.Config) {

.....

err = stack.Register(func(ctx *node.ServiceContext) (node.Service, error) {

fullNode, err := eth.New(ctx, cfg)

if fullNode != nil && cfg.LightServ > 0 {

ls, _ := les.NewLesServer(fullNode, cfg)

fullNode.AddLesServer(ls)

}

return fullNode, err

})

.....

 

This function is registering new service to the stack which is basically a node. It means whenever we start a node this service will be executed. Let’s analyse the line fullNode, err := eth.New(ctx, cfg)

 

Going to the implementation of the ‘eth.New’ we go to ‘backend.go’ file. Looking at the constructor code, this service is nothing but Ethereum service. This constructor function configures the whole Ethereum. As our goal for this tutorial is to change the consensus algorithm following is the code of interest under ‘backend.go -> func New’

 

eth := &Ethereum{

config: config,

chainDb: chainDb,

chainConfig: chainConfig,

eventMux: ctx.EventMux,

accountManager: ctx.AccountManager,

engine: CreateConsensusEngine(ctx, &config.Ethash, chainConfig, chainDb),

shutdownChan: make(chan bool),

networkID: config.NetworkId,

gasPrice: config.GasPrice,

etherbase: config.Etherbase,

bloomRequests: make(chan chan *bloombits.Retrieval),

bloomIndexer: NewBloomIndexer(chainDb, params.BloomBitsBlocks),

}
So the code where we need to make changes is CreateConsensusEngine. Looking at the code it is clear that if we want to have our own consensus algorithm we need to implement ‘engine’ interface. Go to location ‘go-ethereum/consensus/‘ and created a new directory ‘myalgo’. Let’s make a very simple class (consensus algorithm) which will just print the message at various stages of consensus workflow. After that, we will extend it into the more advanced algorithm. For this we need to create two files ‘myalgo.go’ and ‘api.go’ under location ‘go-ethereum/consensus/myalgo’. You can look at those files from GitHub, the content which is important for us to discuss further is listed below:
myalgo.go
func (MyAlgo *MyAlgo) VerifyHeader(chain consensus.ChainReader, header *types.Header, seal bool) error {

log.Info("will verfiyHeader")

return nil

}
func (MyAlgo *MyAlgo) VerifyHeaders(chain consensus.ChainReader, headers []*types.Header, seals []bool) (chan<- struct{}, <-chan error){

log.Info("will verfiyHeaders")

abort := make(chan struct{})

results := make(chan error, len(headers))
go func() {

for _, header := range headers {

err := MyAlgo.VerifyHeader(chain, header, false)

select {

case <-abort:

return

case results <- err:

}

}

}()

return abort, results

}
func (MyAlgo *MyAlgo) VerifyUncles(chain consensus.ChainReader, block *types.Block) error {

log.Info("will verfiy uncles")

return nil

}
func (MyAlgo *MyAlgo) VerifySeal(chain consensus.ChainReader, header *types.Header) error{

log.Info("will verfiy VerifySeal")

return nil

}
func (MyAlgo *MyAlgo) Prepare(chain consensus.ChainReader, header *types.Header) error{

log.Info("will prepare the block")

parent := chain.GetHeader(header.ParentHash, header.Number.Uint64()-1)

if parent == nil {

return consensus.ErrUnknownAncestor

}
header.Difficulty = MyAlgo.CalcDifficulty(chain, header.Time.Uint64(), parent)

return nil

}
func (MyAlgo *MyAlgo) CalcDifficulty(chain consensus.ChainReader, time uint64, parent *types.Header) *big.Int {

return calcDifficultyHomestead(time, parent)

}
func (MyAlgo *MyAlgo) Finalize(chain consensus.ChainReader, header *types.Header, state *state.StateDB, txs []*types.Transaction,

uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error){

log.Info("will Finalize the block")

header.Root = state.IntermediateRoot(chain.Config().IsEIP158(header.Number))

b := types.NewBlock(header, txs, uncles, receipts)

return b, nil

}
func (MyAlgo *MyAlgo) Seal(chain consensus.ChainReader, block *types.Block, stop <-chan struct{}) (*types.Block, error){

log.Info("will Seal the block")

//time.Sleep(15 * time.Second)

header := block.Header()

header.Nonce, header.MixDigest = getRequiredHeader()

return block.WithSeal(header), nil

}

As you can see most of them are just dummy function, always returning success result. In this algorithm, we are not actually finding any nonce, so after plugging this consensus algorithm it is expected that block is generated in an infinite loop without any delay. Let’s plug this algorithm and see what we are getting in logs. To plug this algorithm in Ethereum insert following code to “CreateConsensusEngine” function in ‘go-ethereum/eth/backend.go’ file.

 

if chainConfig.MyAlgo != nil{

fmt.Println(“myalgo is configured as consensus engine")

return myalgo.New(chainConfig.MyAlgo, db)

}

 

So this method basically checks if we have configured our configuration to use ‘MyAlgo’ or not. To enable ‘MyAlgo’ we need to make changes to our ‘privategensis.json’, Replace the content of the file with the following:

 

{

"config": {

"chainId": 15,

"homesteadBlock": 0,

"eip155Block": 0,

"eip158Block": 0,

"myalgo" : {}

},

"difficulty": "2000000",

"gasLimit": "21000000",

"alloc": {

}

}

 

Note the additional change “myalgo:{}” this line will enable our algorithm instead of default one. Delete previously create a directory from tutorial-1 and initialise Ethereum with new configuration:

 

> rm -rf ~/.ethereum/myprivatenet
> ./geth --datadir ~/.ethereum/myprivatenet init privategensis.json

 

Start the node as we did previously in the last tutorial. You will notice some additional logs and module after starting the node:

 

INFO [06-21|16:49:41.400938] Starting P2P networking

 

myalgo is configured as consensus engine

INFO [06-21|16:49:43.658541] UDP listener up self=enode://f08e5aa4fbde25b09ee977001af4a642abfd150c47e571ec5b69deed8c8c644c0a785d47a890adb6b2b55155cd32d7c035f4ec3ce7f8af54cd07219c8ad06ee7@[::]:30301

INFO [06-21|16:49:43.658717] RLPx listener up self=enode://f08e5aa4fbde25b09ee977001af4a642abfd150c47e571ec5b69deed8c8c644c0a785d47a890adb6b2b55155cd32d7c035f4ec3ce7f8af54cd07219c8ad06ee7@[::]:30301

INFO [06-21|16:49:43.661563] IPC endpoint opened url=/Users/hemants/.ethereum/myprivatenet/geth.ipc

INFO [06-21|16:49:43.662937] HTTP endpoint opened url=http://localhost:8101 cors=http://localhost:8000 vhosts=localhost

Welcome to the Geth JavaScript console!
instance: Geth/v1.8.12-unstable/darwin-amd64/go1.9.2
modules: admin:1.0 debug:1.0 eth:1.0 miner:1.0 myalgo:1.0 net:1.0 personal:1.0 rpc:1.0 txpool:1.0 web3:1.0

 

Log message: myalgo is configured as consensus engine, confirm that our configuration has been picked up and now Ethereum is configured with our own consensus algorithm. Also, a new module gets added to the console, it means we can interact with our consensus algorithm from the console. Great !!. This ‘myalgo’ got added due to the following two changes:

 

In consensus/myalgo/api.go:
type API struct {

chain consensus.ChainReader

myAlgo *MyAlgo

}

func (api *API) EchoNumber(ctx context.Context, number uint64) (uint64, error) {

fmt.Println("called echo number")

return number, nil

}
In internal/web3ext/web3ext.go:
var Modules = map[string]string{

"admin": Admin_JS,

...

"txpool": TxPool_JS,

"myalgo": MyAlgo_JS,

}

const MyAlgo_JS = `

web3._extend({

property: 'myalgo',

methods: [

new web3._extend.Method({

name: 'echoNumber',

call: 'myalgo_echoNumber',

params: 1,

inputFormatter: [null]

}),

]

})

 

This ‘echoNumber’ function is just echoing whatever we pass as an input parameter. This is to just show how we can add our own custom console modules.

 

> myalgo

{

echoNumber: function()

}

> myalgo.echoNumber(5)

called echo number

5

 

Now let’s start miner and see which function is being called from our consensus algorithm. As you start the miner you will notice that instantly it start mining blocks, here is the log:

 

> miner.start(1)

null

> INFO [06-21|17:01:52.43016] Transaction pool price threshold updated price=18000000000

INFO [06-21|17:01:52.43104] Etherbase automatically configured address=0x33954C10f5789A5914a09F0B1ef64CD26E0FA66E

INFO [06-21|17:01:52.43109] Starting mining operation

INFO [06-21|17:01:52.43113] will prepare the block

INFO [06-21|17:01:52.431516] will Finalize the block

INFO [06-21|17:01:52.431532] Commit new mining work number=1 txs=0 uncles=0 elapsed=406.763µs

INFO [06-21|17:01:52.431749] will Seal the block

INFO [06-21|17:01:52.431797] Successfully sealed new block number=1 hash=71810a…2882f3

INFO [06-21|17:01:52.436641] ? mined potential block number=1 hash=71810a…2882f3

INFO [06-21|17:01:52.436681] will prepare the block

INFO [06-21|17:01:52.43676] will Finalize the block

INFO [06-21|17:01:52.436768] Commit new mining work number=2 txs=0 uncles=0 elapsed=90.323µs

INFO [06-21|17:01:52.436991] will Seal the block

INFO [06-21|17:01:52.437038] Successfully sealed new block number=2 hash=b6bef3…f2ed9b

INFO [06-21|17:01:52.437498] ? mined potential block number=2 hash=b6bef3…f2ed9b

INFO [06-21|17:01:52.437641] Mining too far in the future wait=2s

INFO [06-21|17:01:54.441536] will prepare the block

INFO [06-21|17:01:54.441837] will Finalize the block

INFO [06-21|17:01:54.441891] Commit new mining work number=3 txs=0 uncles=0 elapsed=2.004s

INFO [06-21|17:01:54.442025] will Seal the block

INFO [06-21|17:01:54.442176] Successfully sealed new block number=3 hash=1d3448…e8ec87

INFO [06-21|17:01:54.442574] ? mined potential block number=3 hash=1d3448…e8ec87

INFO [06-21|17:01:54.442616] will prepare the block

INFO [06-21|17:01:54.442739] will Finalize the block

INFO [06-21|17:01:54.442757] Commit new mining work number=4 txs=0 uncles=0 elapsed=144.138µs

INFO [06-21|17:01:54.442807] will Seal the block

 

Let’s check the height of blockchain :

 

> web3.eth.getBlockNumber(function(e,r){ console.log(r)})
17

 

Wow in a matter of seconds 17 blocks got generated. Now we have our own private Ethereum running locally with our own consensus algorithm, That’s amazing. Now we can implement logic into consensus algorithm.

 

Before we started implementing our own algorithm I would like to highlight one more insight of the code. Ethereum service (backend.go) contains a miner which orchestrate two very important module “worker” and “agent”.

The role of the worker is to generate blocks or work. It collects all pending transaction and makes them part of the block. Please note that here ‘generate block’ does not mean that it mined the block, it simply means we are initialising the block structure and doing some preliminary validations or simply it generate the work. Worker first calls engine’s Prepare function, in case of ethereum’s standard algorithm (ethash), block difficulty is calculated. Then it calls the engine’s Finalize method.

Role of an agent (cpuAgent) is to do work generated by the worker. So the agent actually mines the block, by finding correct nonce in case of POW. The communication between the worker and agent is done via event. When the worker generates a block it generates an event which agent is listening to. Once agent mine the block, it generate the event which worker is listening to and worker starts creating new work for an agent to work on. This loop continues.

For this tutorial, I am planning to implement a very basic consensus algorithm. The details of which are as follows :

 

At Miner side

 


– There is a file called problems.json, which contains 10 problems to be solved by the miner. These problems are a simple arithmetic equation like “3 + 14 * 49”.

– Miner, while sealing the block, will first select the problem from problems.json based upon index which will the first character of current block’s parent hash. So if current block’s parent hash is

“0x29c9f33844df1df9d808412104f08dd318c016f7f06faad241fb0f8c79911c75” then the index to pick problem will be ‘2’ ( ignore 0x which represents hexadecimal),

“0xc4fbd0b2c6bc524794fbff7af77eb0d98ad42aa4c6a34776bc66210502e908e2” then the index to pick problem will be ‘2′ ( c in decimal is 12, so 12%10 = 2)

– Once the miner finds the problem, he will try to solve the problem and once the solution is found he put the solution in ‘nounce’ field of the block.

 

At Validator side

 


– Whenever the validator node got a block, it first selects the problem from “problems.json” using block’s parent hash as described above.

– Validator computes the solution and compares its solution with the Nonce.

– If both are equal, validator put the block in his chain, else he rejects the block.

 

Run Ethereum Local cluster

 


To run multiple nodes we just need to specify different ‘data-dir’ and different ‘rpc-port’. So to initialise node 1

 

./geth --datadir ~/.ethereum/myprivatenet-1 init privategensis.json

To initialise node 2

 

./geth --datadir ~/.ethereum/myprivatenet-2 init privategensis.json

Similarly, you can initialise as many nodes as you want.

To start node, you need to mention previously created ‘data-dir’ and port. So to start node 1

 

./geth -rpc -rpcapi 'web3,eth,debug,personal'  -rpcport 8545 --rpccorsdomain '*' --datadir ~/.ethereum/myprivatenet-1 --networkid 15

To start node 2

 

./geth -rpc -rpcapi 'web3,eth,debug,personal'  -rpcport 8546 --rpccorsdomain '*' --datadir ~/.ethereum/myprivatenet-2 --networkid 15

As of now, these two nodes are running independently without knowing each other. So to connect node1 to node1, go to geth console and find node1 enode address by issuing following command:

 

node1 > admin.nodeInfo.enode

This will return enode address of the node1 like

 

node1 > "enode://c7a9acd7a6f381bb33a51767e07989939448fae4db2ec95cc736f88bebd39c6fce1e75b333225b435a0398ff7377805b08fb8792e6a0886d855ff315e54d5cf6@[::]:30303”

Go to node2 geth console and add node1 using its enode address as follows

 

node2 > admin.addPeer("enode://c7a9acd7a6f381bb33a51767e07989939448fae4db2ec95cc736f88bebd39c6fce1e75b333225b435a0398ff7377805b08fb8792e6a0886d855ff315e54d5cf6@[::]:30303”)

You can check if the connection is successful or not by issue ‘net’ command. This command will return the count of connected nodes.

 

> net

{

listening: true,

peerCount: 1,

version: "1",

getListening: function(callback),

getPeerCount: function(callback),

getVersion: function(callback)

}

 

The complete code is available here: Hope I am able to give you a taste of Ethereum internals and you must be equipped enough to hack it further.

Related posts
VPC Sharing Using AWS RAM (Resource Access Manager)
Uncategorized

VPC Sharing Using AWS RAM (Resource Access Manager)

By shekhar.wagh June 18, 2019
Reduce App size with On Demand Resources
Uncategorized

Reduce App size with On Demand Resources

By shekhar.wagh May 29, 2019
iMessage Stickers and Apps
Uncategorized

iMessage Stickers and Apps

By shekhar.wagh May 27, 2019
What is UX Writing?
Uncategorized

What is UX Writing?

By shekhar.wagh April 08, 2019
AWS ECS (Amazon Elastic Container Service )
Uncategorized

AWS ECS (Amazon Elastic Container Service )

By shekhar.wagh March 26, 2019
Scala code analysis and coverage report on Sonarqube using SBT
Uncategorized

Scala code analysis and coverage report on Sonarqube using SBT

By shekhar.wagh March 19, 2019
Introduction to Akka Streams
Uncategorized

Introduction to Akka Streams

By shekhar.wagh March 08, 2019
Decentralized Applications - Utilizing the Power of Blockchain Technology
Uncategorized

Decentralized Applications - Utilizing the Power of Blockchain Technology

By shekhar.wagh March 05, 2019
App Store Connect API To Automate TestFlight Workflow
Uncategorized

App Store Connect API To Automate TestFlight Workflow

By shekhar.wagh February 28, 2019
Using Custom Metrics for CloudWatch Monitoring
Uncategorized

Using Custom Metrics for CloudWatch Monitoring

By shekhar.wagh February 06, 2019

Stay updated

Get the latest creative news from Fubiz about art, design and pop-culture.