Writing your own Bitcoin miner in Kotlin
Kotlin + Bitcoin = Mining

Writing your own Bitcoin miner in Kotlin

By Alexandru-Balan | Pr0gram Failure | 12 Apr 2020


Before word...

I have to say from the beginning that at the end of this tutorial you won't be able to mine Bitcoin, since you need much more than a piece of software to do that. You need dedicated hardware, probably you need to join a mining pool, etc. Here you will learn what the mining process is, how it works and how to write code that could mine Bitcoin. Since the Bitcoin blockchain is huge (around 200 GB) I also wouldn't be able to connect the mining program I wrote to it due to insufficient space.

With this disclaimer out of the way lets get to writing our own bitcoin miner

What is the mining process?

Mining is the process of verifying and adding transactions to the public ledger. Also it serves as a way to introduce new Bitcoin in the network. Miners are rewarded for each new block they find. 

In order to find a block you must find a SHA-256 hash that meets the difficulty requirements set in the block header. You also need to verify about 1MB worth of transactions in order to be eligible for the reward.

Block header? What is this?

A block header is a set of information about a block. If you want to see the block history there are sites that record that and update the info as new blocks are being added; for example one of them is btc.com

The information included in a block header is structured in different fields as follows:

  • Version -- this is the version of a block (4 bytes number)
  • Previous Block Hash -- The 256-bit hash of the previous block
  • Merkle Root Hash -- This is a special hash updated whenever a new transaction is added to the block
  • Time -- A 4 bytes timestamp representing the number of seconds since the beginning of the epoch (01-01-1970)
  • Bits -- A packed version of the difficulty number
  • Nonce -- 32-bit number

The first 5 components of a block header are obtained from the network. Our job as miners is to find the nonce that when combined with all the other fields will produce a SHA-256 hash smaller than the numerical counterpart of the difficulty bits . 

As per the official Bitcoin documentation we are going to work with the hexadecimal little-endian representation of all fields. In order to find the hash of the block we concatenate all the fields in the block header and we hash this value and the result of that hash to get the hash of the block. If this hash that we found meets the difficulty requirements than we were lucky and discovered the new block that needs to be added to the blockchain, otherwise we increment the nonce value and try again.

Let's start writing the program

We will use the Kotlin programming language and take advantage of the coroutines feature.

We are going to start by defining a class that will represent the block header:

class BlockHeader private constructor(private val arguments : Map<String, Any>) : Hashable {

    // Attributes of the class
    private val version: String = arguments["version"] as String
    private val previousBlockHash : String = arguments["previousBlockHash"] as String
    private val merkleRootHash : String = arguments["merkleRootHash"] as String
    private val timestamp : String = arguments["timeStamp"] as String
    val difficultyBits : String = arguments["difficultyBits"] as String
    private var nonceHex : String = arguments["nonce"] as String
    var nonce : Long = arguments["nonceInteger"] as Long
        set(value) {
            field = value
            nonceHex = nonce.toString(16).swipeEndianity()
        }
    var hash : String = "NOT COMPUTED"

The BlockHeader class will receive at initialisation a series of key-value pairs representing all the information normally present in a block header. As you can see, the class has some special features. First, it implements the Hashable interface; this means that this class must have a hash() method which will return a hexadecimal representation of the SHA-256 hash of the block.

The second interesting aspect is the private constructor. This means that the BlockHeader object will not be instantiated using the constructor, but through a special method that returns an instance of this class. I implemented this behaviour because I wanted to heavily format the information before sending it to the constructor. I needed to make it possible for the developer using my project to pass integer values and normal hash strings when creating a block header, but in reality those values must be written in hexadecimal little-endian format in order to be used in the hashing process.

The way to create such a method in Kotlin is through a companion object, which is similar to static methods in Java.

companion object {
        // Get a new block header instance
        fun newInstance (version: Int, previousBlockHash: String, merkleRootHash: String,
            timestamp: Int, difficultyBits: Int, nonce: Long) : BlockHeader {

            val arguments : Map <String, Any> = mapOf(
                "version" to version.toString(16).swipeEndianity(),
                "previousBlockHash" to previousBlockHash.swipeEndianity(),
                "merkleRootHash" to merkleRootHash.swipeEndianity(),
                "timeStamp" to timestamp.toString(16).swipeEndianity(),
                "difficultyBits" to difficultyBits.toString(16).swipeEndianity(),
                "nonce" to nonce.toString(16).swipeEndianity(),
                "nonceInteger" to nonce
            )

            return BlockHeader(arguments)
        }
    }

Now when someone wants an instance of the BlockHeader class he/she can call the the BlockHeader.newInstance() method to get one. The toString(16) method is used to transform an integer into a hexadecimal string representation. The swipeEndianity() method is an extension that I made to the String class; it is used to get a little-endian string out of a big-endian one and vice versa. You can check on the process online or you can see the actual function in the github project down bellow.

Finally let's see how we can get a hash out of all that information.

override fun hash(hashAlgorithm: Hashable.Algorithm): String {
        val messageDigest : MessageDigest = MessageDigest.getInstance(hashAlgorithm.algorithm)

        val bytes : ByteArray = messageDigest.digest(this.getContent().hexStringToByteArray())
        messageDigest.reset()
        val finalHashBytes : ByteArray = messageDigest.digest(bytes)

        hash = finalHashBytes.toHexString()

        return hash
    }

We are going to use the MessageDigest class that Java makes available. To instantiate  MessageDigest you need to specify the hashing algorithm that you want to use. In order to eliminate all mistake possibilities I created an enumeration, Hashable.Algorithm, that has the right initialisation strings for a couple of hashing algorithms. In our case I passed the value Hashable.Algorithm.SHA256 to the hashAlgorithm parameter.  The getContent() method will return the concatenated version of all the information in our block header.

There are two more extension methods here, the String.hexStringToByteArray() that turns a hexadecimal string representation into an array of bytes, and the ByteArray.toHexString() which will do the opposite. They are necessary because we need to hash the bytes that the concatenated info represents not the actual string that we see and which is human readable. 

The hash method itself looks pretty simple and it actually is. The bitcoin block hash is obtained by hashing the concatenated version of all the information in the block header, then taking that resulted hash and hashing it again. In our method the hashing is done by the digest() method which takes an array of bytes as input and puts out another array containing the hashed bytes. 

Repeating the hashing process

Since we probably won't get lucky on the first try, or even the first million tries, we need to repeat the hashing process and compare the hash we got to the difficulty. Remember, if our hash is smaller then we found a new viable block hash.

We could do all those things in the main method, but it would make more sense to create an object which can do the following:

  • launch multiple threads to search for the hash
  • increment the nonce of the header
  • hash the headers
  • compare the hashes to the difficulty
  • stop all threads when it found the right hash and give us the used nonce and the hash of the block
class Finder (private var blockHeader: BlockHeader) {
    private var found : Boolean = false
    // We first instantiate a collection of jobs
    private val jobs : MutableList<Job> = MutableList(NUMBER_OF_THREADS) { Job()}
    private var difficulty : Double
    private var difficultyString : String = blockHeader.difficultyBits.swipeEndianity()
    private val mutex : Mutex = Mutex()
    var foundNonce : Long = -1
    var foundHash : String = "No Hash Found"

    /**
     * In the constructor we compute the target difficulty of the hash
     */
    init {
        val exp = difficultyString.slice(0 until 2).toLong(16)
        val coef = difficultyString.slice(2 until difficultyString.length).toLong(16)

        difficulty = coef * 2.0.pow((8*(exp - 3)).toDouble())
    }

I created a class called finder to take care of all the above requirements. I've set the number of threads to 10 for my test case and initialised an empty list of jobs. 

In the init method, this is the primary constructor in Kotlin, I calculated the difficulty in the form of a floating point number. The used formula is the one from the Bitcoin official wiki.

Now comes the interesting part. We need to start the jobs and make them search for the right hash.

suspend fun find () : Long? {
        // Launching the coroutines
        for (i in 0 until NUMBER_OF_THREADS) {
            jobs[i] = GlobalScope.launch { search(blockHeader.nonce + i * STEP, blockHeader.nonce + (i+1) * STEP) }
        }

        while (!found) {
            // We check each job to see if there is still one active
            if (jobs.all { it.isCompleted or it.isCancelled }) {
                break
            }
        }

        if (!found) return null

        // If we reach this part it means we found a hash
        return foundNonce
    }

    private suspend fun search (lowerBound : Long, upperBound : Long)  {
        val blockHeaderCopy = blockHeader.copy()

        for (i in lowerBound .. upperBound) {
            // We increment the nonce and hash the block header
            blockHeaderCopy.nonce = i
            blockHeaderCopy.hash(Hashable.Algorithm.SHA256)

            // We check to see if we found the right hash
            if (blockHeaderCopy.hash.swipeEndianity().toDouble(16) <= difficulty) { // We found the right hash

                mutex.withLock {
                    found = true
                    foundNonce = i
                    foundHash = blockHeaderCopy.hash
                    jobs.forEach {
                        it.cancelAndJoin()
                    }
                }
            }
        }
    }

The find() method is public and serves as an interface to the outside world. It starts the jobs, or as they are known in Kotlin, the coroutines and then blocks the calling thread untill all coroutines have completed or were cancelled. When launching the coroutines it specifies to the search() method in which interval it should take the nonce. The STEP is 10.000.000 so in total we are going to go through 100.000.000 values trying to find the right hash for our test purposes.

The search() method is private and this is really the method doing the heavy lifting as it varies the nonce and hashes the block header then comparing the value of the hash to the difficulty. Notice how we used the mutex to perform critical writing operations. Also once a coroutine has found the right nonce it cancels all the others since there is no point in going further.

You might have noticed the suspend keyword in both functions. This tells Kotlin that those functions either launch other coroutines or are to only be used inside coroutines.

Using everything in the main method

In broad strokes we are done, we have a block header and a class that can hash that header a lot of times until it finds a good hash. So let's use it all in the main method in an example

suspend fun main () {

    // 0. Get an instance of the testConstants
    val testObject = TestConstants

    // 1. Get an instance of a BlockHeader
    val blockHeader: BlockHeader = BlockHeader.newInstance(
        testObject.VERSION,
        testObject.PREVIOUS_BLOCK_HASH,
        testObject.MERKLE_ROOT_HASH,
        testObject.TIMESTAMP,
        testObject.DIFFICULTY_BITS,
        testObject.START_NONCE
    )

    // 2. Get a finder object
    val finder = Finder(blockHeader)

    // 3. Use the new finder object to try and find the hash
    // Average time to find this particular hash : 5-6 seconds (Intel i7 3.2GHz with 8 cores)

    val startTime = System.currentTimeMillis()
    GlobalScope.launch { finder.find() }.join()
    val endTime = System.currentTimeMillis()

    println("\nFound nonce and hash of the block: (found in ${(endTime - startTime) / 1000} seconds)\n")
    println("Good Nonce: " + finder.foundNonce)
    println("Block hash: " + finder.foundHash.swipeEndianity())
}

The TestConstants class I filled with some block header values known to yield a result, a block hash that is. Let's see some output from my github project.

The starting block header:

Version	:	0x20400000 (541065216)
Previous Block Hash	:	00000000000000000006a4a234288a44e715275f1775b77b2fddb6c02eb6b72f
Merkle Root Hash	:	2dc60c563da5368e0668b81bc4d8dd369639a1134f68e425a9a74e428801e5b8
Timestamp	:	0x5db8ab5e (1572383582)
Difficulty	:	0x17148edf (387223263)
Nonce	:	0xb2d05e00 (3000000000)
Block Hash	:	NOT COMPUTED

The first 5 hashes:

Nonce 3000000000	:	70ba305ff525556330ab7f3fc3f342f2e82acd8d896e52dee84c0fec07fd8881
Nonce 3000000001	:	e16392883f05773debd5be4f8e9b99d3445c3539b031cd857ac0dc48de85c3f4
Nonce 3000000002	:	e7becb7c0bc3b14370dc33f289822e61b706febaae6f0ba7b9c96f4c0e31ffed
Nonce 3000000003	:	741fe37c2260738ceaeab90429b8adce1f1c1887a2a43855a79353cf35725e05
Nonce 3000000004	:	be3336f846a487f00de37b1c7565e6dcf600b25fbb54025cc7776337b5d6ebc1

Found nonce and hash of the block: (found in 7 seconds)

Good Nonce: 3060331852
Block hash: 0000000000000000000d7612d743325d8e47cb9e506d547694478f35f736188e

Conclusions

Writing my own miner was pretty fun and I've learned a lot from this project. I wanted to write a post about all this and put my code out there for people that might want or need to do something similar and I hoped to spare them some hours of searching and headaches. 

Remember that the presented code is not the full version, you can find the whole project on GitHub.

I hope you found this little tutorial interesting and educational. I would like to hear your opinions in the comments bellow. Should I have done something different? Did you find it useful? Or you have other questions? Let me know. 


Alexandru-Balan
Alexandru-Balan

Happily married to a wonderful woman. Linux enthusiast, software developer and hacker of all things. I may be stupid, but at least I won't try to scam you.


Pr0gram Failure
Pr0gram Failure

A blog dedicated to development subjects that every programmer deals with all the time. Simple things we all google and then ask ourselves "are we even developers?" or things we do when writing software that we are ashamed of and would never share with others. This blog is dedicated to the average Joe of programming.

Send a $0.01 microtip in crypto to the author, and earn yourself as you read!

20% to author / 80% to me.
We pay the tips from our rewards pool.