Welcome to this week's weekly recap. 

We've been working hard on improving the mobile wallet and have been consistent in releasing a new beta release every week. We've fixed many issues, with this week's focus being on unreliability testing. One issue we fixed was a bug in the old mobile wallets where if you created a new account while offline, it wouldn't broadcast and could get lost. We've also made sure that your transaction history is up to date with other nodes when you refresh it, and your mobile wallet is notified of transactions made on other devices. We're testing new builds internally, and the mobile wallet is getting really close to being ready for release. There are still a few outstanding bugs, but we're working on fixing those. Once we conclude internal testing, we'll release a new public beta, which I'll be publishing as I usually do on Twitter for you guys to continue testing.

We've been doing some stability testing and we've noticed a couple of issues. Specifically, when we tried to process a really large Sigchain (we're talking 5 to 10 thousand transactions), the system wasn't handling it very well. We're working on fixing that.

Before we move forward with adding more complexity to the mobile wallet, we want to make sure we've got all the bugs cleaned up. So, we'll be doing one final set of testing for the final release of the mobile wallet after we've addressed these issues.

So, the final mobile wallet build is going to include a lower level database that's been completed since late 2020. Originally, it was planned for version 6.0, but now it's being pulled into the mobile wallet release to provide constant time functionality. This is especially important for the mobile wallet since it has smaller hash maps. Without the constant time functionality, syncing time can slow down over time as more hash maps are needed. The old lower level database maintained some level of constant time, but as more hash maps were required, it required more disk seeks and iterations. With the constant time database, the number of instructions required to find any key is reduced by a factor of three.

I actually got a really good analogy to explain how it works. Imagine you have a set of encyclopedias like the Encyclopedia Britannica and you want to look up something specific, like Geronimo. With a hash map, the key itself (in this case, the letter "G") contains the information needed to locate the subsection where Geronimo is listed. You can think of the letters of the alphabet as being like the keys in a hash map.

So, to find Geronimo, you would start by opening up the "G" encyclopedia, which is like finding the correct bucket in a hash map. Then you would look at the index in that encyclopedia to find where Geronimo is listed, which is like finding the specific value associated with the key in a hash map.

In the example given, Geronimo is listed on page 27 and 57, so you would go to page 27 to find the data you need, and then use that information to locate the rest of the block. This is how a hash map works: it uses keys to quickly locate specific values in a large data set, making it an efficient way to store and retrieve data.

That's essentially how the constant time database works. And that's how we're able to do it without having to search and traverse conventional databases. You'd have to search through a tree of some sorts which would be kind of a separate index that would tell you roughly where that specific thing is. But every time you go up a layer of the tree, a binary search tree, then you actually require one more disk seek. Now on hard drives, disk seeks can be five to ten milliseconds. So it really limits your throughput every time you have to seek. Hard drives can read pretty quickly linearly but once you have to change the position that you're reading from that's when it starts to get much slower. 

Conventional database engines like Googles's LevelDB is a log structured merge tree, which essentially means it's a memory binary search tree that merges these trees together. It's designed for big data, which is great, but I've noticed that when I compare it to lower-level databases, like fstream or memory maps, it tends to slow down significantly. For example, when I write 10 million keys and try to read the very first key, it usually gets buried, and with LevelDB, the more keys you add, the slower it gets because there's more data to sift through. However, with the lower-level database, no matter if you're reading the first key or the 10 millionth key, it's always constant time, around 150,000 with fstream. If you use memory maps, you can even save yourself one memory copy because it manages the page caching and goes directly into the pages.

With the help of the constant time database, essentially, this is going to make synchronization faster when it comes to reading and writing, which will also save a lot of battery life. This is because every time you have to search for data, it requires electricity to do so, and the more operations you have, the more electricity you'll consume. The constant time database is going to be included in the mobile wallet as a sort of Easter egg from the 6.0 update. This is a practical place to test it before setting it towards production. We're also cutting the block size in half, which will reduce the chain size by a factor of two, and your sync time should double. With the constant time database, you could even triple your sync time, which will help with low bandwidth situations and make it easier for people to synchronize with their phones.

If you've recently updated your mobile wallet, you might have noticed a new syncing screen that appears while your Sigchain is syncing and indexing. This loading screen addresses an issue that some users were experiencing, where they would try to load their wallet while the sync chain was still syncing. This process can take some time, especially if you have a large Sigchain. Essentially, you're downloading another blockchain and verifying it through Merkle proofs into the main chain.

Testing has been going well, and we're currently working on adding support for client mode on the desktop wallet. This means that if you're not staking, you'll be able to run a client and save a lot of disk space. Right now, client mode takes up about a gigabyte of space, but we're hoping to cut that down to 500-600 megabytes with the final revisions. This will also double your sync speed, as you won't have to download all that extra data.

We're doing everything we can to reduce the bandwidth required, and we're looking into different compression algorithms to see if we can compress the blockchain data further. However, compression algorithms haven't been very effective so far, as every byte in blockchain data is utilized to some degree.

Basically, there are three phases to the plan. The first phase is already complete, and the second phase is the desktop wallet hard fork, which includes a bunch of new features. The third phase is the mobile wallet, which we're working on now.

The reason we want to release the mobile wallet after the hard fork is because there are going to be some changes to the way the network works, and we want to make sure the mobile wallet is compatible with those changes. Running a node (which is what the mobile wallet does) on a phone is pretty tricky, and we've never done it before. We're verifying all of these smart contracts and managing all of this state in a way that it's actually a node, so we need to make sure everything is working smoothly.

When there's a major network upgrade on Nexus (like the 5.1 hard fork), you'll need to upgrade your mobile wallet too, because there are going to be some changes to the way things are verified. The client mode is essentially processing blocks but with a smaller subset of transactions, so it's important that everything is working correctly.

It's great to see that our team is making consistent progress and getting new builds out regularly. Kendall and Christo are doing a great job, and even though I haven't had much time to work on the new feature build out lately, I've been focusing on tuning the mobile wallet. I came across a couple of issues with proof credit and Sigchain logging that needed to be fixed, but we were able to address them pretty quickly. In fact, we even had to rework some of the indexing systems in the client mode, but it was worth it because it's made the architecture feel really solid. The new mobile wallet release, candidate 11, has been running reliably and we're now optimizing it to reduce battery consumption on Android. Next up, I'll be focusing on the new feature build out, specifically getting the remote login and new API set up.

So, recently, Kesin has been working hard on testing suites, which includes things like unit tests and regression tests. These tests are essential to ensuring that the core code runs smoothly and that all the specific units of code are functioning as they should be. I remember inviting Kesin to share his thoughts on this matter, and he may be joining us soon if he has a microphone. It's always great to have someone who is knowledgeable in testing suites, as they play a critical role in the development process.

He's been doing regression tests and unit tests to make sure everything is working properly. He's been helping with the little details like updating the API semantics, documentation, and error codes. It may seem like a small thing, but it's really important to get all of these details right. I haven't always had the time to focus on these things, so I really appreciate all the work he's been doing. From what I've seen, he's done a great job of getting everything organized so that each test can be run individually.

The team has been working hard to fix bugs and test everything thoroughly. There have been a few beta cycles already and there may be a few more to come, but the release dates should be announced soon. We're also working on some new features and will be bringing in Kesin to discuss those in the next weekly recap. All in all, it sounds like everything is coming together nicely and we'll be able to start using these new features very soon. 

Thanks for your patience and stay tuned for more updates!

For more information on Nexus visit: https://www.nexus.io