When most people think about AI right now and the data powering it they immediately think of the Large-Language Models (LLMs) and the arms race that technology companies are in to create, curate, and deploy them. Working within Congress on AI I have seen a divide in the AI community emerge over the need for ever-growing larger models, also known as hyperscale models, or if more siloed and curated models are going to be the next step because of the cost and the power required to run and maintain large models. For example, if I am creating a person's insurance quote using machine learning (AI) I do not need to know the types of bird calls that exist. This type of information is extraneous and when dealing with LLM extremely costly to have.

As Brave has built out their AI assistant Leo they have partnered with several of the big LLMs out there. As these models have continued to be released an issue is quickly arising, being able to leverage this technology on a cell phone due to the enormous demands that they put on a system. For Brave's Leo being able to host AI locally and thus maintaining users' privacy is a critical component. At this time LLM benchmarking infrastructure is still in its infancy with NIST (National Institute of Standards and Technology) beginning its work on creating these with industry help and deploying them. 

While some NIST standards have been announced and deployed there are still plenty of gaps that need to be benchmarked to help address where AI is good and where it has room for improvement. This is were Brave stepped up as NIST has not yet looked at how these LLM affect phones and where there might be room for improvements. Creating and leveraging a device they called BLaDE (BatteryLab Device Evaluation) as seen below they took for the first time benchmarks and made them public. 

From this they were able to learn both things that we would expect and things that we did not. While larger networks offered larger discharge rates consuming more energy 4-bit codels ran faster than their 3-bit counterparts.

The key takeaways though were as follows

While advancements in hardware and algorithmic breakthroughs such as quantization show promise, on-device deployment currently impacts latency, device stability, and energy consumption. Offloading computation to local edge devices offers a viable alternative, providing improved performance and energy efficiency. Continued innovation in both hardware and software will be crucial for making local AI deployment practical, preserving user privacy, and enhancing sustainability.

I would expect to see continued improvement in the mobile device space and possibly even some devices that optimise AI over current smart phone needs like video playback or even picture taking. Phones have a limited amount of room and so possibly creating a new/different category that deals with AI is in the cards for the future. Android in particular would be the place I would expect to see this before iPhone would create such a thing. 

Please know I am not a financial advisor and make sure you do your own research! If you enjoyed this article and would like to support me further below are a few referral links that if you used when signing up I would appreciate it! Also, follow me on Twitter @Cje95_

BlazeStake

Have SOL you want an easy place stake and also be eligible for upcoming airdrops? Use my referral link here and join the protocol to be able to claim your portion of upcoming airdrops! 

Stacker News

Earn Sats on your posts on Stacker News with my referral link here! Comment on this post to let me know you signed up and what your username is on Stacker News so I can zapp you some sats! 

Fold App

Offers daily wheel spins to win free sats daily! If you sign up for Fold with my link here you will get 20,000 sats for free!  

Robinhood

Robinhood is offering an excellent signup deal if you use my link here where we will both be awarded free stock. All that you have to do is sign up and connect your bank account for the award! With commission-free crypto trading and the ability to set limit orders, it is a great hybrid offering in my opinion! Not to forget the addition of wallets and the ability to send and receive crypto!

Unstoppable Domains

Sign up using my link here and get a free $10 credit towards your first $40+ purchase!