Raven Protocol

In November, Raven Protocol won the community vote for the infamous CryptoDiffer AMA. We spent about an hour hanging out with a some of the smartest minds in the blockchain industry openly discussing the use cases of Raven, diving into the technical details, and all the milestones we’ve achieved. It was great to connect and grow with such a wonderful community. Here is the transcript of the AMA for your reading pleasure! #blockchain #crypto #machinelearning #artificialintelligence #datascience #deeplearning

🐦⬛🪺Ravnest Dev Updates: 1. Speedups and improvements to the existing over the internet training mechanism with gRPC. 2. A new, TCP communication enabled mode for Ravnest that can run our distribution strategies over localised networks, is in the works. This will help cater to requirements which need localised training over LAN, ethernet, InifiniBand networks. 3. Comparisons are being performed against asynchronous distributed training algorithms from the Pipedream, ColossalAI and torchgpipe frameworks. https://lnkd.in/eX9GZYBb

Ravnest Updates 🐦⬛🪺 1. We have conducted a few decentralised training experiments using our algorithm across models like ResNet, Inception and BERT LLM which showed good convergence properties. These results are now available in our arXiv paper. (Link: https://lnkd.in/eiCshEwd) 2. We will be regularly adding more benchmarks and results on popular models. 3. A detailed Readthedocs page is underway with complete documentations and tutorials on our framework. We will be releasing this soon. We will also be updating the contribution guidelines for our GitHub repository and inviting PRs. 4. Many new features, improvements and bug fixes have been developed and tested across different branches. We will be merging them to make a cohesive stable release. Do consider giving our GitHub Repository a star! 🌟 Your support means a lot! GitHub: https://lnkd.in/eX9GZYBb

Hello Everyone - Ravnest Update: 🐦⬛🪺🧠 Some of you open source enthusiasts are noticing activity on GitHub! We’re optimising and benchmarking our algorithm on popular deep learning models and datasets. Specifically, we are recording performance metrics of Ravnest on LLMs and CV models. We have scripts ready for distributed training of ResNet50 (a popular CV model which we are training on the TinyImagenet dataset) It is a good reference for how distributed training can be executed using Ravnest’s functions. Check the GitHub for details: https://lnkd.in/ewvdkX2X Retweet: https://lnkd.in/eziFRawR

We are thrilled to share the blueprints of Ravnest’s architecture. Figure 1 depicts the process of matchmaking, cluster formation and model fragmentation into submodels on an intermediary server. The requester and all available compute nodes will connect to this intermediary server to participate and get the training started. Figure 2 elucidates upon what goes on inside each cluster, running parallelly. It shows how data is fed through the different nodes as micro batches and asynchronous training is achieved using zero-bubble mode parallelism. Figure 3 shows an overview of how Global model parameter averaging takes place across a set of clusters in accordance with the parallel multi-ring all reduce method described in our arxiv pre-print. This is triggered periodically to sync up model parameters across clusters. Link to Research Paper: https://lnkd.in/eiCshEwd Github Repository: https://lnkd.in/eX9GZYBb

We are happy to share that we’ve published a new research paper! 🧠🐦⬛🪺Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices Check the Cornell University Machine Learning Research at arXiv arXiv pre-print for Ravnest, our new decentralized asynchronous training algorithm. Link to Paper: https://lnkd.in/eiCshEwd We request everyone to have a look at this research paper and give us feedback on any improvements for future iterations. The implementation is underway and the updates can be found on the following GitHub Repository: https://lnkd.in/eX9GZYBb Please note that we are not inviting contributions immediately as we are working on the experimental and comparative analysis of our proposed approach and thereby making regular drastic updates to this repository.