About Sundara
Last updated
Last updated
In Sundara, AI inference request is published in the form of tasks, which are scheduled by the scheduler to the underlying miners for computing. Based on vGPU technology and virtualization technology, the miner can handle various model inferences. All you need to do is register in the model registry and provide for the miner to provision in advance, thereby implementing collaborative mining.
After the miner completes the task, the scheduler will update the reputation data based on its performance compared to the benchmark, and use it as a basis for future scheduling.
The validator comprehensively schedules based on the miner's status, benchmark and reputation, to ensure the correctness and stability of the task results. On the basis of ensuring task completion, rewards are given to miners who perform tasks based on performance.
Traditional computing networks focus on aggregating computing power by connecting computing power networks and building computing power clusters. However, they often fail to improve the utilization rate of computing power equipment. In scenarios with fluctuating call volumes or high-performance devices running low-complexity tasks for extended periods, there is a significant waste of computing power.
Sundara optimizes computing power scheduling at the task level, prioritizing the most suitable computing resources to avoid wasting computing power. Additionally, Sundara aims to reuse computing devices by allowing a single device to run multiple models, catering to the diverse needs of different users and scenarios. This approach provides more stable and cost-effective computing power resources.
Sundara further optimizes GPU utilization through vGPU (virtual GPU) and virtualization technology. Large-scale GPU devices are partitioned into vGPUs that are better suited for small model inference scenarios. This technique improves the overall utilization of computing power and increases the platform's capacity to handle computational tasks.
For large-scale training tasks, Sundara offers task slice clusters based on vGPU technology. By decomposing and combining tasks based on layers or data, Sundara meets the requirements of large-scale model training.