Inferless Inc.
Inferless Inc. offers serverless GPU inference services with low cold-start times, scalable from single users to billions, and supports various deployment methods with a pay-per-usage billing model.
Services
Inferless Inc. offers serverless GPU inference services that are designed to handle spiky and unpredictable workloads. Their serverless GPU inference features low cold-start times, dynamic batching for increased throughput, and customizable endpoints with various settings. The platform provides automatic redeployment options, making it easy to ship updates quickly. Inferless supports deployment from popular sources such as Hugging Face, Git, Docker, or CLI. They also offer detailed call and build logs, simplifying the process of model monitoring and refinement.
Scalable and Cost-Effective Solutions
Inferless allows for scaling from a single user to billions, with a pay-per-second billing model that ensures users only pay for their actual usage. They offer both shared and dedicated GPU instances, with dynamic scaling of models to lower fixed costs. The platform includes a developer-friendly usage-based billing module and a detailed pricing model starting at $0.33/hr. Additionally, Inferless provides a free trial with 10 hours of free credits and offers $30 in free credits for startups to begin their compute journey.
Industries and Applications
Inferless supports a wide range of machine learning applications, including computer vision, natural language processing (NLP), recommendation systems, and scientific computing. The platform is designed to be highly versatile, capable of deploying both custom and pre-trained models on serverless GPUs in minutes. Users can access a comprehensive guide to the serverless GPU market, which includes analysis and benchmarking of current players, aiding them in making informed decisions.
Security and Compliance
Inferless ensures the highest levels of data security and compliance. The platform has achieved SOC 2, ISO 27001, and GDPR compliance. Inferless uses isolated execution environments and AES-256 encryption for model storage to protect user data. Regular vulnerability scans and penetration testing are conducted to maintain enterprise-level security. This robust set of security measures assures clients that their data and models are handled with utmost care.
Deployment Flexibility
Inferless offers flexible deployment options, supporting Nvidia T4, A10, and A100 GPUs for inference workloads. Users can deploy machine learning models from several frameworks, including Hugging Face, Sagemaker, Pytorch, and Tensorflow. The platform also supports a Bring Your Own Container (BYOC) approach for deploying container-based GPU instances. Additionally, Inferless provides an open-source library called COG for deploying models on their platform. This flexibility allows developers to integrate quickly into various staging environments, with separate environments for production, non-production, and development available at no additional cost.