Understanding Model Deployment Fundamentals in the ML Lifecycle
AWS Certified Machine Learning Engineer – Associate · Deployment and Orchestration of ML Workflows · Bởi Super Admin
Model deployment integrates a trained model into production so the ML app can return inferences immediately, near real time, or in batches, all while following AWS Well-Architected best practices.
Model deployment is the integration of your model and all its resources into a production environment so it can be used to generate predictions, or inferences. This is captured in the Deploy Model and Derive Inference phases of the ML lifecycle, which are essential for delivering the insights and predictions your model is capable of.
From Training to Deployment
Throughout this discussion, the term "ML app" denotes the application responsible for producing inferences in response to inference requests. The ML app receives inference requests as input, passes this data to the model, and eventually returns inference responses upon receiving an output from the model.
Figure: ML lifecycle diagram highlighting Deploy Model and Derive Inference phases
Response Timing and Payload
The response produced by the ML app after consuming the model is returned to the requestor either immediately (in real time), in near real time (with a few seconds delay), or after an undetermined period of time. In the last scenario, processing of inferences usually occurs in batches, because the requestor does not expect an immediate response. As a result, the period it takes for the ML app to return inferences depends on factors such as the batch size (how many inference requests are being batched) and the amount of data for each inference request, referred to as the payload.
Choosing a Deployment Approach
Depending on your specific use case, you may choose to leverage Amazon AI Services that use pretrained models and obtain inferences by invoking their APIs directly, or you may choose to expose your model with an endpoint provided by Amazon SageMaker AI. Either way, the deployment approach you pursue must comply with well-architected best practices: ensuring efficient development and workload management, protecting identities, data, infrastructure, and applications with robust security measures, and maintaining expected levels of reliability. It is also essential to use resources efficiently to meet system requirements, minimize costs by avoiding unnecessary expenses, and adopt sustainable practices to reduce environmental impact. Adhering to these principles of operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability enables a performant, robust, secure, and cost-effective model deployment aligned with organizational goals and environmental responsibilities.
Key takeaways
- Model deployment integrates model resources into a production environment for generating inferences.
- The ML app receives inference requests, passes data to the model, and returns responses immediately, near real time, or in batches depending on payload and batch size.
- Deployment approaches must align with the AWS Well-Architected Framework pillars, including security, reliability, cost optimization, and sustainability.
Thảo luận / Bình luận (0)
Chưa có thảo luận nào cho bài viết này. Hãy gửi bình luận đầu tiên nhé!
Vui lòng Đăng nhập để tham gia thảo luận.