Understanding Machine Learning and the ML Lifecycle Overview
AWS Certified Machine Learning Engineer – Associate · ML Model Development · Bởi Super Admin
Explains what distinguishes machine learning from classical programming and walks through the early stages of the ML lifecycle, from problem definition through data collection and processing.
Machine learning (ML) is a subset of AI that allows computers to learn from existing information, without being explicitly programmed, and apply that learning to perform other similar tasks. Without explicit programming, the machine learns from data and information it feeds, picking up patterns, trends, or essential features from previous data and making predictions on new data. This ability to learn without being explicitly programmed is the key difference between ML and classical programming.
What Makes Machine Learning Different
Consider a program that translates text from English to Italian. A classical programming approach would require manually coding rules and exceptions for lexicon, grammar, syntax, and vocabulary — an approach that would be extremely complex and not scalable. With ML, models can instead learn to understand context, idioms, and nuances in both languages, continuously improving their translation capabilities as more data becomes available. Real-life applications include recommendation systems: Amazon recommends books based on users' book category selections and purchase history, and Spotify recommends songs based on previously purchased songs and preferred genres, both using large amounts of data to train models and derive recommendations.
The ML Lifecycle
Every ML project follows a holistic lifecycle, starting with the question of what business problem is being solved and how ML addresses that need. This lifecycle — defining the problem, collecting and processing data, training, evaluating, deploying, deriving inference, and monitoring with feedback loops — is generally followed across all ML projects regardless of cloud provider or tools.
Figure: ML lifecycle flowchart
Define ML Problem
Defining the ML problem is the critical first step, involving a clear articulation of the business challenge to be addressed with data and ML techniques. This phase includes identifying the problem's scope, desired outcomes, feasibility of an ML solution, analysis of available data, understanding of business context, and determining performance metrics. ML problems differ from traditional programming challenges because they rely on data-driven, probabilistic models rather than deterministic algorithms, requiring continuous learning and adaptation from new data.
Collect and Process Data
The "Collect Data" phase gathers the raw data needed for training and testing, and the types of data collected should align with the specific problem — numerical data like square footage for housing price prediction, or categorical data like demographics for customer segmentation. Collected data is initially raw and unsuitable for modeling: it may contain missing or noisy values, features in different ranges, irrelevant fields, and must ultimately be split into training, validation, and test sets. The "Process Data" step addresses this through normalizing, scaling, and encoding data for consistency, while feature engineering creates or modifies features — such as binning numerical data, creating interaction terms, or extracting datetime features — to enhance predictive power.
Key takeaways
- ML learns patterns from data without explicit programming, unlike classical rule-based programming.
- The ML lifecycle includes defining the problem, collecting and processing data, training, evaluating, deploying, inferring, and monitoring.
- Defining the ML problem requires understanding scope, feasibility, business context, and performance metrics.
- Raw collected data must be cleaned, normalized, encoded, and split into training, validation, and test sets before modeling.
Thảo luận / Bình luận (0)
Chưa có thảo luận nào cho bài viết này. Hãy gửi bình luận đầu tiên nhé!
Vui lòng Đăng nhập để tham gia thảo luận.