Adopt engineering principles in your ML Cycle

As machine learning moves from a lab-based proof-of-concept to production, teams need to adapt their work methodologies and practices accordingly.

This includes adopting engineering-inspired practices, such as:

• Environment automation - any team member can spin up machines and run multiple experiments
• Experiment tracking and reproducibility - tracking metrics, artifacts, lineage etc.
• Testing automation - ensuring you can trust both the code and the models
• Well designed pipelines - helping reproduce experiments, try out new ideas, and collaborate
• Large scale experimentation - training on larger data, running more experiments concurrently

Unfortunately, engineering practices cannot be forklifted as-is into machine learning projects - they need to be heavily adapted to the machine learning lifecycle, and to the skillset and mindset of the machine learning team.

To successfully adapt these practices requires deep understanding and experience with both classical engineering and with machine learning.