Samuel Ackerman & Guy Barash 
Theory and Practice of Quality Assurance for Machine Learning Systems [PDF ebook] 
An Experiment-Driven Approach

This book is a self-contained introduction to engineering and testing machine learning (ML) systems. It systematically discusses and teaches the art of crafting and developing software systems that include and surround machine learning models. Crafting ML based systems that are business-grade is highly challenging, as it requires statistical control throughout the complete system development life cycle. To this end, the book introduces an "experiment first" approach, stressing the need to define statistical experiments from the beginning of the development life cycle and presenting methods for careful quantification of business requirements and identification of key factors that impact business requirements. Applying these methods reduces the risk of failure of an ML development project and of the resultant, deployed ML system. The presentation is complemented by numerous best practices, case studies and practical as well as theoretical exercises and their solutions, designed to facilitate understanding of the ideas, concepts and methods introduced.The goal of this book is to empower scientists, engineers, and software developers with the knowledge and skills necessary to create robust and reliable ML software.