Introduction Terraform is an extremely useful tool for managing cloud resources as it maintains a state and allows changes to be made when comparing against that state.
Overview Effective use of Airflow (and the python ecosystem in general) is plagued with dependency management problems.
Deployment patterns for Airflow DAGs in multi-user Composer environments Thanks to Casey Zakroff for the idea and Koushik Ghosh for the development support.
Ad-hoc Secure Data Upload to Cloud Storage Thanks to Paul Williams and Thinh Ha for their ideas and advice on Go
Intro Google Batch is a new feature from Google Cloud that is effectively a batch system as a service.
New Year, New Website… I have transitioned to Hugo - Goodbye Jekyell and Mediunm - and hopefully more content will appear (as I do more work)
Books, Blogs, and other materials I was thinking the other day about useful sources of information for modern ML Engineering and Architecture (not data science/algorithm development) - and came up with the following.
[Originally posted at: https://medium.com/fullstackai/building-ml-driven-applications-4e83cb95c0c] Introduction This blog is part of a series on designing and building AI systems.
Alex Shires, AI system designer I am a highly-motivated AI Architect, which means I have built a career out of designing, building and deploying products driven by machine learning algorithms and take data science-based projects from prototype to production.