MANAGING TERRAFORM STATE WITH GOOGLE CLOUD STORAGE OBJECT VERSIONING
Introduction Terraform is an extremely useful tool for managing cloud resources as it maintains a state and allows changes to be made when comparing against that state.
AIRFLOW & GOOGLE BATCH WITH PYTHONVIRTUALENVOPERATOR
Overview Effective use of Airflow (and the python ecosystem in general) is plagued with dependency management problems.
DEPLOYING DAGS USING GIT DIFF
Deployment patterns for Airflow DAGs in multi-user Composer environments Thanks to Casey Zakroff for the idea and Koushik Ghosh for the development support.
SECURE SIGNED URLS ON GOOGLE CLOUD STORAGE
Ad-hoc Secure Data Upload to Cloud Storage Thanks to Paul Williams and Thinh Ha for their ideas and advice on Go
RUNNING NEXTFLOW ON GOOGLE BATCH
Intro Google Batch is a new feature from Google Cloud that is effectively a batch system as a service.
NEW YEAR, NEW WEBSITE
New Year, New Website… I have transitioned to Hugo - Goodbye Jekyell and Mediunm - and hopefully more content will appear (as I do more work)
RECOMMENDATIONS
Books, Blogs, and other materials I was thinking the other day about useful sources of information for modern ML Engineering and Architecture (not data science/algorithm development) - and came up with the following.
ML SYSTEM DESIGN
[Originally posted at: https://medium.com/fullstackai/building-ml-driven-applications-4e83cb95c0c] Introduction This blog is part of a series on designing and building AI systems.
HELLO WORLD
Alex Shires, AI system designer I am a highly-motivated AI Architect, which means I have built a career out of designing, building and deploying products driven by machine learning algorithms and take data science-based projects from prototype to production.