“If you torture the data long enough, it will confess.'”
Ronald Coase
DataOPs
The goal of Data Operations (DataOps) is simple – accomplish business goals by making data actionable. This happens by visualizing the application stack, organizing data, and creating islands of shared databases that applications can use for testing before deployment. When ready, smaller data islands can be moved to production and eventually merged into existing larger ones. This process happens using DevOps resources because that is where data professionals typically lean on for enterprise-level testing, deployment, and day-to-day operations.
For DataOps to accomplish the above goals – data must be as centralized as is practical to make its’ insights available throughout the organization. This means the organization’s IT portfolio begins testing, using, and then migrating to a data-centric architecture.
A data-centric architecture visualizes disparate siloed data from multiple sources. The goal is to make that data internally available so it can be shared throughout the organization (be it between applications, departments, or even third parties).
DATA OPERATIONS (DATAOPS) KNOWLEDGE BASE
Click @ for links
OVERVIEWS
- @ DataKitchen: DataOps in Seven Steps
- @ Puppet: State of DevOps Report (2024)
- @ Gartner: Data and Analytics Essentials
- @ TechTarget: State of DataOps Report (2024)
- @ DZone: The Ultimate DevOps Tools Ecosystem
- @ Harness: The DevOps Tools Lifecycle Mesh for 2018
- @ Tamr: The DataOps Ecosystem & @ Getting DataOps Right
- @ ScienceSoft: DevOps Implementation: Where to Start
- @ DataKitchen: DataOps vs DevOps
TOOLS
- @ GitHub
- @ Git Tutorial: & @2
- @ Docker via the DOW
- @ Software Testing Help
- @ Gartner: DataOps Tools
- @ CodeAcademy: Git tutorial
- @ PluralSight: Agile/Scrum training
- @ DevOps.com: The DevOps Toolbox & @Library
- @ SCRUMstudy: Scrum Fundamentals Certification
- @ edureka!: Top 10 DevOps Tools You Must Know (2020)
- @ Software Outsourcing Directory via Source Seek, @more
- @ US DoE: Best Practices in Energy Data Collection & Tracking
- @ UK Institute of Sustainability Studies: SECR Best Practices for Accurate Data Collection
- @ IEA: Energy end-use data collection methodologies & emerging role of digital technologies
- @ KodeKloud: Docker Tutorial for Beginners – A Full DevOps Course on How to Run Applications in Containers
MICROSERVICES
- @ Martin Fowler: The Microservices Guide (2019)
- @ Microservices Architecture: lessons from Lyft, Square, and others
- @ Mulesoft: Fundamental principles of microservice design
- @ NordicAPIs: Difference Between Monolith and Microservices
- @ Opensource.com: Micorservices primer
- @ TechRepublic: Microservices Cheat Sheet
- @ Mastering Chaos – A Netflix Guide to Microservices
- @ Introduction to Microservices, Docker, and Kubernetes
TRAINING
- @ TrueDataOps
- @ DataOps Live
- @ Analytics Vidhya
- @ edX: DevOps Courses
- @ DevOps-Certification.org
- @ Udemy: DevOps Courses
- @ Whizlabs: Top 5 DevOps Certifications (2019)
- @ 451 Research: DataOps Foundations White Paper
- @ DataForm: A practical guide to DataOps best practices
- @ Codementor: DevOps tutorials & another via @ Microsoft
- @ O’Reilly eBook: Building a Unified Data Infrastructure
- @ Guru99: DevOps Tutorial> Complete Beginners Training
- @ Digitaldefynd: 8 Best DevOps Training & Certifications
- @ Microsoft: Introduction to data engineering on Azure
- @ DevOps.com, Lior Cohen: Filling the Skills Gap for Effective DevSecOps
- @ tutorialLinux: Channel & @ The Practice of Cloud System Administration, Volume 2 (review)
BOOKS
- @ DataTalk: DataOps Library
- @ Chef: Embrace DevOps eBook
- @ Dataiku: 3 Keys Data Architecture
- @ Ebook: DataOps for Dummies (2021)
- @ DevOps Book Lists on Amazon and @Google
- @ DataOps Book Lists on Amazon and @Google
- @ BLGM: Best Devops Books: The Ultimate Collection
- @ O’Reilly: eBook Getting Data Right, S Cutt et al. (2015)
- @ OpenSource.com: 13 must-read books to take your DevOps skills to the next level
MEDIA
NEWSLETTERS
RESEARCH | RESOURCES
- @ DataOps Manifesto
- @ IBM: DataOps Portal
- @ Datacamp: Resource Center
- @ O’Reilly: DataOps Resource List
- @ Semantic Arts: Resource hub page
- @ DataKitchen: DataOps 7 Steps, @Architecture, & @CookBook
- @ DevOps.com: A System Architect’s Guide to Data Operations
- @ DevOps.com: webinars > Avoiding failure; @Microservices pipeline; @AI Risk Prediction
- @ Eckerson Group: DataOps, Industrializing Data and Analytics
- @ Elder Research: Executive Guide to Leading a Data Analytics Initiative
- @ Oracle/MapR: DataOps: An Agile Methodology for Data-Driven Organizations
EDX
ENERGY DATA SCIENCE