Data - The Land DevOps Forgot • Michael Nygard • YOW! 2023
This presentation was recorded at YOW! Perth 2023. #GOTOcon #YOW
Michael Nygard - General Manager of Data at Nubank & Author of “Release It!“ @mtnygard
RESOURCES
ABSTRACT
We’ve transformed our operational systems to microservices with stream-aligned teams that own what they build. We know that working in short cycles with fast feedback produces better value sooner, safer, and happier. But there’s a dark underbelly. Those microservice databases get pulled out into analytics processing that runs in monolithic batches that run for hours on hundreds of cores, then break with an error. Feedback takes days, and the centralized data team can’t keep up with the pace of the business or data growth. There’s another way, gaining traction. Data mesh promises to re-decentralize data processing along with ownership. It applies product thinking to internal data products.
We will talk about moving to data mesh: the promise and the pitfalls, what you can buy and what you’ll need to build, and how to get internal buy-in for the transformation. [...]
TIMECODES
00:00 Intro
02:11 Operational vs analytic
06:35 The opposite of DevOps
08:36 Even more challenges
09:36 Hope on the horizon
11:07 Is that actually a new paradigm?
11:27 Data mesh
13:22 Decentralize ownership
16:36 Good fences make good neighbors
24:20 Simple to implement
34:31 Missing parts
38:10 How to get there?
45:03 Summary
46:10 Outro
Download slides and read the full abstract here:
RECOMMENDED BOOKS
Zhamak Dehghani • Data Mesh •
Eberhard Wolff & Hanna Prinz • Service Mesh •
Piethein Strengholt • Data Management at Scale •
Martin Kleppmann • Designing Data-Intensive Applications •
Sandeep Uttamchandani • The Self-Service Data Roadmap •
#DataMesh #Data #Microservices #SoftwareArchitecture #DevOps #MicroservicesOfData #Decentralize #SLO #MichaelNygard #YOWcon
Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at
Sign up for updates and specials at
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
1 view
0
0
4 years ago 00:03:08 303
HuanZilla - Bang Data
9 years ago 00:08:29 4.6K
Google Data Center 360° Tour
5 years ago 01:16:47 13
Data Mining #1 / Задачи Data Mining [Технострим]
5 years ago 00:06:59 15
Data Luv - “Data Life” Snippet
4 years ago 00:04:05 33
Data Scientist vs Data Analyst (funny!)
8 years ago 00:00:00 180
Data & Science: космос
4 years ago 00:12:14 27
DWH, Data Lake и Data Mesh
5 years ago 00:03:09 265
Project Silica - Storing Data in Glass
4 years ago 00:15:49 139
Pure Data - Drone
11 years ago 00:03:02 608
Holograph: 3-D spatiotemporal interactive data visualization
6 years ago 00:20:05 61
074. Чем отличаются data analyst, data engineer и data scientist – Алексей Натёкин
8 years ago 00:19:52 130
Урок 7 - Spring Data JPA
5 years ago 11:08:00 33
Data Science Course | Data Science Tutorial | Intellipaat
6 years ago 00:02:57 14
Data Dust
8 years ago 00:00:00 415
Data & Science: нейронауки
8 years ago 00:18:25 88
Spring Data JPA - теория
5 years ago 00:57:11 15
PUBG Data Science Tutorial - Part 1 | PUBG Data Analysis | Data Science Training | Edureka
6 years ago 00:02:03 56
RAW DATA
5 years ago 00:04:17 19
Nemy - Data
8 years ago 00:05:36 132
Raw Data
4 years ago 00:08:09 7
Data Binning | Data Preprocessing | Machine Learning