Generating Synthetic Data at Scale w/Help of Modern Execution Technologies (PyData Tel-Aviv Dec21)
Speaker: Or Sher, Infrastructure Team Lead, Datagen
-------------------------
Datagen started creating synthetic images using on-premise consumer GPU machines which did not provide the flexibility and scalability required for larger scale operations.
We needed a scalable system that enables large-scale generation of 3D environments, a CPU intensive process, and rendering the images from within the 3D environments, a GPU intensive process.
This presentation will share our journey of building our internal K8s based, cloud agnostic system to enable us to provision and utilize thousands of GPU and CPU resources exactly and only when we need them..
We will cover aspects of reliability, performance, efficiency, cost optimization, and also:
What is synthetic data
The challenges of generating simulated data at scale serving many customers.
Architecture and coding challenges
Move fast and keep code clean
8 views
41
11
2 months ago 00:20:07 21
Walkthrough - Digital Analog Clouds | Hybrid Morphing Pads for Kontakt
2 months ago 00:05:52 1
Ciftetelli
2 months ago 00:00:10 1
Methylene Blue Benefits.
3 months ago 00:02:13 3
Movies Are Magic
3 months ago 00:03:58 1
OMNIMAR - FVCK AI (OFFICIAL VIDEO) | darkTunes Music Group
3 months ago 00:04:01 1
VoizHub AI Review 🎯 Discount 🎯 $150,000 Bonus 🎯 Voiz Hub AI Review 🎯
3 months ago 00:00:58 1
Mars 1959 Super Panavision 70 | Retro AI Movie Trailer
3 months ago 00:03:26 1
Pray
3 months ago 00:06:26 8
Spline-based Transformers
3 months ago 00:05:12 1
All Night
3 months ago 00:03:17 1
Rolf Köhler (Ai voice) - Synthetic dreams (Ai song Udio, by Romeo Li)
3 months ago 00:09:42 1
Pyramid
4 months ago 01:00:03 1
1hour of Shamanic Drums: For Energetic Breathwork & Movement (Series 3)
4 months ago 00:03:20 2
Моя любовь
4 months ago 00:05:02 1
BillLeeb- Demons
4 months ago 00:01:01 1
Global animal movements based on Movebank data (map) [Updated!]
4 months ago 00:45:19 12
Deathstars - M’era Luna 2024 - ARTE Concert
4 months ago 00:03:44 1
Synthetic Depeche “Not Us“ An AI depeche mode replica track SDM08 Delta Machine Style
5 months ago 00:15:23 1
I build 3D APPS with this SIMPLE Python Stack (Beginner’s Guide)
5 months ago 00:02:14 1
NVIDIA Accelerating the Future of AI & Humanoid Robots