Data Engineering Round-Up

A Collection of Top Data Engineering News, Articles, Presentations & Podcasts September 2022 [InfoQ] Data Engineering Round-Up Sponsored by [AgileLab] [Latest Content](#latest-content), [Top Viewed Content](#top-viewed-content), [Top News](#news), [Top Articles](#top-articles), [Top Presentations & Podcasts](#top-presentations-and-interviews) In this special newsletter we bring you up to date on all the new content and news related to Data Engineering on InfoQ. We are also maintaining a portal page for this content on InfoQ at: [(. [] Latest Content on InfoQ [AI, ML, and Data Engineering InfoQ Trends Report—August 2022]( (articles, Aug 02, 2022) [Meta Develops Dataset Pruning Technique for Scaling AI Training]( (news, Aug 16, 2022) [Debezium and Quarkus: Change Data Capture Patterns to Avoid Dual-Writes Problems]( (articles, Aug 15, 2022) [Machine Learning Systems Vulnerable to Specific Attacks]( (news, Aug 15, 2022) [Amazon's AlexaTM 20B Model Outperforms GPT-3 on NLP Benchmarks]( (news, Aug 19, 2022) [[eBook] The Practical Guide to Successful Data Mesh Implementations]( While organizations see the value and potential of Data Mesh, many struggle to overcome common obstacles that slow down time to value. This guide describes the common challenges to streamline the implementation and adoption process, helping to minimize risks and realize the value of Data Mesh. [Download now](. Sponsored content [DataMeshImplementations]( [] Top Viewed Content on InfoQ [Uber Open-Sourced Its Highly Scalable and Reliable Shuffle as a Service for Apache Spark]( (news, Aug 14, 2022) [Google Introduces Zero-ETL Approach to Analytics on Bigtable Data Using BigQuery]( (news, Aug 11, 2022) [Building Neural Networks with TensorFlow.NET]( (articles, Jul 11, 2022) [LinkedIn Open-Sourced Its Feature Store to Evangelize Productive Machine Learning]( (news, Jul 06, 2022) [Google AI Developed a Language Model to Solve Quantitative Reasoning Problems]( (news, Jul 14, 2022) [] Top News [Amazon Unveils ML-Powered Coding Assistant CodeWhisperer]( Amazon launched CodeWhisperer, an ML-Powered Coding Companion which provides code recommendations based on developers' comments in natural language and their code in the integrated development environment. The machine learning-powered service increases developer productivity. [BigScience Releases 176B Parameter AI Language Model BLOOM]( The BigScience research workshop released BigScience Large Open-science Open-access Multilingual Language Model (BLOOM), an autoregressive language model based on the GPT-3 architecture. BLOOM is trained on data from 46 natural languages and 13 programming languages and is the largest publicly available open multilingual model. [Shopify's Practical Guidelines from Running Airflow for ML and Data Workflows at Scale]( Shopify engineering shared its experience in the company's blog post on how to scale and optimize Apache Airflow for running ML and data workflows. They shared practical solutions for the challenges they faced like slow file access, insufficient control over DAG, irregular level of traffic, resource contention among workloads, and more. [[Analyst Brief] Accelerate Data-Driven Transformation with Data Mesh]( As data sources and use cases rapidly expand, it becomes harder for companies to design a data-driven transformation strategy. Data mesh addresses these issues, offering a technical solution to rationalize the organizational chaos generated by a tsunami of data and a myriad of use cases. [Download now](. Sponsored content [DataDrivenTransformation]( [Amazon Redshift Serverless Generally Available to Automatically Scale Data Warehouse]( Amazon recently announced the general availability of Redshift Serverless, an elastic option to scale data warehouse capacity. The new service allows data analysts, developers and data scientists to run and scale analytics without provisioning and managing data warehouse clusters. [A New Service from the Microsoft and Oracle Partnership: Oracle Database Service for Microsoft Azure]( Recently, Microsoft and Oracle announced the general availability (GA) of Oracle Database Service for Microsoft Azure, a new service that allows Microsoft Azure customers to provision, access, and monitor enterprise-grade Oracle Database services in Oracle Cloud Infrastructure (OCI). [] Top Articles [Streaming-First Infrastructure for Real-Time Machine Learning]( The benefits of streaming-first infrastructure for real-time ML are online prediction for fast responses and continual learning for adapting to change in data distributions in production. [article]( [Creating a Secure Distributed Database Cluster Leveraging Your Existing Database Management System]( Database Plus, a technology applicable to any database, answers Big Data challenges and eliminates switching costs and vendor lock-in. Here's how to easily create a distributed and encrypted database. [article]( [API Friction Complicates Hunting for Cloud Vulnerabilities. SQL Makes it Simple]( With fast, frictionless, uniform API access, by way of Postgres foreign data wrappers, you can skip the grunt work of wrangling APIs and focus on working with the data they return. [article]( [Business Systems Integration is About to Get a Whole Lot Easier]( A new breed of integration software is arising that syncs business data into a simplified data hub and then syncs that data to the destination system. [article]( [[Video] The Technical Keys to Data Mesh Success at Scale]( As data sources and use cases rapidly expand, it becomes harder for companies to design a data-driven transformation strategy. Data mesh addresses these issues, offering a technical solution to rationalize the organizational chaos generated by a tsunami of data and a myriad of use cases. [Watch now](. Sponsored content [TechnicalKeys]( [] Top Presentations & Podcasts [Protecting User Data via Extensions on Metadata Management Tooling]( Alyssa Ransbury overviews the current state of metadata management tooling, and details how Square implemented security on its data. [Alyssa Ransbury]( [InfoQ AI, ML and Data Engineering Trends Report 2022]( In this podcast, we discuss the latest trends that our readers should find interesting to learn about and apply in their own organizations when these trends become mainstream technologies. [podcast]( [Connect with InfoQ on Twitter]( [Connect with InfoQ on Facebook]( [Connect with InfoQ on LinkedIn]( [Connect with InfoQ on Youtube]( You have received this message because you are subscribed to the “Special Reports Newsletter”. To stop receiving this email, please click the following link: [Unsubscribe]( C4Media Inc. (InfoQ.com), 2275 Lake Shore Boulevard West, Suite #325, Toronto, Ontario, Canada, M8V 3Y3