Blog category
Data engineering guides on building AWS data lakes, streaming pipelines with Kinesis, serverless ETL with Glue and Athena, and BI dashboards with QuickSight. Patterns for moving from ad-hoc SQL to production-grade analytics infrastructure.
A lab query still returned a customer the source deleted. Zero-ETL can land DynamoDB and listed SaaS sources in S3 Tables. A self-managed database cannot. Firehose database CDC was removed on 24 September 2025.
Copying curated Parquet into every consumer account is how data platforms drown in storage cost and permission sprawl. On Feb 11, 2026 AWS shipped Lake Formation cross-account version 5 — wildcard RAM shares for hundreds of thousands of tables. A composite 12-account platform cut duplicate curated copies from 3 to 0 and dropped cross-account access tickets from ~11/month to ~3 by standardizing LF-Tags + resource links.
Amazon QuickSight can replace expensive BI tools or become a costly mistake — depending on how you use it. Here is the production guide that covers SPICE, multi-tenancy, cost control, and the cases where QuickSight is the wrong choice.
Virtual data modeling on AWS: Athena/Redshift semantic views without copies. July 2026 — when VDM works, when it burns Athena dollars.
Kinesis Data Streams and Amazon MSK both handle real-time streaming on AWS, but they serve different architectures. Here is how to choose between them for your workload.
Serverless RPUs vs Provisioned RA3 — still not automatic. July 2026 refresh: base 4–1024 RPUs, Max RPU-hours, AI-driven scaling, when RI wins.
Amazon OpenSearch Service powers search, log analytics, and time-series workloads on AWS. Here are the architecture patterns and cost levers that matter most in production.
Kinesis Data Streams combined with Lambda and DynamoDB is the simplest path to a real-time data pipeline on AWS. Here is the complete architecture, code patterns, and operational guidance.