ex-Transunion, Capgemini, Siemens| Application Architecture , People Management, SAAS,Cloud,AI ML, Cybersecurity. PMP GCP-Cloud architect
#bigtable #bigquery #google #cloud Bigtable - for heavy read/write requests like #fintech etc. Bigquery - for relational datasets, olap querying .. terabytes of data in seconds and petabytes in minutes. https://lnkd.in/d5TxF-WQ
Senior Data Engineer @ Scotiabank | Google Cloud Certified | Big Data Developer
#GoogleCloud #BigQuery #DataProc #CloudComputing #Pandas #ETL #ELT #DataAnalytics I tried to put some thoughts in different options of data processing in GCP cloud platform with Dataproc, BigQuery and Pandas. It is a comparative analysis one these three options based on performance and cost. There is no best or worst solution here, it all depends on your need and business requirements. Feel free to read through this short article, do put your thoughts 😊 Hope you enjoyed 😊 😊 😊 😊
Over the last year, Amazon Redshift added performance optimizations for data lake queries across rewrite, planning, scan execution and consuming AWS Glue Data Catalog statistics. This resulted in 3x faster execution of TPC-DS 3TB benchmark, with some queries speeding up by 12x. #aws #awscloud #cloud #amazonredshift #amazonsimplestorageservices3 #analytics #announcements #awsbigdata #awsglue #bestpractices #amazonredshiftspectrum #datalake #optimization #performance
Head of FinOps - Supporting, coaching and advising organisations on getting the best value from their cloud.
BigQuery SQL gets time windowing and gap filling | Google Cloud Blog https://buff.ly/3VBAfcO BigQuery SQL now includes enhanced features for time windowing and gap filling, as highlighted in the Google Cloud Blog post. Stay updated on the latest advancements in data analysis and processing with these new capabilities. Explore how BigQuery SQL can streamline your data queries and ensure accurate results even with missing data points. Dive into the details provided in the blog post to make the most of these innovative functionalities. #BigQuery #GoogleCloudBlog
Solutions & Cloud Architect | Lead Full Stack Engineer | SRE | Certified Google Cloud & Neo4j
🚀 New Article Alert! I’m excited to share my latest article where I dive deep into a comprehensive comparison of Google BigQuery and Amazon Redshift—two leading cloud data warehousing solutions. In this piece, I explore: • 🌐 Their architectural differences • ⚙️ Performance benchmarks • 📈 Scalability considerations • 💰 Pricing models • 🔐 Security features • 🤖 Integration capabilities with other services Whether you’re a data engineer, cloud architect, or tech enthusiast looking to make informed decisions about big data analytics platforms, this article offers valuable insights to help you choose the right solution for your organization. 👉 Read the full article here: https://lnkd.in/eSFG45ee I’d love to hear your thoughts or experiences with these platforms. Feel free to share in the comments below! #BigData #CloudComputing #DataWarehousing #BigQuery #Redshift #DataAnalytics #AWS #GCP #CloudArchitecture
Data Architect | Lead Data Engineer | Technical Lead BI | AI/ML Practitioner | GCP | Teradata | Scrum Master
"Say goodbye to the days of deciphering complex, nested SQL queries. BigQuery pipe syntax ushers in a new era of SQL, specifically designed with the semi-structured nature of log data in mind. BigQuery’s pipe syntax introduces an intuitive, top-down syntax that mirrors how you naturally approach data transformations." 👋 https://lnkd.in/efz8Vm-X #bigquery Google Devoteam I Google Cloud Partner #sql #pipesintax
Data Engineer | GCP PCA, ACE | Python, GCP, SQL, Airflow, ETL,DBT | BigQuery | Python Programmer
🕔 Export BigQuery Data to Google Cloud Storage (GCS) Made Easy! Handling BigQuery datasets, whether small or massive, just got a whole lot simpler. In my latest Medium blog, I cover three essential methods to export your data efficiently to GCS: 1️⃣ Small Dataset (< 1GB): Easily export your BigQuery query results to a single CSV file in GCS. Perfect for smaller datasets and quick data transfers. 2️⃣ Large Dataset (> 1GB): For large datasets, BigQuery automatically splits the export into multiple files. I also explain how to combine these files into one for seamless access in GCS. 3️⃣ Streaming via BigQuery Storage API: Dealing with very large datasets? The BigQuery Storage API streams data in manageable chunks, allowing you to write data directly into GCS without file size limits! ✨ Discover detailed scripts, examples, and best practices to streamline your workflow, no matter the dataset size! 👉 Check out the full guide here: https://lnkd.in/dMVij8tp #DataEngineering #BigQuery #GoogleCloud #GCS #CloudComputing #DataScience #ETL #BigData #MediumBlog
Amazon will discontinue Kinesis Data Analytics for SQL on January 27, 2026. They explain the decision, suggest alternatives, and provide guidance for migrating SQL queries and workloads. #aws #awscloud #cloud #amazonkinesis #analytics #kinesisdataanalytics
AWS Glue Data Catalog now generates aggregation statistics for Apache Iceberg tables in Amazon Redshift Spectrum. This improves query performance and reduces costs. #aws #awscloud #cloud #analytics #announcements #awsglue
Simplify your SQL with pipe syntax in BigQuery and Cloud Logging https://lnkd.in/dXUJSAKD interesting way to run SQL without writing SQL :-) #bigquery #googlecloud #dataanalytics
Amazon Q in Redshift allows generating SQL from natural language. This lets you focus on business insights rather than SQL syntax. #aws #awscloud #cloud #amazonredshift #analytics #generativeai