Arnab Shil’s Post

Using parameters in Azure Data Factory does help me save time when debugging. Check out my new blog post to understand how to use parameters. https://lnkd.in/gkCX-GyF #Azure #DataFactory #Synapse #ETL #ELT #DataPipeline

🔍 Discover how to efficiently ingest and query data from #AzureCosmosDB using Azure Data Explorer. This guide provides all the steps you need. Dive in: https://lnkd.in/epj-mVgV #Azure

Next three days I am going to post videos from "Microsoft Developer's" YouTube channel covering Microsoft Azure Cosmos DB. Azure Cosmos DB is the preferred data storage solution for their diverse operational needs. In this video, Alessio discusses how Azure Cosmos DB supports the storage and management of critical data across multiple domains, link below to a video. #Microsoft #Azure #AzureCloud #Cloud #CloudComputing #AzureCosmosDB #Database #CloudDatabase https://lnkd.in/e7ztXDmt

🚀 DAY-7 of My Azure Learning Series! 🚀 🌐 Copy Data Activity in Azure Data Factory (ADF): A Complete Walkthrough 💻 🔍 Learn how to easily move your data between various sources and destinations using the Copy Data Activity in Azure Data Factory. Whether you're working with cloud storage or on-premise, this guide walks you through every step – from creating your ADF instance to configuring pipelines and monitoring performance. ✨ Key Highlights:  ✅ Step-by-step process 🛠️ ✅ Source & destination setup 🌍  ✅ Data mapping 🔄  ✅ Advanced settings & performance tuning ⚙️ 🔐Your suggestions and feedback are always welcome—let's learn together! 🙌 #DataEngineering #MicrosoftAzure #CloudComputing #AzureDataFactory #BigData #TechInsights #Tech #ETL

🔍 Discover how to efficiently ingest and query data from #AzureCosmosDB using Azure Data Explorer. This guide provides all the steps you need. Dive in: https://lnkd.in/egiuwyns #Azure

Azure Fabric! 📊✨ Ingesting data with a pipeline in Microsoft Fabric automates the process of bringing data from various sources into a unified storage solution, such as a lakehouse in OneLake. These pipelines are designed to handle large-scale data ingestion efficiently, allowing connections to databases, cloud storage, and on-premises systems. In a recent project, I used Synapse Data Engineering to create a workspace and set up a lakehouse. I initiated a new pipeline, connecting to an API for source data, and defining the data source connection and destination. I also created two notebooks: one to set parameters and another with PySpark code to read and process the data. I optimized the pipeline to delete old files, copy new data, and execute the notebooks efficiently. Running the pipeline ensured smooth and effective data processing and management. If anyone has tips or resources, feel free to share! #DataEngineering #MicrosoftFabric #AzureSynapse #DataIngestion #RealTimeData #ApacheSpark #DeltaLake #PowerBI #TechInnovation #Azure

📊 Enhance your data insights by ingesting data from #AzureCosmosDB into Azure Data Explorer. Follow this comprehensive tutorial to master the process. Read more: https://lnkd.in/egiuwyns #Azure

This blog has been a tremendous help as natively accessing UC data from Azure Data Factory isnt as straightforward as we all should hope. Open standards like ODBC ftw.

🚀 Ingest data from #AzureCosmosDB using Azure Data Explorer! Learn how to seamlessly query and analyze your data with this step-by-step guide. Get started here: https://lnkd.in/egiuwyns #Azure #