Darryl Sladden

Do you need an Amazon EC2 instance with a truly stupendous amount of memory? Check out my newest blog post to learn about the new U7i high memory instances with 12 TiB to 32 TiB of DDR5 memory and 896 vCPUs, available today in four #AWS Regions. https://lnkd.in/gVgxJ-bF

772

65 Comments

Effective advertising is impossible without data, and our partnership with Integral Ad Science continues to show the critical importance of measuring and targeting high-quality, high-attention inventory in order to drive demonstrable attitudinal outcomes. Collaboration between layers of the measurement stack is extremely valuable, but it requires great sensitivity and care towards consumer privacy. Clean Rooms, used appropriately, are a powerful tool to enable collaboration while respecting privacy. As a long-time Amazon Web Services (AWS) user, we've been very pleased how easy AWS Clean Rooms has made it to get our integration with IAS off the ground. AWS has always put the needs of developers first, and that commitment shows up strong in AWS Clean Rooms' simplicity, easy of use, and tight integration with other AWS services. It's clear that clean rooms are a key part of the future of collaborative advertising, and we look forward to continuing to build on AWS.

4

1 Comment

I have just posted some detailed analysis of a deal between Confluent and WarpStream and what it means for the data-streaming business. Customers are looking for more flexiblity, a bring your own cloud (BYOC) approach—as well as cheaper pricing! As I will share later, I think this event has implications across the cloud data world, including for Snowflake and Databricks. Check out my analysis on Forbes. More details to follow on Futuriom! $CFLT

4

Redis is fueling the data revolution with Google Cloud Marketplace! ⚡ In a recent episode of The Ultimate Guide to Partnering, Ash Vijayakanthan, SVP Cloud Sales and BD Partnerships, Redis shared how they're leveraging the cloud marketplace to accelerate procurement and scale high-performance data solutions for customers. With the increasing demand for real-time data and the rise of #GenAI, high-performance caching and database solutions are more critical than ever. Their marketplace strategy is not only helping them retain customers but also expand deal sizes and accelerate closing times. 🚀 Key takeaway: Investing in the marketplace and empowering your sales teams can unlock significant business value. Don't miss out on this insightful conversation! #GoogleCloudPartners Ultimate Partner

4

In 2013, Jay Kreps introduced the concept of the log as a fundamental abstraction for real-time data streaming, paving the way for platforms like #ApacheKafka. Over the past decade, the data streaming landscape has undergone a remarkable transformation, particularly with the rise of cloud-native environments and the integration of scalable object storage solutions like Amazon S3. This evolution has given rise to platforms like #ApachePulsar, each offering innovative approaches to log storage and data streaming. In our latest blog post, we explore the evolution of log storage implementations in data streaming and categorize this architectural journey into three distinct stages: 💡 Leader-Based Architecture Leader-based architectures form the foundation of most data streaming technologies used today. Traditional message queues rely on single-node brokers with primary-backup setups for high availability (HA). Kafka was the first distributed implementation, but it remains leader-based, where a designated leader manages replication. Pulsar advanced this by decoupling compute and storage, but it still follows the leader-based model. This approach works exceptionally well for scenarios requiring ultra-low latency (e.g., milliseconds). For such use cases, sending data through a leader remains unavoidable. 💡 Leaderless Architecture The emergence of S3 as a primary storage layer has triggered a shift toward leaderless architectures. Instead of treating S3 as just a tiered storage solution, it now serves as the main storage layer. In this model, brokers become leaderless, meaning any broker can handle reads and writes. Platforms like WarpStream, Confluent Freight Cluster, and StreamNative’s Ursa Engine are adopting this approach, reflecting the growing trend toward distributed and flexible architectures. 💡 Brokerless / Headless Architecture The next frontier is headless (or brokerless) architectures, where metadata and data are entirely decoupled: Data resides in S3, while metadata is managed by a separate metadata or catalog service. With a standard specification in place, operations no longer need to flow through brokers. Vendors can simply adhere to the standard and build their solutions accordingly. This approach can enable seamless scalability and interoperability. Our Ursa Engine is heading toward this exciting future. Explore the full blog post to dive deeper into these trends and the journey from leader-based to brokerless architectures: https://lnkd.in/gCcmv3rU You can also watch our keynote presentation at Data Streaming Summit: https://lnkd.in/g6a3wHrC

61

re:invent is underway, and here are my pick of the first announcements: Amazon EKS Auto Mode: https://lnkd.in/gk9PE9Mx. There will be lots of happy customers with this one. #simplification Amazon EKS Hybrid Nodes: https://lnkd.in/gMZzFHk4. Lots of usecases here, but I'm thinking co-location of related applications that can be latency sensitive, outside of AWS, while all are modernised to containers then migrated together via Kubernetes configuration. #modernisation #easier AWS DMS Schema Conversion with GenAI: https://lnkd.in/gbhTZcjU. GenAI to help with conversion to Postgresql. I was a SQL Server person, but I am now "converted" :) #genai #postgresql I7ie = Big instance local storage, up to 120TB! : https://lnkd.in/g9y9eZF2 AWS Transfer Family Web apps: https://lnkd.in/gqS82DgN. Handy for customers wanting a simple S3 interface via web interface and integrates with AWS IAM Identity centre Quick ones: AMI governance: https://lnkd.in/gZh24k2E Declarative policies: https://lnkd.in/g3KuTeSE Amazon MemoryDB Multi-Region: https://lnkd.in/ghZG66_5 Amazon CloudWatch Database Insights: https://lnkd.in/gawMMxxt Lots of hard work here from the service teams! More to come!

31

2 Comments

The latest episode in Google Distributed Cloud's "AI anywhere" series addresses the challenges of procuring edge hardware and introduces the new customer-sourced option. Manisha Verma, Strategic Partnerships Lead for Google Distributed Cloud, discusses the challenges of procuring edge hardware and how Google Distributed Cloud is addressing these challenges with a new customer-sourced option. Key takeaways from the video: - Greater control and flexibility. The customer-sourced option empowers customers with more control over their hardware choices and procurement processes. - Pricing flexibility. Customers can leverage their existing relationships with OEMs to negotiate better pricing and terms. - CapEx advantages. Keeping hardware on their books can offer CapEx benefits for customers. - Expanded availability. The customer-sourced option is currently available in 30 countries, with plans to expand further. Watch the full video to learn more about the customer-sourced option and how Google Distributed Cloud can help you overcome edge hardware procurement challenges → https://goo.gle/4aUy5sM

1

🎉 Today marks a very special milestone in our journey with #ApachePulsar – we’re celebrating 6 years as a top-level Apache Software Foundation project! 🚀 It’s been an incredible experience watching Pulsar evolve from an idea into one of the most unique and advanced messaging and data streaming platforms out there. What started as a vision to build a system that could handle the complexity of real-time data at scale has grown into something beyond what I could have imagined. The innovation and dedication of the community, the contributors, and everyone who believed in Pulsar's potential have been the true driving force behind its success. From its unique multi-tenancy, geo-replication, and no-data-rebalance capabilities to its ability to power everything from small startups to large enterprises, Pulsar is a testament to what can be achieved through collaboration and open source. Looking back on the journey, I’m beyond proud of what we’ve accomplished, and excited about what’s next. With Pulsar 4.0, #Ursa Engine, UniConn (Universal Connectivity), and even bigger ideas on the horizon, the future is brighter than ever. 🌟 Thank you to everyone who’s been a part of this ride, and here’s to many more years of innovation and community building! 🙌 Check out the latest blog from Matteo Merli and me reflecting on this incredible milestone: https://lnkd.in/gV5W8rQy #ApachePulsar #OpenSource #DataStreaming #Messaging #Queuing #Community #Anniversary #Ursa #UniConn

105

5 Comments

Since we launched Redshift Streaming Ingestion with Kinesis Data Streams (KDS) and Managed Streaming for Apache Kafka (MSK) we have seen great adoption with thousands of customers ingesting data at very high rates with low latency directly to Redshift Managed Storage. One common request was to extend the ingestion capabilities to other Kafka sources. Today we are extending Redshift's Streaming Ingestion to support self-managed Apache Kafka on EC2 as well as partners, such as Confluent Cloud. Go and Ingest! https://lnkd.in/gpzC-JD2 #amazon #redshift #streaming #ingestion #kafka #confluent

200

1 Comment

In this podcast, Google Cloud's Oliver Parker vice president, global generative AI GTM, and Miku Jha, director, AI/ML and generative AI partner engineering, sit down with Bob Evans to talk about how businesses can safely and efficiently scale their generative AI efforts, focusing on high-value opportunities while leveraging Google Cloud’s open platform, extensive ecosystem, and advanced AI/ML capabilities to deliver meaningful results for clients. Scaling Generative AI The Big Themes: Prioritizing cost and impact: Businesses are focused on the dual priorities of cost efficiency and impactful results when implementing generative AI. With numerous potential applications for generative AI, organizations need to evaluate which projects will deliver the highest value. This approach involves collaborating with partners to identify the opportunities that offer the most substantial benefits. Open ecosystem advantage: Google Cloud’s emphasis on openness and flexibility is a key differentiator in the AI landscape. Its commitment to open systems and platforms allows for various models and technologies, including first-party and open-source offerings. This open ecosystem approach enables businesses to leverage various AI models and infrastructure options, enhancing scalability and adaptability. End-to-end offerings: Google Cloud offers a comprehensive AI stack that supports a wide range of applications, from foundational models to advanced infrastructure. This end-to-end tool facilitates the development and scaling of sophisticated AI applications. The integration of various layers, including model capabilities, hardware options, and platform services, enables partners to deliver efficient and effective AI offerings. The Big Quote: “Having great models is one thing, but having differentiated infrastructure and a platform — all those three things come together, and I think we are unique in that sense.” More from Google Cloud: Learn more about Google Cloud and AI.

2

We are pleased to announce that you can now achieve up to an aggregate 1.3 million I/O operations per second (IOPS) and 12 GB/s throughput per OCI compute instance with the OCI Block Volume Service. In addition, you can also attach up to 32 Ultra High Performance (UHP) volumes to a single OCI compute instance. This service update is particularly useful for high I/O workloads, such as OLTP, AI/ML, data warehousing and analytics, databases with large data sets and extensive complex processing. https://lnkd.in/eHErX5mC

2