Carlos Augusto

Industry research shows 1 in 5 applications contain runtime exposure, with 20% of apps having high, critical, or apocalyptic issues during the execution stage. By automating #security for fast-moving, dynamic apps like those that run in #containers, JFrog Runtime addresses the unique visibility and insight needs of cloud-native environments: https://jfrog.co/3THB7Lp

95

A really nice read from Sai Maddali on the benefits of Tiered Storage with Amazon MSK. https://lnkd.in/gGNJ5nyt #ApacheKafka #Kafka #AWS #MSK #tieredstorage #streaming #storage

55

2 Comments

🚀 Exploring AWS Choices? Check out our latest post on when to use Amazon CloudFront over S3 for optimal performance and cost efficiency. Perfect for executives seeking strategic insights without the technical overload. https://lnkd.in/gcp73UCj #AWS #CloudFront #S3 #DataManagement #TechLeadership

1

From big consumer apps to B2B leaders and startups, Chronosphere and Google Cloud's partnership has helped customers of all shapes & sizes. Hear our CEO Martin Mao alongside Google Cloud’s Scott Barneson dive into what we’re hearing from customers, and how we’re helping #cloudnative teams. #observability

41

We are excited to announce the launch of Ursa engine, our Kafka API-compatible data streaming engine that is built on top of Lakehouse. Ursa is not something entirely new; it is built on our many years of experience building and operating #ApachePulsar and #ApacheKafka. To learn more about the #Ursa engine, I recommend our #PulsarSummit keynote presentation and announcement blog post. Here, I’d like to delve deeper into some of its features: 1. Making Kafka API a first-class citizen: In #ApachePulsar, we developed a unified messaging model combining queuing and streaming semantics into one protocol, aimed at enhancing data sharing between business applications. Although powerful and increasingly adopted by enterprises and unicorns, the requirement for application rewrites is impractical in today’s economy. Instead of building data gravity around a protocol, we open up access to the robust stream storage layer by extending support to the Kafka protocol, transitioning from a single protocol to a multi-protocol platform, facilitating greater flexibility and integration. 2. Achieve Stream-Table duality via Lakehouse Storage: Data streams commonly end up in data lakehouses, where integrating and maintaining them can be complex and resource-intensive. Ursa simplifies this by storing data streams as lakehouse tables in object storage. This integration not only streamlines operations but also allows seamless access to operational data for analytical processes. Thanks to its multi-modal storage engine which fosters data sharing across teams and departments. 3. Object storage as the primary storage solution: To better align with cloud economics, particularly for large workloads, Ursa makes ZooKeeper and BookKeeper optional and utilizes a cost-optimized write-ahead log based solely on object storage with leaderless and stateless brokers. This architecture leverages object storage for both data replication and storage, minimizing inter-AZ data transfers and reducing costs for latency-relaxed, high-volume workloads. By integrating these features at the multi-tenant level rather than at isolated cluster levels, Ursa eliminates costly data duplication and movement across clusters, allowing teams to map directly to tenants and be able to share data across teams and departments more efficiently. We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities. Check out our announcement blog post about Ursa: https://lnkd.in/ghEKTYE6 #ApacheKafka #Lakehouse #DataStreaming

100

1 Comment

Scale at ease, Lower cost & Future proof

2

Enterprise AI experts at AWS re:Invent got a deep dive into Domino Volumes for NetApp ONTAP (DVNO) during Josh Mineroff, PhD's talk. 💡 Key Learnings: • Rapid Data Access: Leverage NetApp ONTAP data across hybrid environments without DevOps overhead. • Efficiency Gains: Reduce processing costs and times by up to 50%. • One-Click Simplicity: Seamlessly create, share, and manage storage volumes for AI workflows. Missed it? Visit Domino Booth #1774 today and explore how DVNO powers scalable, governed AI workflows at enterprise scale. https://lnkd.in/gHvd_fvc #AWSreInvent #AI #HybridCloud #DataScience #Innovation

21

1 Comment

If you are at AWS re:invent next week. Feel free to drop by PagerDuty booth to learn how we use Pagerduty with snowflake & AWS @ Snowflake. Gaurav Singodia & I would be presenting this session #pagerduty #sre #aws #snowflake #cortex

102

1 Comment

JFrog and GitHub have strengthened their alliance with a unified dashboard, streamlining vulnerability tracking from source code to deployed binaries. Now, DevOps teams can seamlessly navigate between GitHub Actions Workflows and JFrog Artifactory repositories, with visibility into security scans, permissions, and identities. This bi-directional integration makes it easier to prioritize vulnerabilities and ensure software is secure at every step. https://bit.ly/3TnvjGo #swampUP2024 #DevSecOps

2

Folks I've spoken to about Amazon have likely heard me talk about the importance of the PR/FAQ process, a.k.a. "the working backwards process". This blog posts gives a very nice example of a PR/FAQ. https://lnkd.in/edTeuVRQ. Note that the blog doesn't capture the presumably 100s of reviews/discussions/rewrites that this doc represents, which is a part of the real value. When the process is done it all looks so simple, but getting a large organization to make a decision to something simple and do it in a BIG way is the challenge the fun of it all.

64

3 Comments

I love that AWS posts their prime-day compute metrics because you can extrapolate some interesting numbers about what certain capabilities cost. This one caught my eye. 6,311 Aurora DB instances processed 376 billion transactions (storing almost 3 petabytes) during prime day. That's ~59 million transactions per instance. Conservatively assuming most activity happened during 10 hours of prime day, that's 10.4 million total transactions per second and ~1,654 per second per instance. Average size per transaction based on the storage number is ~8,000 bytes (which seems quite large) and that's ~1/2 a TB in storage per instance. (my apologies if I mess up any of the math or if I use any incorrect assumptions - consider this back of napkin calculation) This Aurora DB cluster provides fast, selective query O(Log N) (I assume using Btree indexes) and ACID transactions. Assuming AWS is really good at pricing their products, we can probably assume that the cost to run this cluster will be similar to the cost to run other sharded clusters of relational DBs large enough to organize 3 petabytes of data with transactional consistency for fast query. All we need to get a rough order of magnitude cost is to guess the instance size and then plug it into AWS' pricing calculator. The cost is going to vary quite a bit by instance type but will be hard to know for sure what instance type they're using. db.r6g.xlarge instances is fairly modest and likely a decent guess (4 VCPU and 32 GB of RAM) - an estimated on-demand cost for 6,311 instances came out to be roughly $2,391,048.57 per month or ~$28 million a year plus storage and IO charges. For us here at Craxel, this gives us a pretty good idea of the cost of indexing 3 petabytes of data using BTrees and keeping the data available for fast query on a sharded cluster of running servers. Hit us up here at Craxel if you would like to hear how our O(1) multi-dimensional indexing algorithm changes the cost structure for organizing petabytes of data for fast query. Thanks AWS for sharing!!!

25

1 Comment

One of the most common questions we hear is “what is the best way to migrate from Apache Kafka to #Confluent?” There is no one answer to this question – context is crucial to guarantee your data management platform is migrated over quickly and efficiently and in the best way to suit an organization's needs.   Fortunately, we’ve created a handbook that can guide you through the best practices for the initial stages and provide you with the resources to maintain your platform post-migration. Take a look: https://lnkd.in/gVsDcZ_3   #ApacheKafka #datastreaming #datamanagement

54

This is incredible news for Amazon Kinesis Data Streams On-Demand users! KDS On-Demand now allows writing up to 10 GB/s, a significant 5x enhancement! The most remarkable aspect is that there is no action required to leverage this feature. #Kinesis #data #BigData #streamingdata #Datastreaming #ondemand #AWS Link: https://lnkd.in/gdm_FXrU

47

2 Comments

Don't let cloud confusion rain on your parade. Simply put, scalability is your infrastructure's ability to handle growing demand. ​ ​ Bonus Tip: Scale UP with vertical scaling for predictable workloads or scale OUT horizontally to handle spikes.​ ​ #LSDOpen #CloudExperts #CloudConsultants #CloudManagedServices​ #AWSAdvancedPartner #CloudSolutions #AWSCloud #CloudConfusion #CloudScalability #CloudElasticity

4

🚀 The 2024 State of Production Kubernetes Report from Spectro Cloud offers critical insights into how enterprises are leveraging K8s today. 📊 The report highlights key trends—Kubernetes adoption is accelerating, but challenges like complexity, cost pressures, and market consolidation are still top concerns for many organizations. At Cuemby, we simplify Kubernetes at scale, helping businesses overcome these challenges with tailored Kubernetes stacks and tools that meet the specific needs of developers. We ensure enterprise-grade security and governance, so you can focus on innovation, not infrastructure. 🤝 Need customized Kubernetes stacks for your enterprise? Connect with our experts: https://lnkd.in/eGRy94et  #cloudstrategy #CloudAutomation #cloudoperations #securecloud

9

1 Comment

🔍 Critical ML Vulnerabilities Uncovered by JFrog! JFrog has identified 22 vulnerabilities in open-source ML projects, exposing risks like remote model hijacking and unauthorized privilege escalation. As AI adoption rises, the need for stronger security has never been clearer. 🔗Learn more: https://lnkd.in/ghvUxY9Q #AI #Cybersecurity #ISBNews

16

AWS re:Invent is happening this week! Check out these live updates for the latest news including all things generative AI, new service announcements, tech demos, and more. #AWS #reinvent #amazon

1