Ziad Osman’s Post

One common use case for the cloud now is for hosting data lakes. To save on storage costs you can store your data in a compressed format but in many cases it will be much more convenient to have data that is ready to be used right away and storage costs are quite small - especially when you're not dealing with millions of files and TB's of size. Using an event-driven serverless approach to handle this seems like it would be an ideal fit in many cases. The example below from Darren Roback shows how you can use events from S3 to trigger AWS Lambda code to decompress archive files as they are uploaded to S3. https://lnkd.in/e856MZ6z

One common use case for the cloud now is for hosting data lakes. To save on storage costs you can store your data in a compressed format but in many cases it will be much more convenient to have data that is ready to be used right away and storage costs are quite small - especially when you're not dealing with millions of files and TB's of size. Using an event-driven serverless approach to handle this seems like it would be an ideal fit in many cases. The example below from Darren Roback shows how you can use events from S3 to trigger AWS Lambda code to decompress archive files as they are uploaded ot S3. https://lnkd.in/e856MZ6z

One common use case for the cloud now is for hosting data lakes. To save on storage costs you can store your data in a compressed format but in many cases it will be much more convenient to have data that is ready to be used right away and storage costs are quite small - especially when you're not dealing with millions of files and TB's of size. Using an event-driven serverless approach to handle this seems like it would be an ideal fit in many cases. The example below from Darren Roback shows how you can use events from S3 to trigger AWS Lambda code to decompress archive files as they are uploaded ot S3. https://lnkd.in/e856MZ6z

One common use case for the cloud now is for hosting data lakes. To save on storage costs you can store your data in a compressed format but in many cases it will be much more convenient to have data that is ready to be used right away and storage costs are quite small - especially when you're not dealing with millions of files and TB's of size. Using an event-driven serverless approach to handle this seems like it would be an ideal fit in many cases. The example below from Darren Roback shows how you can use events from S3 to trigger AWS Lambda code to decompress archive files as they are uploaded to S3. https://lnkd.in/e856MZ6z

Last week a client asked me why their DynamoDB database wasn't working the way they expected. I asked to see the data model and tables. He was treating the table as an SQL database table. The primary key was made up of only a partition key. There was no sort key. So naturally his team had to filter items using the FilterExpression. Most Dynamo users don't realize this, but FilterExpressions still scan every item, not just the filtered ones. So the latency and price remain the same as without the filter. Bottom line: don't rely on FilterExpressions to efficiently filter data. Use what I call intelligent sort key design. There are a few variations. Depending on your use case. The most effective design technique is hierarchical sort key modelling. If you use DynamoDB, you need to learn this. You can get started by booking a free 15 min consultation with me. If you're using DynamoDB and not using sort key modelling, these 15 minutes can change your (IT) life. --- 🔔 Follow Uriel Bitton for more insights on building with the cloud. 📬 Subscribe to my newsletter for weekly posts on saving money in the cloud (in bio) #dynamodb #database #datamodelling #databasedesign #aws #amazon #cloudcomputing

In my latest blog post, I provide a step-by-step guide on how to use AWS DataSync to transfer data from an EFS file system to an S3 bucket. https://lnkd.in/ehifuP8A

In this post, we show you how to maintain a continuously updating export of your table data by doing a bootstrap full export followed by an ongoing series of incremental exports. DynamoDB Continuous Incremental Exports (DCIE, the name of this solution, available as open source on GitHub) enables use cases which require a DynamoDB export of data with the data kept fresh. #AWS #AmazonWebServices #AWSBlogs #Cloud #CloudComputing #Serverless #DynamoDB

Preventing AWS Lambda from Recursively Calling Itself: Best Practices https://lnkd.in/dhwQQq2x Amazon Web Services (AWS) #aws #awslambda #businesscompassllc

📚 In this series, we'll dive deep into various white papers to explore the challenges faced and the innovative solutions developed. First up: AWS DynamoDB, a highly durable and available key-value store. We'll uncover the problems encountered and the techniques used to solve them, shedding light on how DynamoDB achieves its impressive scalability and reliability. #AWS #DynamoDB

In this post, we show you how to maintain a continuously updating export of your table data by doing a bootstrap full export followed by an ongoing series of incremental exports. DynamoDB Continuous Incremental Exports (DCIE, the name of this solution, available as open source on GitHub) enables use cases which require a DynamoDB export of data with the data kept fresh. #AWS #AmazonWebServices #AWSBlogs #Cloud #CloudComputing #Serverless #DynamoDB