Vivek Kushwaha

The more I work with #Elixir the more I come to appreciate the language and its #community. We at DAT AI Systems Pvt Ltd did a major migration from #Kafka to #AMQP without any downtime or client impact 💪. #Queues are the unsung heroes of #reliable, #scalable, and #distributed systems. With the AMQP protocol, an open standard widely adopted across industries, you can easily integrate #messaging into your stack without worrying about vendor lock-in. Why should you use queues in your architecture? - Decouple services for better scalability and resilience - Handle #asynchronous workloads efficiently - Ensure message #reliability and guaranteed delivery - Benefit from strong AMQP support across cloud and on-prem environments As businesses grow, embracing #open standards like AMQP ensures #flexibility, #portability, and long-term #stability. Ready to #future-proof your system? Check out my latest blog on the benefits of queues and AMQP. dev.to: https://lnkd.in/gDvce_nb Medium: https://lnkd.in/gVp9euNu #AMQP #MessageQueues #CloudArchitecture #Scalability #OpenStandards #EnterpriseTech #DistributedSystems

16

3 Comments

The journey towards gRPC starts by understanding RPC conceptually. That's what this video by Arpit Bhayani is all about. https://lnkd.in/gaUGrGGP

1

Weekend AI: Tried Aditi-v6-Llama3, by manishiitg/open-aditi-v6-llama3. Extremely coherant chats in Hindi and it can do even Hinglish. Trained on huge synthetic data, that is open sourced! I have uploaded 5 bit GGUF quants, for us to be able to run locally, using llama.cpp. Search for bharatcoder/open-aditi-v6-llama3-GGUF in HuggingFace. https://lnkd.in/g3kvYwJs

10

🚀 Building DAT Systems: A Journey Beyond AWS ? 🚀 When I embarked on setting up DAT AI Systems Pvt Ltd cloud environment, many assumed I’d turn to Amazon Web Services (AWS) , given my history with some fantastic folks there. But, when bootstrapping a startup, managing #AWS as a one-person team can be overwhelming—being the #Architect, #Developer, #SRE, and handling #DevSecOps all at once under the shared responsibility model is a tall order. Instead, I chose to go fully #serverless. I’m still cautious about using #Lambda without fully grasping its implications 😆, so I opted for a different route. Here’s a peek into our stack: - #Compute: Running globally distributed apps in 5 regions on Fly.io . - #Database: Upstash for #Redis and #Kafka with zero cluster management; Neon for #Postgres, offering bottomless storage with branching ,read replicas and #autoscaling with compute suspend. - #Storage & #CDN: Backblaze for #S3 storage paired with bunny.net #CDN for global content delivery, with no egress from #B2. - #Frontend: Built on Retool for low-code development with instant 1-click updates. - CI/CD: CircleCI integrated with GitHub for seamless #CI/#CD. - #Observability: AppSignal for #APM, Sentry for error tracking, and Grafana Labs for #monitoring and #logging. All of this wouldn’t have been possible if I had to wrestle with the stack we built our services on— #Elixir made it happen. Shoutout to the developer tools that made the journey smoother: #Helix, #nerdctl, #lazygit, #direnv, #git-secrets, and #hopscotch, to name a few. So, did I skip AWS? Not exactly. Most of these amazing services run on AWS or leverage its infrastructure. They just gave me the power of AWS while handling the responsibility. Plus, their #startup-friendly plans allow us to dream big. Here’s our current architecture—4 months in and we’ve scaled from 100 messages/month to 150,000 messages/month. Our 6-month goal? 10 million messages/month or 100,000 users, whichever comes first 😉. Reach out to Tasawar Ali to know more. #Serverless #StartupLife #CloudComputing #DevOps #Elixir #Architecture #TechJourney #AWS

34

3 Comments

My first ever article is now published on the Google Cloud Community Blogs, and I couldn’t be happier! 💫 We worked on splitting Looker instances, a case for multiple production instances, which was both innovative and challenging. This approach significantly improved the performance, which inspired us to document it as an article and share it with the broader Looker community. 🚀 A huge shoutout to my ex-team at Cimpress and special thanks to Maryia Borukhava for her invaluable collaboration and support in writing this article. 🙂 Check it out here: https://lnkd.in/dVdmt3Yf #Looker #Google #Article

41

16 Comments

One of the key capabilities that sets Memgraph apart from the competition is our deep path traversal algorithms. Come read our latest blog explaining all about it: https://lnkd.in/gBtWPJuZ

27

2 Comments

It's fantastic when the team leverages platform goodness for our customers! I love how I'm able to follow along the IPL match even when I'm on the go! As with most things we do, we had to scale what came out of the box to make this work for our customer base. Read on for practical tips on how we got there.

200

2 Comments

BharatPe’s move to Coralogix was fast, efficient, and impactful. 🚀 Now handling over 40TB of data daily, Coralogix provides BharatPe with top-tier log management, metrics, and traces, along with tools like the TCO Optimizer and #Kubernetes Dashboard for a comprehensive #observability solution. Ready to see how observability can fuel your business growth? Check out BharatPe’s journey!

5

1 Comment

Web is one of the key player in PhonePe ecosystem. Security is very core to the web development practices we follow at PhonePe. You can learn more about our experience of implementing DPoP at scale through Shikhar Kapoor from his recent talk at JSConf Budapest 2024.

20

Jyoti Bansal recently spoke with thenewstack to discuss developer toil and how platform engineering offers a "golden path" to boost developer productivity and shorten deployment times. For the full discussion, check out the article: https://lnkd.in/gAQeZ9BC #DeveloperExperience #CEO #SoftwareEngineering #DevOps #PlatformEngineering

1

Ok let's migrate a transactional datastore with 2TB volume and serving 200k RPS and zero downtime So my team and I at Helpshift were tasked with this challenge. Migration is the bread and butter of any backend core infra / platform team. So we had done a fair share of migrations so far including a AWS region to region migration. But that's a story for another day. Migrating primary transactional data from MongoDB to Yugabyte was one of the biggest undertaking. We faced several challenges right from choosing the correct dual read/write strategy, connection handling and data consistency. We had spent a lot of time creating multi phase plan for this migration. It's fulfilling to see the execution happening at steady pace on this multi-year project. Please check out this insightful talk by Somya Maithani and Shivang Raina. It is filled with valuable insights about migrations in general. Hat tip to Vineet Naik for all the awesome work he had put in for connection handling.

16

2 Comments

Unleash the power of generative AI leveraging all your data with AWS analytics for your data foundation! Don't miss "ANT302 | Data foundation in the age of generative AI" with Ippokratis Pandis, Imtiaz Sayed and Nitin Arora at #AWSreInvent to learn how AWS is scaling and evolving its data analytics for building data-driven generative AI applications. #AI #DataFoundation #GenerativeAI #AWSAnalytics #AWSData https://lnkd.in/gGMEbeD5

112

10 Comments

Avoid False causality is one of the esoteric principles of product performance. It refers to the mistaken assumption that a visible symptom is the direct cause of performance issues when the root cause is elsewhere. This phenomenon often occurs because performance problems in complex systems can cascade or manifest unexpectedly, leading testers to blame the wrong source. In load testing, the system is subjected to simulated high traffic or load to understand how it behaves under stress. When performance issues arise—like increased latency or timeouts—it’s tempting to attribute these issues to whatever system metric is visibly abnormal. However, this can lead to false causality, where testers focus on superficial symptoms without uncovering the actual underlying issue. Common Scenarios of False Causality in Load Testing are: 1. CPU Spike Misattributed to Application Code 2. High Memory Usage Blamed on Application Leaks 3. Database Latency Blamed on Network Latency 4. Disk I/O Bottleneck Blamed on High Traffic 5. Application Slowness Blamed on Backend Services 6. Request Timeouts Blamed on High Traffic When false causality occurs in load testing, the wrong problem is tackled, leading to wasted time and ineffective solutions. Even worse, addressing the wrong issue can sometimes exacerbate the real performance problem. Do you have any experience in dealing with such False causality in a complex distributed system? If yes, you may want to check this open position: https://lnkd.in/gaU2hv5T At HashiCorp, we’re growing our Operational Readiness team, and we're currently looking for a talented Performance Engineer to help us ensure our platform's scalability and reliability. In this role, you'll have the opportunity to work on cutting-edge cloud technologies, optimizing performance for large-scale systems, and driving improvements that directly impact our users' experience. You’ll collaborate with an innovative team that’s passionate about building solutions that are both robust and efficient. #Hiring #PerformanceEngineering #HashiCorpJobs #Careers #HashiCorp Shyamendra Singh Trilok Ramakrishna Shereen F

15

1 Comment

#meetup_alert - Another exciting meetup coming up!!! Join us to learn about optimising your AWS costs. We'll be sharing learnings from our mistakes so you don't have to make your own. Register using the link in the post below. Avalara India #meetup #aws #saas

40

A super clear, straight forward example of how to do cost effective RAG using LangChain for your document pipeline and Groq for compute resources. The code demonstrates how to speed up RAG and reduce costs using quantized embeddings (reducing 32 bit values to 8) Nice job Sahibpreet Singh! https://lnkd.in/gf5QMCKK

44

2 Comments

At the heart of PreProd OpnSrc, there is a vision and a mission to make foundational learning at ZERO cost Please visit the post below and the page at PreProd Opnsrc for more information PreProd Corp PreProd Emulet PreProd Consult Krishnav Dave

10

Thanks again for the opportunity. Here are the slides from my talk today - https://lnkd.in/guBJfbpR. AsyncAPI Initiative Specmatic #AsyncAPI #ContractDrivenDevelopment #GooglePubSub #Kafka

9

1 Comment