Anil Gulecha

Zerodha's 1.5+ million PDF contract notes processing time was reduced from 8 hours to 25 minutes is truly impressive Nomad was used instead of Kubernetes as an orchestrator. Zerodha always impresses with their engineering work. Here is another story of how rearchitecting helped Zerodha improve its overall performance and reduce costs significantly, driving customer and product success I will keep the story very short and leave a link to read the entire blog. Zerodha had a challenge processing millions of contract notes that had to be sent to the customers after executing the stocks, and this had to be completed in a day; sometimes, it took 7 to 8 hours. The old stack ran on a large single vertically scaled server to complete the process, which was very time-consuming and inefficient. The new stack is horizontally scalable and elastic. Worker nodes are used to complete the job independently. AWS S3 stores and processes the files by each worker node across various stages. Redis stores the state of each job and is also used as a backend broker, distributing the jobs among the workers. Each job's execution is distributed across EC2 instances, which write to shared storage S3 Nomad is used as an orchestration framework, and, along with Terraform, it's used to provision the nomad cluster, where auto-scaling groups are leveraged, each handling different groups of servers for various jobs like generating pdf, sending emails, and signing PDFs. Nomad deploys job worker programs on the designated nodes. Rundeck is used for administration management and execution After the processing is completed, the environment is destroyed with the help of a monitoring system along with Redis #zerodha #performanceengineering #systemdesign #performance #aws

1,455

93 Comments

Is'nt every backend exists to serve front end? otherwise why it would called so? So you are suggesting 2 types of back ends? 1. "new" back end to front end 2. traditional back end? Question to all ? Where is bulk of business logic ... Now ? Tracing the history from 2000's 1. VB/Oracle Desktop App : Front end 50%, Backend (DB) 50%. Stored procedures and PL/SQL code encoded business logic 2. Early Web Apps - Front end (ASP/JSP) - 20%, Middle Tier (App server) 50% and database (30%) 3. Modern Rich Internet apps, Front end (10% or less), Microservices (85%) and 5% in databases. What do you think?

2

How do you become a better programmer while being an SRE? Many SigNoz users and customers are SREs and DevOps engineers. In my conversations with them sometimes they lament the fact they don’t “code” any longer and want to sharpen their edges by writing application code. But they rarely get a chance to do this at their job. Coding is also like building a muscle. If you go months and months without writing any application code, when you do finally need to write code, suddenly you will find that you have lost the muscle. It's like not benching for a year and then trying to bench press. (Infrastructure-as-code is a different muscle entirely, and more config files than code). While strictly speaking SRE job does involve writing code - scripts, yaml, etc. IS CODE. But to those who miss writing application code, here’s some tips on getting started: 1. Pick a language you want to code in first, decide a hobby project and build it from end to end. With AI assistants like ChatGPT and CoPilot easily available now, an easy way is to chat with the LLMs to develop a plan and build the project. They can act as a sparring partner for you and also help in rubber ducking or resolve errors which may come up 2. Contribute to an open source project : One problem with doing something which is not part of your main job is to keep yourself motivated an accountable. I find open source communities to be a great way to help in this. Rather than choosing a hobby project of your own, you can just choose to work on a open source project. Start asking questions, joining community calls and interacting on Github/Slack. You will find problems in the project that need to be solved - and you can raise your hand to solve it. The advantage is you have people in the community to help you if you get stuck and if your PR gets accepted finally then you have a “greener” github profile to show for 🙂 3. Practice Leetcode everyday : For people who want something more piecemeal, leetcode could be great way to start. Agree it is not strictly application code, but it still helps you solve problems with code. And if you agree that writing code is like building a muscle, Leetcode is doing 10 pushups everyday. It is not tough to do - but if you do it everyday, you are on your way. 4. Build sample apps and demo projects for your team : This may not be applicable to all teams, but if you work in a developer focused product (like SigNoz) your team may need sample apps for their docs and demos. Raise your hand when you get such an opportunity. 5. Study books on design patterns and check code structures of open source projects: While this is not as good as “writing code”, if you really can’t get time to put keystrokes to your code editor, just start reading how other open source projects are solving problems. Studying more about design patterns and their application in real projects will help you get the juices flowing and who knows may give you enough motivation to get started 🙂

53

6 Comments

𝐋𝐞𝐯𝐞𝐫𝐚𝐠𝐢𝐧𝐠 "𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 𝐂𝐚𝐥𝐥𝐬" 𝐚𝐬 𝐚 𝐠𝐫𝐨𝐰𝐭𝐡 𝐜𝐡𝐚𝐧𝐧𝐞𝐥 𝐟𝐨𝐫 𝐝𝐞𝐯 𝐭𝐨𝐨𝐥𝐬 𝐬𝐭𝐚𝐫𝐭𝐮𝐩𝐬 Pranay Prateek sharing some gems on dev tools GTM in the latest episode of An Operator's Blog. Link to the full episode in comments!

24

1 Comment

India’s Oyo, once valued at $10 billion, is finalizing new funding at a $2.5 billion valuation. This marks a steep decline from its $10 billion valuation in 2019. The startup, which has raised over $3 billion from backers like SoftBank, Peak XV Ventures, Lightspeed, Airbnb, and Microsoft, has struggled to attract institutional investors. In recent months, it has aggressively pitched to high-net-worth individuals after withdrawing its IPO application twice in the past four years. Once a red-hot Indian startup active in markets including the U.S. and Europe, Oyo has now scaled back its international presence. #venture #downround https://lnkd.in/gE2GjYnM

5

Sunday morning chai reflections ☕️ Been thinking a lot about the shift from SaaS to "services" Do you think being an operationally efficient (e.g. high margin / profitable) recurring Services business would ever be valued higher than a low margin / high burn sometimes recurring SaaS business? The key assumption here is that the "recurring" nature of SaaS is in question, where you still have to earn your renewal every single year. Also AI eating SaaS and putting tech-enabled services at the forefront (read: agents). Curious to hear your thoughts 🧠 #SaaS #Margins #Valuations

17

2 Comments

How to get deep Observability of messaging queues with OpenTelemetry and SigNoz Live now at #Kubecon

29

2 Comments

🚀 In an effort to enhance the OpenTelemetry ecosystem, we are open sourcing Zap OTLP, an encoder and sync for Zap - a popular structured logging library for Go. 🎉 You can use this plugin to send Zap logs to any observability backend which can understand OTLP format like SigNoz. Zap OTLP is designed to enhance logging capabilities in Go applications by enabling the transmission of logs from the Zap logger to an OpenTelemetry Protocol (OTLP) endpoint. This integration allows developers to seamlessly incorporate their application logs into any tool that supports OTLP. The plugin provides a straightforward setup process where users can create two encoders: one for console output and another for OTLP. 👉 It also includes functionality to initialize an OTLP sync, which handles the actual sending of log data to the specified OTLP endpoint. The sync can be configured with options such as batch size, resource schema, and service name, allowing for customization of how logs are grouped and identified. 👉 Additionally, the plugin offers features like batching logs for efficient transmission, setting batch intervals, and the ability to include trace context in log entries. This makes it particularly useful for applications that require detailed logging with distributed tracing capabilities. The plugin also supports secure connections and can be easily configured to work with any OpenTelemetry native observability backend like SigNoz. Do have a look and let us know what you think? Feel free to start discussions in the repo PS : Github repo link in comments

118

7 Comments

The topic of "Why Kubernetes" has been a discussion point constantly. To get the ground reality, Kalpesh Bhalekar & I have had multiple interviews across Startups and Mid-Market companies who gave multiple reasons for moving to K8s: 1. High up-time requirement in markets like E-commerce, Travel & Healthcare. 2. Single powerful API to cover AWS, Azure & GCP. 3. Cost efficiency with effective resource utilization. Here are a few reasons why Kubernetes is becoming a household name across the market. If you plan to evaluate Kubernetes, just ping me or sign up for our waitlist. (Link in comments) #kubernetes #devops #developerexperience #cloudtechnology

14

1 Comment

Moving applications to https has become simpler than ever! Using letsencrypt and certbot, I was able to migrate https://ramayanquiz.com to https in less than 10 minutes. certbot makes the entire process seamless. With a single command, I was able to generate the certificate and instruct certbot to automatically configure nginx for https support. https://howhttps.works illustrates why https is crucial and how it works in an interesting way!

5

𝙂𝘼𝙍𝘽𝘼𝙂𝙀 𝘾𝙊𝙇𝙇𝙀𝘾𝙏𝙄𝙊𝙉 - 𝙇𝙞𝙠𝙚 𝙩𝙝𝙚 𝙆𝙖𝙗𝙖𝙙𝙞𝙬𝙖𝙡𝙖! 🛢️ 🧠 𝐉𝐚𝐯𝐚 𝐁𝐮𝐟𝐟𝐞𝐭 : Day 1 of #30DaysOfJava Ever called the kabadiwala (scrap dealer) to clean out old newspapers and bottles piling up in your house? Every now and then, we need to get rid of things we don’t use anymore to free up space. In Java, this cleanup process is handled by something called Garbage Collection. Just like the kabadiwala goes around looking for items you don’t need, the Garbage Collector (GC) in Java hunts for objects in memory that aren’t being used by your program anymore. Once it finds them, it clears them out to free up memory, making sure your program runs smoothly without any memory clutter. Now, imagine calling the kabadiwala too frequently; he might start interrupting your routine. Similarly, if the Garbage Collector runs too often, it can slow down your application—a phenomenon called GC overhead. On the flip side, if he comes too late, your home will start overflowing with junk, just like memory issues in Java! 📌 Key Takeaway: Java’s Garbage Collector is like your friendly neighborhood kabadiwala, keeping your memory clean. But timing is everything—clean up too often, and you’ll slow down, wait too long, and you’ll run out of space! #java #oracle

21

5 Comments

Worth giving your 2 minutes..

6

Ever driven a hatchback ( like VW Polo ) with the same engine of a Sedan ( like VW Vento ) ? Facets.cloud is launching a SaaS platform for Startups that is based on the same powerful tech that our current customers have been using for the last 3 years to manage their production environments. How is our SaaS platform (peppy hatchback) different from the existing enterprise one (trusted sedan)? 1. Quicker to get up and running with best cloud practices baked in. 2. Modeled for Developer self service to help stay lean and focused on your core domain. Since the platform will be running on the same core codebase, startups can be rest assured that not only day-two , but year-two (and beyond) solutions are battle tested and readily available. Our current customer base spans some of the leading names in D2C, Healthcare, Logistics, Gaming and Commerce space. How will our current customers benefit ? 1. Enjoy a smoother onboarding experience for new developers with remarkable improvement in usability and aesthetics.  2. Utilize current features more effectively with reduced complexity. 3. Experience increased stability and reliability as we build for speed, scale and wider audience. We are excited and eager for feedback.  Calling the “Early Adopters” and Cloud enthusiast to sign up for Beta testing here : https://lnkd.in/gWccS4vv

8

1 Comment

if you know, you know Vinod Khosla of Khosla Ventures. nerd. founder. wise. man. been there done that sv. wisdom. it's missing. let's face it. it's always been missing over time. for.ever. so it's a blessing truly to have an o.g. that keeps it for reals. yes we are a fan. much like Jensen Huang Fei-Fei Li ilya sutskever ...and other good people in sv. they are the ones who make this place so special. indeed they do. here https://lnkd.in/exWqey9K we have sam i am that's all that i am sam (alt)man having a 1-1 with the man. this is a must watch to provide a foundational (no llm required here) mindset as to how sv is at it's best (and worse). caution. watch out for the truth bombs. vinod is (in)famous for this style and we <love> him for it. thank g-d he exists. if you meet the dbags in sf/bayarea your experience you will be quickly lonely, dazed and confused. best to listen to these wise words from vinod and from that determine your spot in the sv landscape. with an insight on the sv playbook from a master. well. need i say more. it's about the very spirit of sv that makes it so special. and this is indeed embodied in a few who are masters. deserve our attention. yeah. yeah. i know attention is all you need. although what happens if your attention is being distracted from building. being in the company of those who will join your mission ala brian chesky of Airbnb. missionaries. search for the good missionaries in sv. keep away and keep in mind of the mercenaries. they are there. pg. he's another gift (as opposed to the grifters) paul graham wrote about this too https://lnkd.in/eEYmQwqZ - being a chesky-esque missionary. these are good things to follow on the path of achieving escape velocity as an (ai) startup. oh. hollywood has entered the ranks of sv. yes it has. so there is a bit of performance art that is required. not enough that you've created something that's novel (truly, novel). also requires a bit of showmanship. audacious in fact. it's a well known sentiment that audacious ideas motivate people. sun microsystems. audacious. juniper. audacious. OpenAI. vinod as an early investor. audacious. Vinod Khosla was born audacious. so fitting he found a place in this world where many have benefitted from this sentiment. perhaps years ago a bit on the abrasive side. now with those white hairs. the elder statesman. non-diplomat. rules, the roost of that which allows for others to flourish. note if you manage to thread the needle of vinod means that it's a very very very good beginning. nothing is guaranteed of course. as my lawyer 'friends' like to say. * buyer beware. be aware. Vinod Khosla is. are you (ai)? are you?? perhaps yes. perhaps, non. < . >

Had an incredible evening co-hosting our most electrifying AI mixer yet with Hemant Mohapatra at Lightspeed India! 🙏 We had the brightest AI researchers and founders from NVIDIA Meta X LinkedIn Uber Tesla Google Google DeepMind Square Amazon, and really exciting pre-seed/seed startups! Highlights from our discussions (h/t Hemant Mohapatra for the summary!) - The nature of knowledge: Can LLMs truly create it? - Next-gen GPUs: What's on the horizon? - AGI and scaling laws: How close are we? - LLMs + Robotics: The next frontier - Founder journeys: Finding PMF - Breaking out in crowded markets: Strategies that work - Capital markets: Current trends and future predictions - Tech industry cycles: Navigating CapEx vs OpEx These quarterly mixers keep leveling up, and I'm already excited for the next one 🔥 #AI #LLMs #AGI #Startups #Founders

64

3 Comments

The total number of NRIs and PIOs stands at about ~3.3 crores, possibly the best Indians to sell any homogenous Indian product. Just the top 6 countries account for ~1.9 Crores. The minimum income/month would be around 1-1.5 Lakhs, and net-worth easily 30 Lakhs+ Should Finsire map their net worth across geographies?

39

2 Comments

🚨Need help - How would you best love to deploy your apps on cloud? (on production) 1. Will use AWS and setup on raw EC2? 🤯 2. Will setup over EKS / AKS service? React with 🫴🏼 3. Will setup over ECS / App Runner / Azure Web Apps? React with 💡 4. Will love if some PaaS platform solves everything -- deployment, scale, cheap price, logs, alerts? React ❤️ Any other way you think you deploy your apps super fast with ease? Please do comment! 🤔 I would love to hear thoughts on pro / cons of the above platforms, any other new platforms 😇

45

23 Comments

Until a year ago I had a bias, 'Kubernetes is only relevant for late-stage companies'. During our market research, we were surprised to see startups as small as 20 employees using Kubernetes from day 1. Vedant Vyawahare & I have seen multiple reasons why companies moved to Kubernetes: 1. Serving unpredictable markets like travel and e-commerce. 2. Optimizing resources and avoiding provisioning. 3. Need to leverage Cloud credits across AWS, GCP & Azure Here are a few reasons why Kubernetes is becoming a household name. If you are planning to evaluate Kubernetes, feel free to ping me or sign up for our waitlist. (Link in comments) #Kubernetes #devops #devtools

30

3 Comments

Open sourcing "Kube Watchtower"... My initial idea was to create an open-source alternative to k8s lens. v0.1.0 is marked as pre-release. Features include - 1. An overview of the entire Kubernetes cluster. 2. Live logs of the pods. 3. Quick overview of deployments, services, and config maps. 4. Real-time view/update/scale of deployments and config maps. 5. Single binary with embedded UI resources. 6. Ollama integration for fully private AI assistance. source code: https://lnkd.in/gJJ-bbuA pre-built binaries: https://lnkd.in/g-rVPNnn #kubernetes #opensource #go

168

13 Comments