We are developing a product with a data lake where the ingestion layer processes 100,000 data points per day. The data then goes through a multi-phase cleaning and grouping mechanism to present business-ready data. Initially, we used MongoDB to develop the MVP. This allowed the customer to attract clients and pilot the business. While MongoDB has good geospatial querying capabilities, after 11 months, the total data volume reached 3 billion, making it difficult to manage and resulting in performance bottlenecks. We eventually transitioned to Elasticsearch, where storing and querying billions of data points can be performed in a matter of seconds. So we introduced solutions within the financial limits of the customer allowing them growth.

We always work under tight budgets given that customers also hard pressed to spend. so i always start with what are the mandatory features to implement - identify use of license free software e.g. opensource both for product features and development environment. Though use of open software softwares come with their own risks, maintaining them is far more cheaper than going with the licensed ones. Some of the e.g.s of the open source softwares we used for our custom solutions are - Drupal/PHP for content management, Kubernetes secrets for password storing, gitlab-ci for automating builds with VMs as runners and devops automation using a combination of Ansible, python and shell scripts.

Balancing budget constraints with system architecture needs is about smart prioritization and long-term thinking. I start by separating core functionality from bells and whistles—focusing on what the system absolutely must deliver. In one project, we faced tight finances, so we leveraged open-source frameworks that allowed us to scale without hefty upfront costs. I also look at the bigger picture: investing in foundational architecture now can save massive costs down the road. For example, implementing a scalable cloud solution may seem pricier upfront, but it avoids costly migrations or downtime later. The trick is building with both today’s and tomorrow’s needs in mind.

Need to stretch in that case, make provisions to work overtime. Focus on what’s important and make sure you get pizza to your employees. But really you will have to compensate the for the hard work. If success is your goal you can’t compromise on quality, and you definitely have to meet the deadlines. Hope this helps.

First, recognize that you will never reach the "perfect" balance; you need to find "good enough" and move on. Budget constraints now suggest more surprise budget constraints later. Bias the architecture towards flexibility and robustness towards budget constraints in the relevant phases (Development, production, operations, life-extension, …). Architectures that can bind late capabilities and capacity are valuable here. If that is unaffordable, consider a target system that can be discarded and replaced later. If there are system architecture "needs" you cannot afford, is the system still viable? Be willing to say "This is why this system cannot succeed within the available budget," and to stop spending money that will never be recouped.

The first step is understanding the project requirements and goals. Identify and prioritize the essential requirements of your system architecture, focusing on your architectural drivers. Adopt an iterative approach to development, starting with a minimal viable architecture that meets core requirements within budget. Be prepared to make bold decisions when necessary, such as opting for quick iteration loops instead of striving for a perfect solution from the outset. Remember, the best architecture is the one that best suits the specific context of your project.

Balancing budget and needs is achievable with a strategic approach in the upstream and execution in the downstream. Active finops governance from the start, along with a framework like TOGAF, ensures transparency and balance between budget and needs, focusing on critical requirements. Another point to consider is using an appropriate architecture for each need. Each has advantages and disadvantages in terms of cost, scalability, and complexity. Since complexity equates to cost! Accidental complexity (added to the project) is where balance can be achieved. As for essential complexity (inherent to the business), little can be done. So focus on the appropriate choice that best fits your budget and project needs.

Aqui la teoria del MVP creo que va muy bien. Al lanzar un MVP para validar, solo lanzar lo necesario, nada superfluo. Cuando tienes constraints en tu presuesto, manejar lo escencial, es crucial, pero sumaria el buscar colaboracion con las nubes o empresas tech que puedan ayudar, AWS, GCP, Azure, incluso MongoDb estan dando creditos que puedes ayudarte a alivianar los costos y asi seguir creciendo.

Encontrar o equilíbrio entre restrições orçamentárias e necessidades da arquitetura do sistema é um desafio constante para profissionais de TI. A abordagem de avaliar recursos essenciais, explorar soluções modulares e considerar a economia de longo prazo é fundamental para tomar decisões estratégicas e garantir o sucesso do projeto, mesmo com recursos limitados. Além das estratégias mencionadas, gostaria de acrescentar algumas outras que considero importantes: - Comunicação transparente com as partes interessadas - Negociação e busca por alternativas - Monitoramento e avaliação contínua