Sane Branch Management of Version Control Systems for Teams

[NOTE : This is still a 'draft', please point out mistakes and suggest changes/improvements]
Some time back Nagaraj Mali asked a question about 'best practices for repository branch management' on one of our whatsapp groups. This is commom query. 'bad branching strategy' or 'no strategy about branching' are very common mistakes in project teams. These mistakes can seriously impact the teams productivity and quality. Unfortunately very few manager/scrum masters or tech leads really understand devastating impact of 'bad version control practices'. In this blog, I am attempting to explain my views and logic behind various practices that I recommend.

Mistake : Too many branches

A developer need to understand the difference between a 'branch' and a 'tag'. Many times teams use 'branch' where tag is sufficient. 'branch' is an 'active line' of development. Common practice is product teams is to support 3 previous versions (i.e major releases) for bug fixes and one new release'. For example your current release is Ver10 and you may supporting major Versions9, 8 and 7. Essentially you have 4 'lines of development'. Then you need 4 branches. Ideally you should checkout 'only' the  branch that you are working on. There is no need to checkout all branches. Since 'branches' represent 'live lines of development', once you stop supporting a particular major release, you should 'delete' the branch for that release. Typically you will have 4 to 6 branches and many/many tags.

Mistake : Creating new branch for every minor release

Assume a major release is 'version 9'. Then 'version 9.1' is developed on 'version 9'  branch ONLY. Remember 'Version 9' is the supported major release and 'live development line'. Version 9.1 is usually a 'NOT live development line'. Any code changes/bug fixes for Version 9.1 should be developed in 'Version 9' branch'

Common complaint : We spend long time in merging.

This is another symptom of bad branching and bad practices followed by team. Consider following branch scenarios.

1. Release branches for past releases
   
   Lets assume that new releases are developed in 'master' and team is supporting 2 previous release 'Version1' and 'version2'. Now team have done a bug fix in 'Version1'. Obviously customers will expect that same bug fix is available on 'patch release on Version2' and also available when new Version3 becomes available. So every day branches must be merged 'upstream'. i.e. merge Version 1 commits to Verson 2. And merge version 2 commits to 'master'. If you get any merge failure fix them. Daily changes are typically small and can be easily merged. This simple practice ensure that time required to do merges are drastically reduced and bug introduced because of merge problems are almost entirely eliminated.
   
   
2. Feature branches
   
   Many teams create 'feature' branches for every new feature or bug fix. However, they don't delete the feature branch one the feature development is over. At end of 'feature' development feature branch must be merged into 'master' and then delete the feature branch. It is best to use 'git flow' plugin/workflow for working on feature branches.
   
   Typical feature branch flow will be 
   • create feature branch from 'master' (so 'parent branch' will be 'master')
   • keep making changes and commiting in feature branch. Its ok to push feature branch.
   • every day merge 'parent branch' to 'feature' branch. This way any changes to 'parent' branch done by other developers are available to you and any conflicting changes are detected early.
   • once the feature is done, merge the 'feature branch' to 'parent branch' (usually 'master'). Since you are regularly merging the 'master' to 'feature', you will not see any conflicts when you merge 'feature' to master. Now delete the feature branch
   
   In both scenarios 'key' is the regular (preferably every day) merge from 'release branch' to master' or 'master to feature' branch.

Remember It is OK to delete a branch

Remember in version control there is nothing like 'permanent delete'. When a branch is deleted, you will not see it 'branches list'. But that does not mean all the history of such branch is also deleted. Recommended practices is to create a 'tag' from branch and then delete the branch. This will keep the history intact. It will also allow to restore a branch from the tag, if you need to do some critical bug fix in an older (now unsupported) release

Bottomline

Remember 'version control' is 'productivity tool' for the development team and NOT just a backup tool. Learn how to take advantage of the version control system and you will see signficant increase in productivity of your team.  Start by defining policy for 'branch creation/branch deletion and daily merge'.

Manual Code Review of every single change is OVERRATED

Many software companies have this policy that every change has to be manually reviewed and approved by some peer (or senior) developer either before it is commited or before it is merged in main development branch. This practice is recommended by many software luminaries. However, when implemented in a project/product, I found many practical issues with 100% manual code reviews and reached a conclusion that "manual code review of every single change is OVERRATED"

Over years, I realized that manual code review of every single change is OVERRATED. It does not appreciably reduce post shipment defects but does introduce many inefficiencies in your development process. Manual Code Review of every single change usually have -ve ROI. I find that nobody writes about bad side effects of forcing developers into 100% manual code reviews. Recently I found one article and that triggered me to write this blog post.

So here is the list of problems that I found in 100% Manual Code Review process

Problems in Manual Code Review of every single change

1.  ROI on Code Review - 
   
   Is reviewer finding kind bugs in code review such that if they remain undetected will take more efforts than the time spent in Code Review ? If yes, then ROI of Code Review is +ve else ROI of Code Review is -ve.  Most of the Code Reviews have -ve ROI.  
   Reviewer may be a good developer but he may not be really good or may not have an 'eye' for defect. Some years back I did an informal study of 'code review' comments reported in my company's code review system. Majority of the code comments were about 'coding style' or 'comments' or 'naming conventions'. Very rarely a serious bug is detected. Effectively Such code reviews have a '-ve ROI'.
2. Reviewers time and attention.
   
   Typically Reviewer is mostly one of the senior developers. He/She also has their own commitments on new feature development. Usually these commitments take priority over code reviews. Hence Code Reviews remain stuck in queue and total feature development time increases unnecessarily (elapsed time from start to feature is delivered in end user's hand).
3. Team's Most productive developers are 'reviewing' rather than 'writing new code'.
   On the other hand, if Senior developers spend majority of their time doing reviews, then 'most productive developers' are effectively not writing any code. Either case results in lower productivity for the entire team.
4. Lower developer productivity while fixing bugs reported during code reviews.
   Code review is helps only if review happens within short time writing the code (preferably within a day). However, in general actual review happens after many days. In this time, developer loses the context of code. When finally the code is reviewed and he/she has to fix something in the code, it takes longer because now developer has to 'get in the context' again. Effectively it lowers the developer's productivity again.

Over year's almost every team that I have worked with has made this mistake and effectively reduced the productivity. How do we fix this ? Whats the alternative ?

How to regain the productivity lost by "manually review every code change policy" ?

1. Institute "Zero Warnings" policy for your code.
   This is easiest and most effective way to eliminate bugs. Every new version of compiler improves in ability to detect potential problems and report them as 'warnings'.  Set your compiler to 'highest warning' level. If you are using Visual Studio, turn on 'report warnings as errors" setting. Fixing warnings is 'practically no-brainer'. Warnings have to fixed immediately. DO NOT wait till end of sprint or end of release to fix warnings.
2. Take advantage of Static Analysis Tools for your programming language.
   Every programming language now has static analysis tools (linters, tools like Klocwork, Coverity etc). This Wikipedia article will give you a list of static analysis tools. Make static analysis part of your 'CI/CD' pipeline. Make sure that static analysis bugs are fixed within day. DO NOT wait till end of sprint or end of release for fixing static analysis errors.
3. Identify your critical files and review only those manually.Stop reviewing very single line of change. Identify your critical files (use complexity analysis, analyze version history to find most frequently changed files etc).  Usually 80-20 rule applies 20% are your critical files. Mark those files in system. Any commits in these files should trigger your manual code review process. Manually review these critical files ONLY after warnings and static analysis bugs are fixed.

Development processes/practices followed in a typically software development team usually have serious inefficiencies. By using 'common sense' (rather than following common practices) it is possible to achieve an 'order of magnitude' (2x to 10x) improvement in the team's productivity and delivery quality. It DOES NOT require a change in methodology (like move to Agile) or any new expensive tools. It requires some disciple and taking advantage of many low hanging fruits. The results are usually visible in about 3 months. 

[Shameless Plug]If you want to know how to improve your team's productivity by 2x to 10x, I am available for 'consulting/coaching'.

References

1. Code Reviews Do Not Find Bugs - How the Current Code Review Best Practice Slows Us Down
2.  

Things you should do after joining your first job

This blog post is for my friends who recently graduated and now joined their first job. 

You have now joined a company and started working. Your responsibilities have now changed. You are now an earning member of your family. The habits that you develop in this first year will determine your future (in your career, in your earnings, etc). So what are the first things you need to do ?? 

Given below is a partial list of things that you need to do immediately or start doing regularly.

1. Buy a medical insurance. 
   You will already be covered in your company's medical insurance or you may be covered in your parent's medical insurance. If you are covered in parents medical insurance, that cover may now be over.  Also it is unlikely that you will stay in the same job. If you change that job, start freelancing, may be between the jobs then you may NOT have medical insurance cover during that period. As you become older, it will be more difficult to get medical insurance. So Buy a Medical Insurance on your own. See if you can get 'family insurance' plan. Add your parents to this insurance.
   
2. Start Investing and building your Financial Portfolio
   Don't spend all your salary on parties and malls. Learn the basics of investing in Mutual Funds, Shares and other financial 'instruments'.  Few years later when you want to buy a house, get married, go to honeymoon, do it with your own money. Don't depend on your parents.
• At bare minimum start creating Fixed or Recurring Deposits in your bank account. 
• All investment companies like ICICI Direct has short courses  that introduce you to basics of investing. Attend those courses.
• Register for SIPs (Systematic Investment Plans) for Mutual Funds/Shares. SIPs will allow you to start investing in small amounts but regularly.
  
3. Start Investing and Building your Knowledge PortfolioBuilding your knowledge portfolio as more important than building your financial portfolio.Sometimes your company will reimburse some of these expenses. Remember The money you spend on books, courses etc are NOT expenses. They are investments. Invest in learning. These investments pay off many times over.
• Buy Books. Don't depend on company or other libraries. You must create your own library of favourite books and books critical for your work. For example, if you are in a programmer you must have your copy of 'Code Complete2'
• Join online courses like Coursera, EdX, Udacity. Pay for those certificates
• Attend conferences. 
• Join Professional Associations like ACM, Computer Society of India, Stepin Forum and meet-ups
  
  
4. Start Investing in your Communication Skills and Business Etiquette
   Good communication skills are critical for your career advancement. One of my mentors gave me a critical advice at the start of my career. He said
   

   "Nitin, what you know is important. But who knows what you know is EQUALLY important"

   "Who knows what you know" gives you opportunities. For that to happen you must be able to clearly articulate your thoughts and ideas.
   
   
   • Practice to write good professional emails and documents. Don't write 'bcos, yo, etc' (i.e.typical sms/whatsapp slang) in your official mails, mails to your boss or colleagues.
   • Practice to speak fluently in your mother tongue and in English. Consciously avoid saying 'You know' or 'ya' in every sentence. 
   • Join organization like Toast Masters where you are practice your presentation skills
     
     
5. Learn to deal with A**holes
   You will find some a**holes in every organization. You have to learn to deal with these on your own. Your parents, brothers/sisters etc are not going to help you now. You will encounter 'passive aggressive bosses', 'vining co-workers', people trying to emotionally blackmail you for various reasons/agendas.
• Read books of Prof. Bob Sutton (No Asshole Rule,  The Asshole Survival Guide: How to Deal with People Who Treat You Like Dirt )
• Learn some meditation. You can do it on your own. You don't have to attend some fancy/expensive class. Checkout Quiet Kit
• Develop your own support system - Your friends, colleagues with whom you can talk and share your problems, who can help you with suggestions and intelligence.
  
6. Find a Mentor
   Your first team/Manager has a HUGE and disproportionate impact on your career. The habits you develop in your first project/team/job, stay with for life long. Make sure you develop 'Good habits'. Make sure to work in a team/manager who will help you learn and develop the good habits.
   
   Check this comment from Prashant Kale. I fully agree with this advice
   
   Do you feel you can reach to the expertise level of your manager/current mentor in 3-4 years? THEN you have 'bad role model'.  Find out someone where you must feel "To reach to level him/her, it will take me 10 years'.  When you find such person, stick with him/her. Stay continuously in touch with him. Meet him/her regularly. Get his/her advice. Discuss your problems/career choices/aspirations openly with them. 
   It is really really difficult to find such Gems. When you find one, make sure you don't ignore or lose them. (NOTE : I was fortunate that I found many such Gems in my initial years in Telco and In Geometric Ltd. It shaped my career. I still keep in touch with my mentors.)
   If you feel your current mentor/team is not adding value to you, change the team (may be even the company). 

If you have any other suggestions to add to this list, please write a comment.

[Update]  I posted this article on LinkedIn. There are many thoughtful comments and advice posted  on this article. Check the replies and suggestions on LinkedIn