Category Archives: ALM

The NextGen (Agile) “Tester”

As Agile gets larger acceptance across the industry, the (manual) testing community has been concerned about how it will impact them. They hear about developers doing testing, test automation and hence wonder – what role am I going to be play? Is this going to kill my job? The concern is natural given a general perception that in Agile, developers should be doing testing. This concern has been part of the resistance that some of the organizations are facing in adopting Agile thinking.

This concern is misplaced and if anything, Agile thinking only makes the Tester more central to the teams. Fact is that developers were always supposed to test their own code! It’s a different matter that this was not happening, and when it did, generally not with the right spirit. Agile makes this even more imperative – you cannot be truly Agile without developers doing testing and automating it on a continual basis. Of course, if you can do Test (First) Driven development, you would have addressed the long term challenge of “Test Coverage” (a topic for another day)!

Development and Testing: Ne’er the Twain Shall Meet!01

Coming back to the topic, let us understand why development and testing was split traditionally. We all grew in the industry being told that developers and Testers had to be separated for the following reasons:

  • Reason 1: How can the developer be trusted to tested his own code? He/she would always pass his/her code!
  • Reason 2: If the developer has understood the functionality wrong, he/she will build it wrong and he will test it wrong, even if he tests with utmost sincerity.
  • Reason 3: testing needs special skills. The ability to write boundary value test conditions, negative test cases, do test data planning, compliance to various standards (UI, navigation, etc.), complex functional test scenarios were not exactly a developer’s core competency or his area of interest.

Test Automation: Nice to Have…

Test Automation was always an optional practice – good to have but too expensive to maintain. Focus used to be largely on automating functional test cases. These would break frequently because of changes in the UI. Newer approaches like keyword based test automation addressed this problem, but it still was an expensive process. It needed a lot of effort, took long execution time, took a lot of hardware resources and needed high maintenance.

The Agile Approach

Agile thinking, with some of the associated engineering practices, makes some fundamental changes:

  • Agile thinking puts the responsibility with the team – not with the managers! In other words, we are encouraged to trust the team. Interestingly, this way of execution actually works! This killed “Reason 1”.
  • Agile thinking accepts that Requirements will never be accurately written or fully understood in one pass. It shifts focus to the developer sitting with the business or Product Owner and understanding the requirement, in a conversational process. It insists on small and early releases to the user and taking their feedback. This killed “Reason 2”.

Test Automation: With Test (First) Driven development (TDD) and with the correct implementation of the Testing Pyramid, Test Automation is faster and practical. Functional test automation is a small % of the total. Bulk of the test automation is at the unit level. It is extremely fast and has no dependence on the UI.

What Agile thinking has not changed is that testing still needs special skills (Reason 3). In fact, with the rapid evolution of technology, the demand for special testing skills has only increased over time.

Whither Manual Testing?

So, how does the role of Manual Tester change? It truly doesn’t. However, their skill profile and how they work in a team changes. Let us understand these:

  • Early Involvement: testing was generally considered to be a late in the SDLC cycle activity. While some testers would start early test case development or test data planUntitledning, the bulk of the testing team members would join much later. With Agile thinking focus on continuous delivery, testers need to get involved early in the project life cycle, understand the business and be an integral part of the team. They are expected to almost become the “next level” domain experts!
  • Understanding the business requirement: Gone are the days that someone will define Requirements in a Word document and that will be taken as the single/only version of the Requirement. Agile accepts that Requirements need to be discussed and understood in a manner of conversation with the user. Accordingly, the tester needs to participate in this requirement conversation, with the user or the product owner (whosoever is the requirement owner in the environment), understand the requirement and draft the functional test cases. Thus, the role graduates to the next level – from writing test cases based on a MS Word requirement document to actually participating in a discussion with the user, understanding the business requirement and then, writing the test cases.
  • Accepting requirement change: Agile thinking accepts requirement change as a continuous process. Hence, test case drafting isn’t a one time activity. As requirements evolve from one iteration to another, test cases keep evolving. Testers need to accept it… not crib about it! Layering test cases in one of the possible approaches to address this.
  • A different mindset: A tester’s responsibility is not just to find defects! His/her job is to improve the quality of what the team is delivering. A tester is part of the team that is delivering the scope in that iteration/value stream. A tester cannot be pointing fingers at the developer. If he/she sees that a developer is not clear about a functionality, he/she should clear it for them! A tester needs to think like the user… they are the closest to the functional user inside the team.
  • Automate: A tester cannot be manually doing regression testing as the product functionality evolves from one iteration to the next. They have to automate, automate… and automate. Automate everything! A tester will have limited value to the team if he/she is not willing to learn and excel in any of test automation frameworks.

In summary, be rest assured that a tester’s job isn’t going away! If anything, they are now a central part of the team. They should act like one! They should excel in the domain… and drive the overall work product quality.

Strengthening Execution with ALM

We all are aware of the abysmally low rates of projects being delivered on time, within budget and with the desired quality. One of the most prominent reasons is poor Execution. Project Managers often are challenged with having to satisfy multiple stakeholders – their boss(es), customer(s), team members, auditors, etc. Consider the political and matrix nature of most organizations and you have a situation where he/she is being pulled in multiple directions, trying to balance between the opposing political forces.

Most of the time therefore, gets spent in trying to manage your stakeholders, communicating to them what they want to hear in a way they want to hear (so that they really “hear” you), escalating risks and issues in a manner that escalates the point but yet does not offend people, setting expectations of how the project is “really” doing, etc. In a reasonable size project in a constrained environment, this becomes a full time job. Activities like task planning, analyzing slipping tasks, tracking risks and their probability of actually happening, focusing on product quality, handling your team – tasks that the project manager should actually have been doing – get less time and focus. Depending on where the escalation is, the project manager is fire fighting one issue after another, running from one meeting to another! Before the guy takes time to breathe, the day is over… the entire environment is a high stress environment.

The irony is that because the Project Manager is not able to do what he/she should have done in the first place, the situation actually gets worse. Everyone is working 10-12 hours but the situation spirals out of control.

ALM tools significantly help you in getting the situation under control. They help the Project Manager by creating the WBS, inheriting risks from past projects or similar projects so that you do not miss out something that others experienced, tracking the project progress, getting real accurate status based on actual time sheets, building reports for different stakeholders with s single click, give the metrics that identify the right trend from the wrong trend, documenting MOMs and Action Items and tracking them to closure… the list can go on. This is 60-70% of what the Project Manager should have been doing but gets very less attention. Even when it does get the attention, it is done in a hurry or not in the right frequency. ALM tool restores this balance by automating almost 70-80% of these tasks. The Project Manager now needs to spend only 20-30% of their time on these tasks. This leaves him/her bandwidth for doing all the tasks that need proactive thinking, looking ahead with the crystal ball and trying to forecast potential problems or points of failure, do expectation setting with all stakeholders and spending time with their team members.

People sometimes complain about the high volume of cumbersome data entry. I keep reminding them of two things – 1. You would be doing 70% of these tasks today. However, it would be spread in multiple spreadsheets and documents. 2. If you really want to get data from the system on which you can make dependable decisions, then you do need to execute the project with granular, accurate, real-time data. The final result is quite worth it!

Are you still without an ALM tool?

In my earlier Blog, I asked the question on why ALM tools have not become a project necessity? Why are they not more “visible”? While I tried to answer some of the challenges, are you one of those still wondering if YOU need one? Let me give you some real data from our own example to help you decide…

Our code base is 1.4MLOC running on 2 databases, 2 app servers on 2 OS. That means 8 stack combinations, ignoring that we run on both IE and FF. Further, because we have many customers on old versions of the product, we maintain a minimum of 3-4 maintenance branches at any point of time. So, if you fix a defect on one branch, you have to replicate that across all other branches. How many hands-on people do you believe would be required to maintain this application with a steady flow of enhancements? If you did not know our strength, I would suspect that your answer is around 40-50. We do it with an average of 15 FTEs! If one has to look for productivity examples, I cannot think of anything better than that.

We maintain our Backlogs on the tool, we rank them and estimate them and then, depending on the capacity, determine how much we can do in the next release. That just defined our Release Scope! In all my past 20 years as a delivery person, that process has taken 4-6 weeks! We do it in a couple of days.

Once our Release Scope is defined, we execute our enhancements using workflows.

Our test cases, manual and automated, are all online. They are defined module-wise and graded on their importance. Depending on the impacted modules in the Release, we identify our scope of regression testing with a few clicks.

Test execution and defect tracking is all the system. We don’t have to maintain long lists in spreadsheets trying to build pivot tables on status, resource-wise, age, etc.

Our customers file issues that we convert into to defects or enhancements. Once again, we do our impact analysis on the system, review it on the tool with comments and then, assign it for coding and testing using workflows. When we commit our changes (we use SVN), we identify the defect or enhancement it is for and completely traceability is established automatically.

My experience is that organizations handling an application of this size would have 3-4 Technical Leads managing a team of 30-35 FTEs and a full time Manager. We have less than 1FTE playing the role of a Release Manager. We meet once in a week for 2 hours. That is the extent of Project/Program Management that we do. Rest is all online…

My CEO gets his full dashboard on the system, real time. No multiple versions from different managers, no data manipulation, no dedicated operations staff preparing Excel/Word reports on Friday evenings or over the weekend.

The results: we have made a release every quarter for the last 2 years. We have slipped once by 3 days. We have brought our defect rates down by over 30%. Of course, we have built a great team but without one integrated tool, we would have been in a mess, doing a lot of rework!

So, if you are not on an ALM tool, get on one! In 6-9 months, you can’t imagine building software projects without it. How many times do you handwrite a letter anymore?

ALM tools – why is their visibility limited?

I have often wondered what has prevented the modern day ALM tools from proliferating in the market place. Haven’t we all heard about an extraordinary percentage of projects failing? So, why have most decision makers in the industry refrained from procuring such tools? The answer is pretty simple – usage of such tools have not improved the success rate of these projects.

Any successful initiative is based on three dimensions – people, processes and tools. ALM tools fulfill the Tools dimension well. Unfortunately, tools are only as good as the people who use the tools or the processes that they choose to implement. Of the three dimensions, tools are perhaps the least influential in make a visible impact.

If the passion of successful projects is what drives companies ALM tool vendors, then I believe these are areas that cannot go unfocussed. Tool competency has to be accompanied by services that will help make Project Managers more successful as well as processes that will make the overall “package” more effective.

There are companies that have addressed at least two of these dimensions. They preach the methodology that they practice and provide the toolset built on that methodology. Companies like Miller-Heiman do this in the area of Sales practices. Rally, Thoughtworks do that in the area of Agile practices. Rational was perhaps the most successful company in preaching the UML way of doing things and selling the entire software that goes with it. There are examples of other companies having reasonable success focusing on at least two of these three dimensions.

These companies did completely miss out the entire Project Management space but lack of it has not been a strong negative. We will discuss that in a later topic.

There is one more challenge for generic ALM tools. A tool that tries to automate every methodology is, to a large extent, “generic”. It does not capture the essence of any methodology. It is a combination of screens, reports and metrics of every methodology. However, it’s UI and usability does not reflect the finer nuances of any specific methodology. Its training material does not capture the essence of any specific methodology. Its consultants do not speak the language of any specific methodology. It’s Sales team does not sell the spirit of any specific methodology. Last but the not the least, its users, who use the toolset, use in a combination of different ways, which is more often than not, a “confused combination” of different methodologies.

For now, most organizations in this domain have completely ignored the third dimension. This has been left to the combination of soft skill training programs and domain specific PMI training programs. One can only wonder how successful a company could have been that defines the methodology, trains Project Managers on becoming more successful with that methodology or other soft skills and provides the toolset to automate that methodology.

Of course, no business problem has one solution. However, this seems to me one of the critical missing links that prevents the “mass” adoption of ALM tools as I would expect.