Category Archives: Agile

10 (understated) Lessons from TPS!

There is a  lot of of literature on application of  Lean and TPS to software industry. The Kanban Method takes the majority share of discussion around this. However, there are some key takeaways from Lean thinking that need to be understood/appreciate but haven’t got adequate focus in the software community. This article puts into perspective some of these learnings from TPS and what it means for us in the industry.

The Production Pyramid:

Production Pyramid is basic of a Toyota Production System. There are many variations of it in the public domain. However, one of the versions that I was introduced is this:

04

Production pyramid has some key insights for us.

Lesson I: Getting the foundation right!

The foundation of any improvement starts with 5S activities. As we get better, we focus on existing wastes (the 3MUs) and supplement them with Visual Management methods to improve our process.

Lesson II: Be patient to get the foundation right!

There is no direct co-relation between the final objective of Profit and these foundation activities. Yet, without putting into place a strong foundation, any expectation of sustained profit is misplaced.

Corollary: When organizations focus on short term profits (monthly or quarterly), they will often drive themselves to ignore these foundation activities, causing them long term negative impact.

Understanding 5S Better:

One can refer to Wikipedia for a more detailed elaboration of 5S. However, for the purpose of this article, I will summarize the same:

  1. Seiri (Sorting): Sort articles into necessary/unnecessary categories; remove those that are not needed
  2. Seiton (Orderliness): Necessary articles at necessary time; you should be able to take out any article at any time.
  3. Seiso (Cleaning): Inspection to find abnormalities, keeping ergonomics in mind
  4. Seiketsu: Maintain a clean state (that you have got from the above 3S)
  5. Shitsuke: Develop the habit of abiding by what is laid down.

Lesson III: 3S vs 2S

It is important to recognize that while the Top 3S help to get better using Kaizen, the Bottom 2S help us to sustain the gains from the Top 3!

Continuous practice of these activities should be a part of the team habit/culture; not an afterthought from some corrective action or a senior person pushing the team to move in this direction.

While there are a lot of areas of application of Seiri, Seiton and Seiso in the manufacturing sector, there are some impacts areas for us in the software industry, though exactly not in the same buckets. Some of these, but not limited to, are:

  • Technical Debt: there are many things that contribute to technical debt and one needs to focus on all of the same
  • Workplace hygiene: once again, there are many things that fall into this bucket and team should focus on them
  • Standards/guidelines for design, development, etc. The importance of this extends to naming conventions, folder structure, etc.
  • Information sharing across teams using notice boards, portals, team meetings, etc.
  • Right infrastructure for the team/job profile, including hardware/software/other resources
  • Clear role/responsibility definition with person identification
  • Inspection to find work product abnormalities, be it for code/test cases/design documents, etc.

Many of these would also get repeated when we focus on removing Muda. There is already a lot of literature on the same in the public domain that I am not repeating.

Lesson IV: Three stages of Seiso

There are 3 stages of the Seiso process that can be well adapted to the software development:

  • Start off with the actual fixing work (defect fixing, design fixing, test case fixing, infra fixing, etc.). In other words, focus on “resolution”.
  • Kaizen the process of fixing – how can this be improved itself?
  • What can we do so that this “fixing” can be eliminated? In other words, focus on “prevention”.

Software professionals from the CMMI background are well familiar with Corrective Action and the DAR process area. It isn’t hard to recognize the similarity between the two.

This 3 stage process can help teams go much beyond just the “fix”. In most development teams, the urgency to “fix” things prevents the team from getting deeper into the next 2 activities. Its counter-productive in the long term!

Lesson V: Keeping a “clean” state

Seiketsu is about how to keep a “clean” state! Its analogous to keep a level of code quality once it has been achieved. Traditionally, project teams have been weak in this. We have a large Testing phase to clean the code only to see the code quality go bad in the next Development phase; then, we again hit the Testing phase.

Agile thinking, with its associates engineering practices (TDD and CI) focus on this specific aspect. One cannot really be Lean or Agile without these engineering practices.

Lesson VI: Using Visual Management when abnormality is detected

The other aspect of Seiketsu is to use Visual Management methods to alert teams when abnormality is detected. This is analogous to a build failure that gets notified to the team participants immediately for attention and resolution! There may be many other points of failure that the team would like to focus on visually. Kanban Boards have 2 such examples: a) Lanes on a Kanban Board getting highlighted in RED when WIP limits are exceeded is one such example b) A card that is not able to flow forward getting highlighted using Blockers. A project teams should identify such points of failure/abnormality and define visual methods to highlight the same.

Lesson VII: Implementing Shitsuke

Implementing Shitsuke is done in the following 3 stages:02

  1. Decide what needs to be defined; make others aware of it
  2. Follow the procedures; make everyone follow it
  3. Confirm if the procedures are being followed; appreciate the people who are following it and guide the people who aren’t.

Lesson VIII: Improvement initiatives are harder to sustain

It is also well acknowledged that while one time improvement initiatives are doable and achievable, it is lot harder to sustain these initiatives and get them absorbed in the team DNA. In other words, of the 5S, the latter 2S is much harder than the former 3S. If not sustained with persistent effort and management focus, the benefits gains from the first 3S fades away over time.

Lesson IX: Keep broadcasting your success

Once your succeed in reaching a milestone of process improvement, share it with others and talk about it. Obviously, it motivates the team and helps them gain recognition for the good work they have put in.

What isn’t obvious is the phenomenon that happens when you continue to display the results! If the results do come down for some reason, the team is strongly motivated to set it right. Its embarrassing to fail on something that you have just been acknowledged for!

Lesson X: Kaizen vs Sustenance

This brings us to the most important learning – while Continuous Improvement is important, the team needs time to sustain and internalize the improvements from the earlier Kaizen cycles. My experience is that this is where many teams fail!

The importance of this alternate cycle between Sustenance and Kaizen is understated in current Agile/Lean literature in IT (to the extent that I have read). We keep reiterating the importance of Continuous Improvement. It is assumed that once a good practice is identified, it will be adhered to over time. In reality, it is much harder.

In the Kanban Method, we talk about the importance of Defining Policies. While that definitely moves in that direction, there is more focus needed to sustain.

Maintenance/Sustenance means setting standards and carrying out activities as per the standards. One typically undertakes a SDCA (Standardize, Do, Check, Act) cycle in this phase. A team needs to develop Standards/SOPs/Checklists to help the team follow the same and stabilize current practices and methods.

Kaizen means to set targets higher compared to current standards and then carry our activities to make it happen. One typically carries out the commonly understood cycle of PDCA for this.

A team needs to move in alternate cycles of Kaizen and Sustenance. Without this, any gain is temporary and it fizzles over time.

05

Kaizen leads to newer practices and methods that need to be standardized and internalized by the team BEFORE the next cycle of kaizen improvements are brought to the team.

So, don’t just focus on improvement… keep stabilizing and improving!

I want to end this blog by acknowledging Norio Suzuki for his inputs and helping with my deeper understanding of Lean and TPS.

The NextGen (Agile) “Tester”

As Agile gets larger acceptance across the industry, the (manual) testing community has been concerned about how it will impact them. They hear about developers doing testing, test automation and hence wonder – what role am I going to be play? Is this going to kill my job? The concern is natural given a general perception that in Agile, developers should be doing testing. This concern has been part of the resistance that some of the organizations are facing in adopting Agile thinking.

This concern is misplaced and if anything, Agile thinking only makes the Tester more central to the teams. Fact is that developers were always supposed to test their own code! It’s a different matter that this was not happening, and when it did, generally not with the right spirit. Agile makes this even more imperative – you cannot be truly Agile without developers doing testing and automating it on a continual basis. Of course, if you can do Test (First) Driven development, you would have addressed the long term challenge of “Test Coverage” (a topic for another day)!

Development and Testing: Ne’er the Twain Shall Meet!01

Coming back to the topic, let us understand why development and testing was split traditionally. We all grew in the industry being told that developers and Testers had to be separated for the following reasons:

  • Reason 1: How can the developer be trusted to tested his own code? He/she would always pass his/her code!
  • Reason 2: If the developer has understood the functionality wrong, he/she will build it wrong and he will test it wrong, even if he tests with utmost sincerity.
  • Reason 3: testing needs special skills. The ability to write boundary value test conditions, negative test cases, do test data planning, compliance to various standards (UI, navigation, etc.), complex functional test scenarios were not exactly a developer’s core competency or his area of interest.

Test Automation: Nice to Have…

Test Automation was always an optional practice – good to have but too expensive to maintain. Focus used to be largely on automating functional test cases. These would break frequently because of changes in the UI. Newer approaches like keyword based test automation addressed this problem, but it still was an expensive process. It needed a lot of effort, took long execution time, took a lot of hardware resources and needed high maintenance.

The Agile Approach

Agile thinking, with some of the associated engineering practices, makes some fundamental changes:

  • Agile thinking puts the responsibility with the team – not with the managers! In other words, we are encouraged to trust the team. Interestingly, this way of execution actually works! This killed “Reason 1”.
  • Agile thinking accepts that Requirements will never be accurately written or fully understood in one pass. It shifts focus to the developer sitting with the business or Product Owner and understanding the requirement, in a conversational process. It insists on small and early releases to the user and taking their feedback. This killed “Reason 2”.

Test Automation: With Test (First) Driven development (TDD) and with the correct implementation of the Testing Pyramid, Test Automation is faster and practical. Functional test automation is a small % of the total. Bulk of the test automation is at the unit level. It is extremely fast and has no dependence on the UI.

What Agile thinking has not changed is that testing still needs special skills (Reason 3). In fact, with the rapid evolution of technology, the demand for special testing skills has only increased over time.

Whither Manual Testing?

So, how does the role of Manual Tester change? It truly doesn’t. However, their skill profile and how they work in a team changes. Let us understand these:

  • Early Involvement: testing was generally considered to be a late in the SDLC cycle activity. While some testers would start early test case development or test data planUntitledning, the bulk of the testing team members would join much later. With Agile thinking focus on continuous delivery, testers need to get involved early in the project life cycle, understand the business and be an integral part of the team. They are expected to almost become the “next level” domain experts!
  • Understanding the business requirement: Gone are the days that someone will define Requirements in a Word document and that will be taken as the single/only version of the Requirement. Agile accepts that Requirements need to be discussed and understood in a manner of conversation with the user. Accordingly, the tester needs to participate in this requirement conversation, with the user or the product owner (whosoever is the requirement owner in the environment), understand the requirement and draft the functional test cases. Thus, the role graduates to the next level – from writing test cases based on a MS Word requirement document to actually participating in a discussion with the user, understanding the business requirement and then, writing the test cases.
  • Accepting requirement change: Agile thinking accepts requirement change as a continuous process. Hence, test case drafting isn’t a one time activity. As requirements evolve from one iteration to another, test cases keep evolving. Testers need to accept it… not crib about it! Layering test cases in one of the possible approaches to address this.
  • A different mindset: A tester’s responsibility is not just to find defects! His/her job is to improve the quality of what the team is delivering. A tester is part of the team that is delivering the scope in that iteration/value stream. A tester cannot be pointing fingers at the developer. If he/she sees that a developer is not clear about a functionality, he/she should clear it for them! A tester needs to think like the user… they are the closest to the functional user inside the team.
  • Automate: A tester cannot be manually doing regression testing as the product functionality evolves from one iteration to the next. They have to automate, automate… and automate. Automate everything! A tester will have limited value to the team if he/she is not willing to learn and excel in any of test automation frameworks.

In summary, be rest assured that a tester’s job isn’t going away! If anything, they are now a central part of the team. They should act like one! They should excel in the domain… and drive the overall work product quality.

Applicability of Agile/Lean/Kanban Methods for fixed scope/budget projects (with short duration)

The Pune Chapter Meet of the Limited WIP Society was held on Mar 8, 2014. This session objective was to address challenges faced by Lean-Kanban practitioners in their projects. Participants from 4 different organizations attended this session. A couple of problems were posed by the participants (Meetup Link) ahead of time. We decided to work on the first one for this session.

Problem Statement: Some projects are fixed scope, fixed budget and have relatively small duration – 3-4 months. In this case, would it make more sense to go for: a) Pure Critical chain or Microsoft project plan based date based approach b) Combination approach: Critical chain based planning followed by Kanban execution. c) Pure Kanban flow based execution approach with no date based plan for individual stories.

The team started the session by working in teams to identify what aspects of Lean/Kanban facilitate small project execution that have fixed scope, fixed budget and small duration (“+ives”) and what aspects don’t (“-ives”).01

Next, they came to the white board, brought all their points and grouped them together. The result looked something like this!01

Result: The team, working independently, had identified 18 positives vs 14 negatives in adapting Lean/Kanban/Agile principles to fixed scope, fixed budget projects with a short duration That a very positive reinforcement of their suitability. Therefore, Option C was taken as the way forward for the rest of the session.

While the session started with a focus on projects with short duration, it was evident that except for a couple of points, this discussion was valid for project of fixed scope/budget, independent of any duration.

The following Positives were identified when applying Agile/Lean methods to these projects:

Positive Contributors

Planning

  • If using SCRUM, defined planning activity
  • Small Cycles
  • Estimate upfront (high level) and understand the feasibility of delivering
  • Divide into stories; complete logical business flows
  • Can help in sustainable pace

Quality:

  • Higher Quality Delivery
  • Stable Software
Scoping and Prioritization:

  • Early Start of mature (well defined items)
  • Focus on prioritizing – very relevant
  • Principles of Agile help; focus on blockers
  • Power visualization

Early Feedback

  • Still relevant
  • Demos are confidence building

Team

  • Team empowerment is positive
  • Collaboration
  • Encourage small team size
  • Cross functional teams

After a brief discussion on the positives, since the positives were stronger than the negatives, the focus on the remaining session was how to mitigate the negative factors in these projects.

The group decided to focus on the negative contributors. Each group was given one of the negative areas to focus on. They were asked to break up the negative contributors to the next level (identified as LI below) and identify possible resolutions for the same. Then, all teams converged and refined the possible resolutions in each of the areas. The following negative contributors were identified with a clear path to resolve them.

Negative Contributors
Planning Area:  01
Negative Factor:MSP is used for Portfolio and Program Planning..Resolution Approach: Lean Methods exist for Portfolio Level Planning, like Portfolio Boards. We need to educate that it a myth that Agile = No planning.
Hence, training, education and pilot programs need to be considered as a means to bring about awareness on the importance of planning and the methods available.
Negative Factor: For such small projects, the team might have good visibility of the scope. Budget, scope and timeline for such projects are often well defined. Hence, they might be tempted to plan heavily upfront and execute based on that..Resolution Approach: High level planning is good and if the details are available and if everyone feels assured about it, then doing isn’t a negative. Avoid detailed estimation, planning to man hour level and then getting into variance tracking. This will lead to timelines getting met at the expense of quality and work-life balance. Negative Factor: In a very dynamic environment, MSP helps in quick impact analysis..Resolution Approach: This is not accurate. With MSP, the defined path was cumbersome. One would find the critical path, then do all kinds of jugglery like fast tracking, crashing, etc. It was based on estimates and we all understand that estimates are dead on arrival!So, we need to move all stakeholder discussions to velocity/throughput centric discussions. To enable this to happen, a lot of training needs to happen for all stakeholders involved.
Negative Factor: No Dates; hence, things pile up.Project Manager does not get sense of delays; lack of timeline; project delivery date is variable..Resolution Approach: The obvious answers the team came up with was to have time boxes (what SCRUM does). Also, it was discussed that Kanban does not have anything against Due Dates. In fact, many Kanban team use Due Dates. The risk is that Due Dates should not construed as milestones that puts pressure to finish the job, no matter what.
Resource Management:  01
Negative Factor: MSP helps in Resource Planning, Capacity Planning..

  • Cards pile up in the end; no flexibility to take care of scope uncertainity.
  • No targets for SWP; ST is not able to plan testing
  • Sudden resource requirements that come up case disruption to the whole project
Resolution Approach:

  • The best way forward is to have stable teams. However, if this is not possible in a given environment, we should be able to visualize this constraint and give time to stakeholders to plan resources. The way to do this is document in this blog.
  • Another approach is to under-allocate critical resources. They are often called in to help others or take care of some critical work in the pipeline.
Testing:  01
Negative Factor: One has to test repeatedly, specifically regression. Since the project duration is short, the tendency is to test at one shot in one batch.Assumption Testing and development are being done by 2 different people..Resolution Approach: Automation
Where automation is not practical or feasible for whatever reason, the team felt that right documentation or Knowledge Transfer to the tester can help reduce repeat testing. If the tester understands the scope of each user story well, then he can focus on the scope of that user story and in subsequent test cycles, focus on scenario based testingIn a specific environment where ST is expected to “Certify” the product quality and hence, need to re-test “all” test cases, including unit level test cases, the recommended approach is to focus on reviewing the comprehensiveness of the Unit Test Cases and Unit Test Results. That would help Dev teams get better by not taking Unit Testing lightly. By ST doing unit level testing, it makes Dev teams continue with their current practices.In cases where Dev teams are mandated to Junit testing, ST teams can seek the Junit Code Coverage data to understand the level of comprehensiveness of Unit Testing.
Env/Infrastructure:   01
Negative Factor: The team identified Configuration, Sanity, Missing HFs, Connectivity, Overloading of environment as all related issues to environment, infrastructure..Resolution Approach:

  • Environment/Infrastructure planning cannot be a batch mode process. Just like capacity planning, Env/Infrastructure needs to be continuous process. Whenever a card gets prioritized within the backlog, all its environment and infrastructure needs can be defined on a card and put in a parallel swim lane to track them to closure (see screen shot)
  • Get people from the Infrastructure/ Environment teams involved in the Sprint planning process; if you are following the Kanban method, pull them in whenever Backlog grooming happens.

01

Better appraisals with ALM solutions

In my industry, one of the most difficult experiences growing up was the appraisal process. Filling up 4 to 5 pages of a Word document was a test of English language – both for me and my Manager. I recall sitting with the Thesaurus to find synonyms for the same adjective, just to avoid being repetitive. My manager had the same challenge! As a result, it wasn’t a meaningful exercise for the most part. Don’t get me wrong – I am not saying that I did not get feedback for improvement. The problem was the scoring process that was quite ambiguous and vague.

This experience was different was when I moved to a Direct Sales function. In Sales, you learn quickly that you are as good as your numbers of the past quarter. No ambiguity, no ill-feelings. That learning stayed with me when I transitioned back to software development.

In my previous organization, I was introduced to the BSC model. Though it was a vast improvement over the earlier appraisal methodologies, the data necessary to adopt that model was still lacking! Hence, its effectiveness was also marginalized.

Here comes the value of a tool like Digite Enterprise. Let me share how we measure our development team. They are measured on 3-4 parameters. First, how did the quality of your delivery improve? We measure this by measuring the % reduction in defects, QoQ. Next, we measure on how long a defect aged in a developer’s queue. Further, we measure how many regression defects were introduced by the developer. We also measure Time Sheet compliance by measuring the difference between muster hours and time sheet hours + process compliance by doing sample audits. It is objective and measurable. With the deployment of a tool like Digite Enterprise where all defects and hours are being tracked, resource wise, this data comes out as a breeze.

The testing team is measured on Defect Containment and the quality of the Test Cases they write (measured by the number defects in their test cases). Test automation team is measured on test case productivity (measured as a percentile of others productivity) and the quality of automated test script (measured the number of defects in their automated scripts). Once again, because test cases and test execution run data is all on Digité Enterprise, this data is available with a few queries.

Our implementation team is measured on schedule and effort variance in their implementation projects. Similar goals can be easily be set for other organizations like Support because all tickets are being tracked on Digite Enterprise.

As a result of all this, we are able to finish appraisals shortly after the quarter end. Our employee frustration with appraisal ambiguity has disappeared. There are no perceptions of manager biases. Our appraisal discussions are done in 20-30mts. In fact, most of the discussion happens around tweaking the objectives for the next quarter based on what we plan for that specific individual for the next quarter.

In the last 20 years, my experience has been that if you limit the appraisal criteria to 3-4 objectives that the organization wants to focus on and can be quantified, the appraisal process is a pleasant experience for all.