Tuesday, 15 December 2009

Handling the people side of Agile software development

Articles about process, metrics, iterations and techniques are great, but Agile software development is more than that: Agile development involves a culture shift. That's something that isn't talked about enough and has significant impact on testing practices.

In this tip, let's look at why "going Agile" might not be as easy as it sounds, what some of the prerequisites are, and what you can do about it. First, I'd like to thank this tip's peer reviewers: Lanette Creamer, Markus Gaertner, Catherine Powell, Marisa Seal and Patrick Wilson Welsh.

The Agile problem

A week ago, I sat back, looking at my articles on Agile testing . Boy, was I proud. I'd covered how to compress testing to fit in an iteration, how to make smart use of automation and how to distinguish Agile testing from "excellent" traditional testing. Then, slowly, I came to an awareness that something was wrong. I had published three articles on Agile Software Development without talking about people.

Let's take another look at the Agile Manifesto , the document that defines Agile Software Development: "We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools;
Working software over comprehensive documentation;
Customer collaboration over contract negotiation;
Responding to change over following a plan


That is, while there is value in the items on the right, we value the items on the left more.

Yes, that's right. Three articles about process. Three articles about tools. Yes, the process and tools were designed to enable changing the plan, but it certainly wasn't explicit and there certainly wasn't much about engaging with the customer. While all of those things are good, the tools and process are supposed to serve and enable the team, not be it's master. Tools and process are the means, not the end. So let's talk about the hard stuff, the people side of Agile development, and why it matters.

Developing ideas, not manufacturing

Ever since Henry Ford made his first million, the American people have been in love with the assembly line. "Standardize the process, decrease variation and crank out widgets," goes the thinking in a stable, predictable and repeatable format. In fact, even before Henry Ford, old Frederick W. Taylor suggested that we separate the worker and the work, to make sure the process is done the same, every day, defined by a class of process engineers.

If we are truly developing the same thing day in and day out, there may be some benefit in standardizing the work. But intellectual property is different every time. Instead of separating the worker from the work, we need to engage the worker in the work to think actively about the work itself and continuously attempt to improve. To paraphrase author and poet Richard Bach, we must own our process, or else we shall ever be its victim.

Cross functional project teams

Consider the typical cubicle-bound "testing" team. The team is organized by specialty, not project and rolls from project to project, reporting to a single test manager. In what sense are they a team? The 'team' members work on different projects, compete directly with each other for raises, and never work together. Can you imagine a sports league with "teams" of pitchers, first basemen, shortstops, and so on, that are "loaned" to build temporary project teams to play games? That's preposterous. Agile software teams emphasize a project team. They may do this in a number of ways, from having a reorganization, to working in Amazon's Two Pizza Team structure, to just having a war room and focusing on the project team.

A cross-functional team should have all the capabilities it needs to get the work done. No waiting for a DBA to create a table, no waiting for an architect to approve the design. The bottleneck on the team will be the team members themselves.

Self organizing, Self directed work teams

If the bottleneck is a team member, then how about shifting the load off that team member? That might mean a business analyst tests, or a tester learns the build system or helps out with some queries. The point is the business gives the team a problem, and the team itself figures out the best way to solve it. Of course, the business can add requirements for what documentation to produce, or what audit steps to put in place, and the team can report right back on the cost of those new requirements.

This means that instead of 'appointed' leaders making decisions in a command and control fashion, leadership is emergent and consensus driven. Yes, I once had a manager tell me he could never get Bill and Bob to agree but you would be amazed what locking people in a room will do to help them decide.

So far, we're talking about changing the model, from prescribing what people are to do and making decisions for them, to one of enabling the team to decide for itself when to work, where to work, how to work, and if the customer is embedded in the team what to work on.

But how do you treat people?

That "individuals and interactions" bit in the manifesto is also the major value; if I had to reword it, it would be treat people as humans. Humans are inconsistent; we laugh, cry, and show emotion. We have good days and bad days, and we are not all the same. We also have temperament, something I have very aware of as a left handed person. Some people just plain prefer to do work differently. Some are chatty, more effective when they know the person they are working with well; Others connect through the actual doing of the work. Some take pride in following a process; others value a successful outcome. Some like constant feedback; others might just turn around and say "stop encouraging me and let me finish."

Traditional documentation and process driven software development has clear desire to remove the unpredictability from the process. What it fails to notice is that humans are unpredictable; thus traditional development robs the process of its humanity. As hard as it is to measure, this basic humanism is a large part of what it means to be Agile. Many of the teams talking about "going agile" would be well off to look carefully at how they treat people, and ask "would I want to be treated this way?"

CargoCult 'Agile'

So if a team adopted the artifacts of Agile development but missed the point, would that look like? It would certainly start out with a loud voice, declaring for the team that "We are going Agile." It would put people in boxes. It would tell them what the process should be on a meta-level. It would ask them to contribute as specialists, and it would still tell them what to work on, where to do it, when to do it, and how to do it.

There are many fake forms of Agile, but the one I'm writing about would continue to focus on process and tools at the expense of the individual. It might throw the word "iteration" around a lot, but it will still insist on a firm promise for a very long feature list to be completed in exactly nine months.

Dealing with fake Agile practices


Let's say just for argument's sake, that you know of an organization that wants to use Agile Practices without changing its fundamental culture. Well, I suppose that's possible . You might even see some modest improvements in team velocity and project success. But the deep stuff of Agile development doesn't come from adopting a process it comes in setting the team free to adopt the process it wants.

Once you recognize that you're dealing with an impostor, that command and control thinking still remains entrenched in your organization, there are several things you can do. You can try to push, inches at a time, in the direction of cross-functional teams and team autonomy. You can invite the friendly DBA, the available architect, and anyone else who "isn't on the team" to the daily standup. You can find the project manager who "gets it" and request to work for them. Likewise, you can look for the troubled project, the desperate project, the one where the project manager might actually be willing to implement the team's crazy plan to get the thing done by December 1st.

If your company is just starting to talk about "agiling up" it's software process, my advice is to forget about time-boxed iterations and test driven development. Oh yes, those are good things, and they can enable agile software development, but let's be very clear if the team isn't self-directed, cross-functional, respecting people, it's not Agile software development. It's ... something else.

Hold on to those ideals. Communicate about them clearly. Fight for them, leave no room for misunderstanding, and you might just make your little part of the software development world a better place.

I think that is noble, and worth fighting for.

About the author: Matt Heusser is a technical staff member of SocialText, which he joined in 2008 after 11 years of developing, testing and/or managing software projects. He teaches information systems courses at Calvin College and is the original lead organizer of the Great Lakes Software Excellence Conference, now in it's fourth year. He writes about the dynamics of testing and development on his blog,Creative Chaos.

Use cases for software requirements tutorial: Strengths, flaws, formats

The use case process is a popular requirements documentation format that can aid communication and also can facilitate testing as a by-product benefit. While extremely valuable, use cases do have weaknesses which often are not recognized and thus can lead to risky over-reliance.

Let's get on the same page about what a use case is. defined as "how an actor interacts with the system," a use case usually relates the actor to the user, but the actor could also be another system or a piece of hardware.

A use case is titled in terms of its goal, such as "Place an Order." Some business or temporal event trigger invokes the use case. For example, a business event might be that a customer is looking through items the seller has for sale and decides to purchase one. A temporal event might be that the customer has agreed to renew their order automatically on a given date each year.

The heart of the use case describes the interaction step-by-step from the perspective of the actor: the actor does this, and the system does that in response. Such concrete behavioral descriptions make it easier for the user and developer to have a common understanding of how the system will operate. Consequently, use cases can enable the developer to develop a system that works just the way the user expects!

What a delightful change this can be from the typically too common disappointing situation in which the developer designs dynamically -- i.e., guesses, usually without being aware of guessing -- how the system should function, only to discover afterward that it's not what the user wants. Because of use cases' success in reducing such discrepancies, they have become probably the most widely-used method of documenting requirements.

Narrative and one-column formats

The simplest use case format is a narrative, where each sentence is a separate user behavior or system response. For example, a narrative format use case for placing an order from the perspective of the customer placing the order might be:
Identify the customer in the database by Customer ID. System displays customer's identification information. For each item to be ordered, select the item from a menu of items for sale. For each item to be ordered, system displays the item's ID, description, and unit price. For each item to be ordered, indicate the quantity of the item ordered. For each item to be ordered, system calculates the extended price equals quantity times unit price. Indicate the order is complete. For the entire order, system calculates the total amount due equals sum of extended prices for all items ordered. Pay the total amount due.

Because the narrative format inherently can be hard to understand, use cases most commonly are documented in a "one-column" format, where each step is an individual numbered line. While in practice the amount of verbiage may differ and thereby obscure the one-line-per-step essence, the same use case in a one-column format would be:
- Identify the customer in the database by Customer ID.
- System displays customer's identification information.
- For each item to be ordered, select the item from a menu of items for sale.
- For each item to be ordered, system displays the item's ID, description, and unit - price.
- For each item to be ordered, indicate the quantity of the item ordered.
- For each item to be ordered, system calculates the extended price equals quantity - - times unit price.
- Indicate the order is complete.
- For the entire order, system calculates the total amount due equals sum of extended - prices for all items ordered.
- Pay the total amount due.

Since these nine steps would be the normal way the use case is executed, they often are called the "happy path" or "main path." Note, even though these nine steps seem evident and straightforward, it's possible someone else could have described a somewhat different set of happy path steps for this use case. There's not a single one-and-only way to characterize the steps inherent in a situation. I don't think most use case writers realize how significantly the depiction of use case steps can vary from writer to writer.

While these days, use cases' step-by-step descriptions from the user's perspective are assumed to be intuitively desirable for developers, that wasn't always the case. I've used the technique since long before Ivar Jacobson gave it the clumsy "use case" name; and I've got to tell you that back then traditional programmers were not at all comfortable working with specifications from the user's perspective.

Differentiating between Functional and Nonfunctional Requirements

Many of the difficulties encountered in defining requirements relate to the functional-nonfunctional distinction. As so often happens, some of those difficulties are things of which people are aware, largely realizing after-the-fact that nonfunctional requirements were missed. However, perhaps more, and certainly the more insidious difficulties, involve other issues of which people may not be aware and often may be incurring intentionally due to mistaken understandings.

Functional Requirements

Most requirements definition focuses mainly on functional requirements, which are based upon the expected functioning of the product or system to be created. Functioning, typically is equated with product/system features for which you might have a menu or button choice, such as: identify a customer, select an item to order, and calculate the amount due.

All things considered, requirements definers probably are best at identifying functional requirements, although they often overlook and get wrong more of the functional requirements than they ordinarily recognize. On hindsight reflection, they frequently do realize that many of the problems which surface later, and thus are harder and more expensive to fix, are attributable to inadequately addressed nonfunctional requirements.

Nonfunctional Requirements

Nonfunctional requirements refer to a whole slew (I've identified more than 30) of attributes including performance levels, security, and the various "ilities," such as usability, reliability, and availability. Invariably, requirements definers get wrapped up in how the product/system is expected to function and lose sight of these added elements.

When such factors are not addressed adequately, seemingly proper product/system functioning in fact fails to function. For example, a system may identify customers in such a slow, insecure, and difficult to use manner that it can cause mistakes which make data unreliable, provoke frustration-based attempted work-arounds that can create further problems, and ultimately lead to abandonment.

That's the recognized way in which nonfunctional requirements impact product/system success. Other often unrecognized issues also need to be appreciated.

"Nonfunctional" is misleading

First, I would encourage calling them "quality factors" or "quality attributes" rather than "nonfunctional requirements," which I believe is misleading. "Nonfunctional" implies they exist in the abstract or can be pigeon-holed, which can only lead to nonsensical questions such as, "How much usability do you need?" You don't need a bunch of usability. Rather, these requirements are relevant only with respect to functionality. For example, usability requirements pertain to particular usage situations; and a given product or system can have many different usability requirements—for each of the respective usage situations.

Also I'll caution against calling them a common alternative term, "quality requirements," which implies erroneously that these factors by themselves are what is meant by "quality." Instead, quality first and foremost comes from suitable functionality. Speed, ease of use, and the like contribute to quality if and only if associated with useful functionality.

Templates and checklists are helpful for remembering to find out about these factors but should not dictate the manner in which they are stored. That is, you shouldn't just have a group of usability requirements, a group of security requirements, and so on. Rather, all the relevant quality factors modifying a particular requirement should be captured with it.

Both Are Really Design

A second, far less recognized but very important issue with functional and nonfunctional requirements is that they actually are forms of high-level design pertaining to a product or system which is expected to be created. A tip off is that frequently they are called "specifications" instead of or in addition to "requirements." "Specification" is design.

Problems occur when the bulk of requirements definition is focused on specifying the product's functional and nonfunctional requirements, which is usually what happens. The product/system will provide value if and only if it actually satisfies the REAL, business requirements.

When most of the attention is directed toward the product/system presumed solution, there's too little understanding of what the product/system must accomplish to provide value; and what seems like a suitable product/system often turns out not to provide the value it should.

The key is first to discover the REAL, business requirements deliverable whats that provide value when met and then design a product/system how to satisfy the whats.

Friday, 3 July 2009

Agile and CMMI

First, Agile and CMMI are not in the same or opposite fields to be compared. Hundred of companies worldwide are said to have adopted an agile practice and got certified the CMMI accreditation.
Second, to define both...CMMI stands for Capability Maturity Model Integration (which is by definition of the SEI) "is a process improvement approach that provides organizations with the essential elements of effective processes". But in a nutshell it boils down to three main components: Generic goals, specific goals and documents to guide the organization to reach an effective process, which on the other hand means a set of deliverables to be available on hand before the time of appraisal for the accreditation.
An agile practice has a lower scope as it mainly focuses on the project management rather than the organization effectiveness; it looks for the work to be done as “the finished work is the measure of quality”. So in a sense, the agile practices can be utilized in a way of practicing of the process guided by the CMMI standards.
Hence, CMMI and an agile practice can really work side by side if the team applying it had the enough knowledge and will.

Tuesday, 7 April 2009

“We must educate the customer” !!

I have been thinking for more than 3 weeks now about a sentence I heard during a meeting on the CMMI preparation and it really grabbed my attention with a wide smile.
“We must educate the customer”
Well, I think that those words are not only a dream but also a suggestion like making the sun shines from the west one day.
Customers don’t want to be educated, they want an output in their wallet and I hope the following example explains what I mean.

Most of the IT companies especially here in Egypt try hardly to please their customer in any means possible (or even impossible); even if it meant lending them a programmer for a couple of weeks to serve the customer needs. Ofcourse the customer at any industry is most appreciated or as they say here in Egypt 3la 3iny we raassy (which literally means: on my eye and head!!) but the project managers have to be reasonable on how they are using their resources especially if the words human and reputation are involved.

Let’s consider the example of Sa3d (saa'd), a nationwide furniture manufacturer and seller, Sa3d is a self-made man who started only as a poor and ignorant worker and ended to be a tycoon.
Sa3d asks a company called UYRS (Upon Your Request Sir) to develop an ERP system to help him manage his business and expected a duration of maximum 3 months for development.
UYRS as a CMMI level 3 certified company and a proud IT company bragged with its products infront of Sa3d and assured him that the software will take a much shorter time.

But..
UYRS’s software has to be done by standards and… methodology and… life cycle and… coding techniques to assure its quality and its max performance measures and… and… and above all apply all the CMMI standards to the letter.
By the end of the 3 months Sa3d’s accountant called UYRS manager to ask about the software delivery date and follow up with him to send him his cheque and finalize the deal.
The manager responds that there has been some delay due to their goal to produce the best quality possible to Mr. Sa3d who is one of their best customers. Mr. Sa3d is not happy he wants to see his money’s value working so he himself calls up the manager and asks to meet him.
The manager reviews with his development team about the source for the delay and how to fix it. The humble project manager argues that Mr.Sa3d’s requirements were very ambiguous and complicated and they had to produce the software with the appropriate CMMI standards. And after the manager asks him to a solution he responds that he must be educated about the process and how the software lifecycle needs should be given.

Mr. Sa3d sits with the project manager who explains to him the benefits of CMMI and how his software would be a jewel in the field of software industry in Egypt, and that he would send him a programmer to finish the project installation and deployment at Mr.Sa3d’s company premises.
After a couple of weeks, some angry phone calls and meetings, Mr. Sa3d cancels the project and asks his 4th year computer science student nephew to finish the project and take 1000 pound cash… he finishes it after one week and finally Mr.Sa3d (or elma3lem Sa3d) is happy smoking hookah in his large office and reviewing the system by Hamada his nephew.


Let me repeat that word again (happy). A happy customer is all what industry is about, a happy customer comes back, a happy customer speaks well about the company, a happy customer is a fire starter among his peers and turns to be an indirect source of marketing for the company and above all a happy customer is a measure of success.
I do not disagree with CMMI standards, on the contrary I am a fan of CMMI and respect its guidelines but it can sometimes or (unfortunately here in Egypt) most of the times be a two ways weapon if it was used incorrectly.
There’s a great difference between activity and productivity, thinking towards the solution in a value-driven way is the best potion for success.

Tuesday, 17 February 2009

What is Agile

What Is Agile Software Development?

Never do anything that is a waste of time - and be prepared to wage long, tedious wars over this principle.

While interest in agile methodologies has blossomed in the past two years, its roots go back more than a decade. Teams using early versions of Scrum, Dynamic Systems Development Methodology (DSDM), and adaptive software development (ASD) were delivering successful projects starting from the mid-1990s.

The Agile Problem Domain: Fitting the Process to the Project

All problems are different and require different strategies. While battlefield commanders plan extensively, they realize that plans are just a beginning; probing enemy defenses (creating change) and responding to enemy actions (responding to change) are more important. Battlefield commanders succeed by defeating the enemy (the mission), not conforming to a plan.

I cannot imagine a battlefield commander saying, "We lost the battle, but by golly, we were successful because we followed our plan to the letter." Battlefields are messy, turbulent, uncertain, and full of change. No battlefield commander would say, "If we just plan this battle long and hard enough, and put repeatable processes in place, we can eliminate change early in the battle and not have to deal with it later on."

A growing number of software projects operate in the equivalent of a battle zone - they are extreme projects. This is where agile approaches shine. Project teams operating in this zone attempt to utilize leading or bleeding-edge technologies, respond to erratic requirements changes, and deliver products quickly. Projects may have a relatively clear mission, but the specific requirements can be volatile and evolving as customers and development teams alike explore the unknown. These projects, which I call high-exploration factor projects, do not succumb to rigorous, plan-driven methods.

The critical issues with high-exploration factor projects are as follows: first, identifying them; second, managing them in a different way; and third, measuring their success differently. Just as winning is the primary measure of success for a battlefield commander, delivering customer value (however the customer defines it) measures success for the agile project manager. Conformance to plan has little meaning in either case. If we want to be agile, we have to reward agility.

The concepts and assumptions behind empirical and defined processes are fundamentally different. The practices of agile software development -
- Short iterations
- Continuous testing
- Self-organizing teams
- Constant collaboration (daily integration meetings and pair programming for example)
- Frequent replanning based on current reality (rather than six-month- old plans)
are all geared to the understanding of software development as an empirical process.
On the other hand, the fundamental basis of the Capability Maturity Model® (CMM®) and CMM IntegrationSM (CMMISM) is a belief in software development as a defined process.

As such, tasks can be defined in detail, algorithms can be defined, results can be accurately measured, and measured variations can be used to refine the processes until they are repeatable within very close tolerances
For projects with any degree of exploration at all, agile developers just do not believe these assumptions are valid. This is a deep, fundamental divide - and not one that can be reconciled to some comfortable middle ground. It is part of having a chaordic (meaning a combination of chaos and order as coined by Dee Hock, founder and former CEO of Visa International) perspective on the world as described in the next section.

While agile practices - refactoring, iterative feature-driven cycles, customer focus groups - are applicable to nearly any project, I believe the agile sweet spot is this exploratory projects problem category. The more volatile the requirements and the more experimental the technology, the more agile approaches improve the odds of success.

Agile methods
Some of the well-known agile software development methods:
• Agile Modeling
• Agile Unified Process (AUP)
• Agile Data Method
• DSDM
• Essential Unified Process (EssUP)
• Extreme programming (XP)
• Feature Driven Development (FDD)
• Getting Real
• Open Unified Process
• Scrum

Agile practices
• Test Driven Development (TDD)
• Behavior Driven Development (BDD)
• Continuous Integration
• Pair Programming
• Planning poker
Note: Although these are often considered methodologies in and of themselves, they are simply practices used in different methodologies.

Agile beyond software development
Agile software development depends on some special characteristics possessed only by software, such as object technologies and the ability to automate testing. However, related techniques have been created for developing non-software products, such as semiconductors, motor vehicles, or chemicals.

Monday, 16 February 2009

Introduction to Agile..the Egyptain way




The word agile itself is very similar to the word Aa'gel in Arabic (which means soon) and it is defined in the English dictionaries as Mentally quick or alert.

"The reason most people never reach their goals is that they don't define them, or ever seriously consider them as believable or achievable. Winners can tell you where they are going, what they plan to do along the way, and who will be sharing the adventure with them" Denis Watley

Agile software development methodologies emerged from tons of lessons learnt by IT soldiers and leaders to provide efficient solutions to the road blocks that usually face their projects.

Though the IT technology dates back to the late fifties, people working in that industry rarely had enough courage to face the real cause behind the failure (fully or partially) of the projects. I consider delivering a project a day after its due time a partial failure. And that was what really pushed me towards expanding my horizons and engaging in a raging war against the dusty clichés of the software mentalities in Egypt.

Let's consider this example; the time is 8 pm in elkasr El3iny Street downtown on a late day of Ramadan. The race is between Contestant A, who is on foot and free to use any public transportation method; Contestant B who is driving a Mercedes S350. The roads are practically blocked as it happens to be a football match on that particular night at Cairo stadium.

The target is to travel from elkasr El3iny Street to the end of makram Abeid Street in the shortest time possible.
Contestant B drives proudly his 3500 cc car to towards the 6th October bridge and get stuck there for almost an hour. After he finishes the bridge he continues to get delayed by the traffic jam to increase his trip duration into more than two hours. Ah… and don't forget the extra 15 minutes he spent after he arrived searching for a parking spot.

Contestant A takes the subway all the way from elSydah Zeinab to eldemerdash metro station and takes a healthy-looking microbus. The microbus driver is aware of all the traffic jams in hand, so he drives through elnasr road then skips from Eltayaran Street to the middle of abbas Elakkad Street to the middle of makram Abeid Street assuming that all the microbus' passangers agreed to take that route and covers the journey in less than 30 minutes. And hence, the total duration for contestant A is 45 minutes only.
The result: Contestant A wins with the 45 minutes only and arrives earlier than contestant B with 2:15 hours.

Contestant B planned his journey on his ego, his fast car and systematic planning but he forget that each journey has its own inputs to handle
If we could describe the method Contestant A took to reach his target destination we'd call it: flexible, fast, simple and both proactive to change and incremental (as after each stage he decided which way to go and how). And that's what exactly gather up to form the meaning of the word agile.

Agile thinking is all about open-minded productivity and smart performance.


From my point of view the agile methodologies mainly revolve about certain principles:

Human development from inside out
Any agile project environment should Motivate the spirit of collaboration and team spirit all over the project members, everybody has his opinion and shares a slice of the responsibility cake.

Focus on the target all through the project
Agile teams are goal driven; a project is divided into a set of intervals (sprints is commonly used) where a part of the project is delivered and accumulatively integrated.

Best utilization of resources
The most important precondition for applying an agile methodology is involving only the people of the required skills to make sure that they can really deliver.

Customer satisfaction and word of mouth marketing
A big satisfied smile is only drawn on a customer face only if he had a tangible result in his hands, that's why the measure of quality in any agile project is through working software. Good reputation comes from sharp deliveries.

"Get it done or die trying"
Some people think that agile is the magic wand that can deliver any project with. From my personal experience, one of the biggest problems about agile here in Egypt is that people mix it with agility, because agility means rapid development but not necessarily sticking to any values. So a good advice is to make sure that you can do a thing before signing on it.

Special thanks to Passanta Ahmed for her support and information in making this article

Wednesday, 11 February 2009