A process is a collection of activities, actions, and tasks that are performed when some work product is to be created. An activity strives to achieve a broad objective (e.g., communication with stakeholders) and is applied regardless of the application domain, size of the project, complexity of the effort, or degree of rigor with which software engineering is to be applied An action (e.g., architectural design) encompasses a set of tasks that produce a major work product (e.g., an architectural model) A task focuses on a small, but well-defined objective (e.g., conducting a unit test) that produces a tangible outcome. Describes a process-related problem that is encountered during software engineering work, identifies the environment in which the problem has been encountered, and suggests one or more proven solutions to the problem In other words Provides you with a template - a consistent method for describing problem solutions within the context of the software process Template for describing a process pattern Type. The pattern type is specified. Stage pattern —defines a problem associated with a framework activity for the process. Since a framework activity encompasses multiple actions and work tasks, a stage pattern incorporates multiple task patterns (see the following) that are relevant to the stage (framework activity). An example of a stage pattern might be EstablishingCommunication . This pattern would incorporate the task pattern RequirementsGathering and others Task pattern —defines a problem associated with a software engineering action or work task and relevant to successful software engineering practice (e.g., RequirementsGathering is a task pattern). Phase pattern —define the sequence of framework activities that occurs within the process, even when the overall flow of activities is iterative in nature. An example of a phase pattern might be SpiralModel or Prototyping Initial Context. Describes the conditions under which the pattern applies. Prior to the initiation of the pattern: (1) What organizational or team-related activities have already occurred? (2) What is the entry state for the process? (3) What software engineering information or project information already exists? For example, the Planning pattern (a stage pattern) requires that (1) customers and software engineers have established a collaborative communication; (2) successful completion of a number of task patterns [specified] for the Communication pattern has occurred; and (3) the project scope, basic business requirements, and project constraints are known Problem. The specific problem to be solved by the pattern. Solution. Describes how to implement the pattern successfully. This section describes how the initial state of the process (that exists before the pattern is implemented) is modified as a consequence of the initiation of the pattern. It also describes how software engineering information or project information that is available before the initiation of the pattern is transformed as a consequence of the successful execution of the pattern. Resulting Context. Describes the conditions that will result once the pattern has been successfully implemented. Upon completion of the pattern: (1) What organizational or team-related activities must have occurred? (2) What is the exit state for the process? (3) What software engineering information or project information has been developed? Related Patterns. Provide a list of all process patterns that are directly related to this one. This may be represented as a hierarchy or in some other diagrammatic form. For example, the stage pattern Communication encompasses the task patterns: ProjectTeam, CollaborativeGuidelines, ScopeIsolation, RequirementsGathering, ConstraintDescription, and ScenarioCreation. Known Uses and Examples. Indicate the specific instances in which the pattern is applicable. For example, Communication is mandatory at the beginning of every software project, is recommended throughout the software project, and is mandatory once the Deployment activity is under way. Waterfall Model There are times when the requirements for a problem are well understood— When work flows from communication through deployment in a reasonably linear fashion This situation is sometimes encountered when well-defined adaptations or enhancements to an existing system must be made E.g. An adaptation to billing software that has been mandated because of introducing GST It may also occur in a limited number of new development efforts, but only when requirements are well defined and reasonably stable The waterfall model, sometimes called the classic life cycle , suggests a systematic, sequential approach to software development that begins with customer specification of requirements and progresses through planning, modeling, construction, and deployment, culminating in ongoing support of the completed software Problems with Waterfall Model Real projects rarely follow the sequential flow that the model proposes Although the linear model can accommodate iteration, it does so indirectly As a result, changes can cause confusion as the project team proceeds It is often difficult for the customer to state all requirements explicitly The waterfall model requires this and has difficulty accommodating the natural uncertainty that exists at the beginning of many projects

V-model A variation in the representation of the waterfall model It depicts the relationship of quality assurance actions to the actions associated with communication, modeling, and early construction activities As a software team moves down the left side of the V, basic problem requirements are refined into progressively more detailed and technical representations of the problem and its solution Once code has been generated, the team moves up the right side of the V, essentially performing a series of tests (quality assurance actions) that validate each of the models created as the team moves down the left side In reality, there is no fundamental difference between the classic life cycle and the V-model The V-model provides a way of visualizing how verification and validation actions are applied to earlier engineering work Incremental Process Models There are many situations in which initial software requirements are reasonably well defined, but the overall scope of the development effort precludes a purely linear process In addition, there may be a compelling need to provide a limited set of software functionality to users quickly and then refine and expand on that functionality in later software releases In such cases, you can choose a process model that is designed to produce the software in increments The incremental model combines the elements of linear and parallel process flows discussed earlier The incremental model applies linear sequences in a staggered fashion as calendar time progresses Each linear sequence produces deliverable “increments” of the software When an incremental model is used, the first increment is often a core product That is, basic requirements are addressed but many supplementary features (some known, others unknown) remain undelivered The core product is used by the customer (or undergoes detailed evaluation) As a result of use and/or evaluation, a plan is developed for the next increment The plan addresses the modification of the core product to better meet the needs of the customer and the delivery of additional features and functionality This process is repeated following the delivery of each increment, until the complete product is produced Evolutionary Process Models Software, like all complex systems, evolves over a period of time Business and product requirements often change as development proceeds, making a straight line path to an end product unrealistic Tight market deadlines make completion of a comprehensive software product impossible, but a limited version must be introduced to meet competitive or business pressure A set of core product or system requirements is well understood, but the details of product or system extensions have yet to be defined In these and similar situations, you need a process model that has been explicitly designed to accommodate a product that grows and changes Evolutionary models are iterative They are characterized in a manner that enables you to develop increasingly more complete versions of the software Prototyping Often, a customer defines a set of general objectives for software, but does not identify detailed requirements for functions and features In other cases, the developer may be unsure of the efficiency of an algorithm, the adaptability of an operating system, or the form that human-machine interaction should take In these, and many other situations, a prototyping paradigm may offer the best approach Although prototyping can be used as a stand-alone process model, it is more commonly used as a technique that can be implemented within the context of any one of the process models noted earlier Regardless of the manner in which it is applied, the prototyping paradigm assists you and other stakeholders to better understand what is to be built when requirements are fuzzy The prototyping paradigm begins with communication You meet with other stakeholders to define the overall objectives for the software, identify whatever requirements are known, and outline areas where further definition is mandatory A prototyping iteration is planned quickly, and modeling (in the form of a “quick design”) occurs A quick design focuses on a representation of those aspects of the software that will be visible to end users (e.g., human interface layout or output display formats). The quick design leads to the construction of a prototype The prototype is deployed and evaluated by stakeholders, who provide feedback that is used to further refine requirements Iteration occurs as the prototype is tuned to satisfy the needs of various stakeholders, while at the same time enabling you to better understand what needs to be done. Ideally, the prototype serves as a mechanism for identifying software requirements If a working prototype is to be built, you can make use of existing program fragments or apply tools that enable working programs to be generated quickly Both stakeholders and software engineers like the prototyping paradigm Users get a feel for the actual system, and developers get to build something immediately. Prototyping – Issues Stakeholders see what appears to be a working version of the software, unaware that the prototype is held together haphazardly, unaware that in the rush to get it working you haven't considered overall software quality or long-term maintainability As a software engineer, you often make implementation compromises in order to get a prototype working quickly. An inappropriate operating system or programming language may be used simply because it is available and known; an inefficient algorithm may be implemented simply to demonstrate capability The Spiral Model The spiral model is an evolutionary software process model that couples the iterative nature of prototyping with the controlled and systematic aspects of the waterfall model It provides the potential for rapid development of increasingly more complete versions of the software The spiral development model is a risk -driven process model generator that is used to guide multi-stakeholder concurrent engineering of software intensive systems It has two main distinguishing features One is a cyclic approach for incrementally growing a system’s degree of definition and implementation while decreasing its degree of risk The other is a set of anchor point milestones for ensuring stakeholder commitment to feasible and mutually satisfactory system solutions Using the spiral model, software is developed in a series of evolutionary releases During early iterations, the release might be a model or prototype During later iterations, increasingly more complete versions of the engineered system are produced A spiral model is divided into a set of framework activities defined by the software engineering team The first circuit around the spiral might result in the development of a product specification Subsequent passes around the spiral might be used to develop a prototype and then progressively more sophisticated versions of the software Each pass through the planning region results in adjustments to the project plan Cost and schedule are adjusted based on feedback derived from the customer after delivery In addition, the project manager adjusts the planned number of iterations required to complete the software The spiral model is a realistic approach to the development of large-scale systems and software Because software evolves as the process progresses, the developer and customer better understand and react to risks at each evolutionary level The spiral model uses prototyping as a risk reduction mechanism but, more important, enables you to apply the prototyping approach at any stage in the evolution of the product It maintains the systematic stepwise approach suggested by the classic life cycle but incorporates it into an iterative framework that more realistically reflects the real world The spiral model demands a direct consideration of technical risks at all stages of the project and, if properly applied, should reduce risks before they become problematic Concurrent Models The concurrent development model, sometimes called concurrent engineering, allows a software team to represent iterative and concurrent elements of any of the process models described earlier For example, early in a project the communication activity (not shown in the figure) has completed its first iteration and exists in the awaiting changes state The modeling activity (which existed in the none state while initial communication was completed) now makes a transition into the under development state If, however, the customer indicates that changes in requirements must be made, the modeling activity moves from the under development state into the awaiting changes state Concurrent modeling defines a series of events that will trigger transitions from state to state for each of the software engineering activities, actions, or tasks For example, during early stages of design (a major software engineering action that occurs during the modeling activity), an inconsistency in the requirements model is uncovered This generates the event analysis model correction, which will trigger the requirements analysis action from the done state into the awaiting changes state Concurrent modeling is applicable to all types of software development and provides an accurate picture of the current state of a project Rather than confining software engineering activities, actions, and tasks to a sequence of events, it defines a process network Each activity, action, or task on the network exists simultaneously with other activities, actions, or tasks Events generated at one point in the process network trigger transitions among the states associated with each activity The Unified Process An attempt to draw on the best features and characteristics of traditional software process models, but characterize them in a way that implements many of the best principles of agile software development The Unified Process recognizes the importance of customer communication and streamlined methods for describing the customer’s view of a system (the use case) It emphasizes the important role of software architecture and “helps the architect focus on the right goals, such as understandability, reliance to future changes, and reuse” It suggests a process flow that is iterative and incremental, providing the evolutionary feel that is essential in modern software development Inception Phase The inception phase of the UP encompasses both customer communication and planning activities By collaborating with stakeholders, business requirements for the software are identified; a rough architecture for the system is proposed; and a plan for the iterative, incremental nature of the ensuing project is developed Fundamental business requirements are described through a set of preliminary use cases that describe which features and functions each major class of users desires Architecture at this point is nothing more than a tentative outline of major subsystems and the functions and features that populate them Later, the architecture will be refined and expanded into a set of models that will represent different views of the system. Planning identifies resources, assesses major risks, defines a schedule, and establishes a basis for the phases that are to be applied as the software increment is developed The Elaboration Phase The elaboration phase encompasses the communication and modeling activities of the generic process model Elaboration refines and expands the preliminary use cases that were developed as part of the inception phase and expands the architectural representation to include five different views of the software—the use case model, the analysis model, the design model, the implementation model, and the deployment model In some cases, elaboration creates an “executable architectural baseline” that represents a “first cut” executable system The architectural baseline demonstrates the viability of the architecture but does not provide all features and functions required to use the system In addition, the plan is carefully reviewed at the culmination of the elaboration phase to ensure that scope, risks, and delivery dates remain reasonable Modifications to the plan are often made at this time

The Construction Phase The construction phase of the UP is identical to the construction activity defined for the generic software process Using the architectural model as input, the construction phase develops or acquires the software components that will make each use case operational for end users To accomplish this, analysis and design models that were started during the elaboration phase are completed to reflect the final version of the software increment All necessary and required features and functions for the software increment (i.e., the release) are then implemented in source code As components are being implemented, unit tests are designed and executed for each In addition, integration activities (component assembly and integration testing) are conducted Use cases are used to derive a suite of acceptance tests that are executed prior to the initiation of the next UP phase

The Transition Phase The transition phase of the UP encompasses the latter stages of the generic construction activity and the first part of the generic deployment (delivery and feedback) activity Software is given to end users for beta testing, and user feedback reports both defects and necessary changes In addition, the software team creates the necessary support information (e.g., user manuals, troubleshooting guides, installation procedures) that is required for the release At the conclusion of the transition phase, the software increment becomes a usable software release

The Production Phase The production phase of the UP coincides with the deployment activity of the generic process During this phase, the ongoing use of the software is monitored, support for the operating environment (infrastructure) is provided, and defect reports and requests for changes are submitted and evaluated Requirements Engineering Building software is so compelling that many software developers want to jump right in before they have a clear understanding of what is needed They argue that things will become clear as they build, that project stakeholders will be able to understand need only after examining early iterations of the software, that things change so rapidly that any attempt to understand requirements in detail is a waste of time, that the bottom line is producing a working program, and that all else is secondary But each argument is flawed and can lead to a failed software project The broad spectrum of tasks and techniques that lead to an understanding of requirements is called requirements engineering From a software process perspective, requirements engineering is a major software engineering action that begins during the communication activity and continues into the modeling activity It must be adapted to the needs of the process, the project, the product, and the people doing the work Requirements engineering builds a bridge to design and construction Requirements engineering encompasses seven distinct tasks Inception Elicitation Elaboration Negotiation Specification validation and Management It is important to note that some of these tasks occur in parallel and all are adapted to the needs of the project Inception How does a software project get started? Is there a single event that becomes the catalyst for a new computer-based system or product, or does the need evolve over time? There are no definitive answers to these questions At project inception, you establish a basic understanding of the problem, the people who want a solution, the nature of the solution that is desired, and the effectiveness of preliminary communication and collaboration between the other stakeholders and the software team Elicitation An important part of elicitation is to establish business goals Your job is to engage stakeholders and to encourage them to share their goals honestly Once the goals have been captured, a prioritization mechanism should be established, and a design rationale for a potential architecture (that meets stakeholder goals) can be created A number of problems that are encountered as elicitation occurs: Problems of scope occur when the boundary of the system is ill-defined or the customers and users specify unnecessary technical detail that may confuse, rather than clarify, overall system objectives Problems of understanding are encountered when customers and users are not completely sure of what is needed, have a poor understanding of the capabilities and limitations of their computing environment, don’t have a full understanding of the problem domain, have trouble communicating needs, omit information that is believed to be “obvious,” specify requirements that conflict with the needs of other customers and users, or specify requirements that are ambiguous or untestable Problems of volatility occur when the requirements change over time. To help overcome these problems, you must approach the requirements-gathering activity in an organized manner Elaboration The information obtained from the customer during inception and elicitation is expanded and refined during elaboration Elaboration is driven by the creation and refinement of user scenarios that describe how the end user (and other actors) will interact with the system Each user scenario is parsed to extract analysis classes—business domain entities that are visible to the end user The attributes of each analysis class are defined, and the services that are required by each class are identified The relationships and collaboration between classes are identified, and a variety of supplementary diagrams are produced Negotiation It isn’t unusual for customers and users to ask for more than can be achieved, given limited business resources It’s also relatively common for different customers or users to propose conflicting requirements, arguing that their version is “essential for our special needs.” You have to reconcile these conflicts through a process of negotiation Customers, users, and other stakeholders are asked to rank requirements and then discuss conflicts in priority Using an iterative approach that prioritizes requirements, assesses their cost and risk, and addresses internal conflicts, requirements are eliminated, combined, and/or modified so that each party achieves some measure of satisfaction Specification In the context of computer-based systems (and software), the term specification means different things to different people A specification can be a written document, a set of graphical models, a formal mathematical model, a collection of usage scenarios, a prototype, or any combination of these It is sometimes necessary to remain flexible when a specification is to be developed Validation The work products produced as a consequence of requirements engineering are assessed for quality during a validation step Requirements validation examines the specification to ensure that all software requirements have been stated unambiguously; that inconsistencies, omissions, and errors have been detected and corrected; and that the work products conform to the standards established for the process, the project, and the product The primary requirements validation mechanism is the technical review Management Requirements for computer-based systems change, and the desire to change requirements persists throughout the life of the system Requirements management is a set of activities that help the project team identify, control, and track requirements and changes to requirements at any time as the project proceeds Eliciting Requirement Approach for eliciting requirement: Collaborative Requirements Gathering Quality Function Deployment User Scenarios Elicitation Work Products Collaborative Requirement Gathering Meetings are attended by all interested stakeholders. Rules established for preparation and participation. Agenda should be formal enough to cover all important points, but informal enough to encourage the free flow of ideas. A facilitator controls the meeting. A definition mechanism (blackboard, flip charts, etc.) is used. During the meeting: The problem is identified. Elements of the solution are proposed. Different approaches are negotiated. A preliminary set of solution requirements are obtained. The atmosphere is collaborative and non-threatening. Flow of event – Outline the sequence of events occurs
Requirement gathering meeting ( initial meeting) During meeting Follow the meeting. In initial meeting, distribute “Product request” (defined by stakeholder) to all attendee. Based on product request, each attendee is asked to make List of objects (Internal or external system objects) List of services( Processes or functions) List of constraints ( cost, size, business rules) and performance criteria( speed, accuracy) are developed. Collect lists from everyone and combine Combined list eliminates redundant entries, add new ideas , but does not delete anything Objective is to develop a consensus list in each topic area (objects, services, constraints and performance) Based on lists, team is divided into smaller sub-teams : each works to develop mini-specification for one or more entries on each of the lists Each sub-team presents its mini-specification to all attendees for discussion. Addition, deletion and further elaboration are made Now each team makes a list of validation criteria for the product and present to team Finally, one or more participants is assigned the task of writing a complete draft specification Elicitation Work Products Elicitation work product will vary depending upon the size of the system or product to be built. Statement of need and feasibility. Statement of scope. List of participants in requirements elicitation. Description of the system’s technical environment. List of requirements and associated domain constraints. List of usage scenarios. Any prototypes developed to refine requirements. Agile Requirements Elicitation Within the context of an agile process, requirements are elicited by asking all stakeholders to create user stories Each user story describes a simple system requirement written from the user’s perspective User stories can be written on small note cards, making it easy for developers to select and manage a subset of requirements to implement for the next product increment Proponents claim that using note cards written in the user’s own language allows developers to shift their focus to communication with stakeholders on the selected requirements rather than their own agenda Although the agile approach to requirements elicitation is attractive for many software teams, critics argue that a consideration of overall business goals and nonfunctional requirements is often lacking In some cases, rework is required to accommodate performance and security issues In addition, user stories may not provide a sufficient basis for system evolution over time 8 Steps to successful story point estimation

1. Identify base stories. It is very important to identify one or multiple base or reference story against which you would do relative sizing of the backlog. This story is picked from current product backlog or a different story which we have done earlier. But what is important is the understanding of this story is same among everyone on the team. Team should be confident of this base story.

2. Talk through the requirements of the story. Product Owner or a business analyst will answer questions and provide explanation about what exactly this story entails.

3. Discuss and jot down things you want to remember when implementing this story. These can be bullet points on the story card or text in the “notes” section of a tool. This is best done by Scrum Master who can add these details as and when discussions are on.

1. Some of these questions team ask themselves when they start sizing. . Design: What will we have to learn before we can start work on this story?
2. Coding which include tests: How much code will need to be written for this story? Have we written similar code before?
3. Acceptance Testing: How much work is involved in helping the customer to automate the acceptance tests for this story?
4. Integration Points: Does this story have external dependencies?
5. Expertise: Does anyone of us have done similar story before?
6. Complexity: Is it a straightforward story or it includes complexity either from business logic perspective or from technical perspective

7. Uncertainty/risk: How certain we are for instance in getting the dependencies in time or test data we requested.

8. Find some point of relative comparison. If this story is about the same amount of work as one you have already sized, give it the same number of points. If it is more difficult, give it a proportionally higher value. If this story is similar to another but less complex because of the learning from previous story, give it a lower value.

9. Reach a consensus among entire team present as to the size of the story as per Definition of Done. \

10. Validate that your estimates are internally consistent among stories as you go along.

11. Periodically ensure that all of the 1’s are about the same, all of the 2’s match, etc. Likewise, the team should agree that a five-point story is roughly twice as much work as a two-point story. In story point estimates, it does not matter if your estimates are correct or incorrect as long as team arrives to a shared understanding on a user story.

what is sprint? In Agile software development, a sprint is a time-boxed iteration of the development process during which a team works on a set of features or user stories. Sprints typically last between one and four weeks, with two weeks being a common duration, and they are designed to provide a focused period of work in which the team can deliver a working increment of the software.

Sprint planning meeting In Agile software development, a sprint planning meeting is a collaborative meeting between the development team and the product owner that takes place at the beginning of each sprint. The goal of the sprint planning meeting is to plan the work that will be completed during the upcoming sprint.

During the sprint planning meeting, the team and the product owner work together to identify and prioritize the user stories or features that will be developed during the sprint. The team then estimates the effort required to complete each user story or feature, based on their past experience and knowledge of the system.

Sprint backlog In Agile software development, a sprint backlog is a list of all the user stories or features that the development team has committed to completing during the upcoming sprint. The sprint backlog is created during the sprint planning meeting, and it serves as a guide for the team's work during the sprint.

The sprint backlog typically includes the following information for each user story or feature:

User story or feature: A brief description of the user story or feature that will be developed during the sprint.

Estimated effort: An estimate of the effort required to complete the user story or feature, typically in story points or hours.

Priority: The priority of the user story or feature, as determined by the product owner.

Status: The current status of the user story or feature, including whether it has been started, in progress, or completed.

Assignee: The team member who is responsible for completing the user story or feature.

The sprint backlog is a living document that is updated throughout the sprint as the team completes work and new tasks or issues arise.

What is Agile Testing? A software testing practice that follows the principles of agile software development is called Agile Testing. Agile is an iterative development methodology, where requirements evolve through collaboration between the customer and self-organizing teams and agile aligns development with customer needs.

Stage 1 During first stage or iteration 0, you perform initial setup tasks. It includes identifying people for testing, installing testing tools, scheduling resources (usability testing lab), etc. The following steps are set to achieve in Iteration 0: a) Establish the boundary conditions and the testing scope b) Outline the key requirements and use cases that will drive the design trade-offs c) Outline Test case architectures d) Identifying the risk

Stage 2 The second phase of testing is Construction Iterations, the majority of the testing occurs during this phase. In construction iteration, agile team follows the prioritized requirement practice: With each iteration they take the most essential requirements remaining from the work item stack and implement them. Construction iteration is classified into two, confirmatory testing and investigative testing. Confirmatory testing concentrates on verifying that the system fulfills the intent of the stakeholders as described to the team to date, and is performed by the team. In Investigative testing, tester determines the potential problems in the form of defect stories. Investigative testing deals with common issues like integration testing, load/stress testing and security testing.

Stage 3 The goal of “Release, End Game” is to deploy your system successfully into production. The activities include in this phase are training of end users, support people and operational people. The final testing stage includes full system testing and acceptance testing. In accordance to finish your final testing stage without any obstacles, you should have to test the product more accurately while it is in construction iterations.

Testing fundamentals The goal of testing is to find errors, and a good test is one that has a high probability of finding an error Therefore, you should design and implement a computer-based system or a product with “testability” in mind At the same time, the tests themselves must exhibit a set of characteristics that achieve the goal of finding the most errors with a minimum of effort Testability - “ Software testability is simply how easily [a computer program] can be tested.”

Testability Characteristics Testability Operability “The better it works, the more efficiently it can be tested.” Observability “What you see is what you test.” Controllability “The better we can control the software, the more the testing can be automated and optimized.” Decomposability “By controlling the scope of testing, we can more quickly isolate problems and perform smarter retesting.” Simplicity “The less there is to test, the more quickly we can test it.” Stability “The fewer the changes, the fewer the disruptions to testing.” Understandability “The more information we have, the smarter we will test.”

A good test has a high probability of finding an error

A good test is not redundant

A good test should be “best of breed”

A good test should be neither too simple nor too complex

Principles of Agile Testing Testing Moves the Project Forward Testing is NOT a Phase Every one Tests Reduce Feedback Loop Keep the Code Clean Lightweight Documentation Tested is part of Done

Types Of Performance Testing LOAD TESTING STRESS TESTING VOLUME TESTING SECURITY TESTING RECOVERY TESTING

Performance Testing Process planning:Determine the performance testing objectives Describe the application to test using a application model

Describe the Hardware environment Create a Benchmark (Agenda) to be recorded in Phase 2.

Define what tasks each user will perform Define (or estimate) the percentage of users per task.

RECORD RECORD the defined testing activities that will be used as a foundation for your load test scripts. One activity per task or multiple activities depending on user task definition MODIFY MODIFY load test scripts defined by recorder to reflect more realistic Load test simulations. Defining the project, users Randomize parameters (Data, times, environment) Randomize user activities that occur during the load test

Advantages of Agile Testing Agile Testing Saves Time and Money Less Documentation Regular feedback from the end user Daily meetings can help to determine the issues well in advance

Challenges in Agile Testing Requirement changes and updates are inherent in an agile method, becoming the biggest challenge for QA Testers are often required to play a semi-developer role New features are introduced quickly, which reduces the available time for test teams to identify whether the latest features are according to the requirement and does it truly address the business suits Test execution cycles are highly compressed Very less time to prepare test plan For regression testing, they will have minimal timing

Agile testing Quadrants Quadrant 1: ❑ Technology-facing tests that support the team ▪ Quadrant 2: ❑ Business-facing tests that support the team ▪ Quadrant 3: ❑ Business-facing tests that critique the product ▪ Quadrant 4: ❑ Technology-facing tests that critique the product ▪ Planning your strategy

Goal of Quadrant One Test 19CSE314 Software Engineering Agile Testing Quadrants Dr C.Arunkumar Amrita Vishwa Vidyapeetham, Coimbatore Assistant Professor (SG), Dept of Computer Science and Engg Agenda ▪ Overview of Quadrants ❑ Purpose of testing ▪ Quadrant 1: ❑ Technology-facing tests that support the team ▪ Quadrant 2: ❑ Business-facing tests that support the team ▪ Quadrant 3: ❑ Business-facing tests that critique the product ▪ Quadrant 4: ❑ Technology-facing tests that critique the product ▪ Planning your strategy 2 Levels of Planning – Product, Release, Iteration 3 The Agile Testing Quadrants idea .com Using the Quadrants ▪ Quadrants help ensure we accomplish all goals ❑ Support team ❑ Critique product ❑ Ensure business needs met ❑ Ensure technological needs met ▪ Shared responsibility ❑ Special skills may be needed ❑ Focus on collaboration 5 Quadrant 1 6 Goal of Quadrant One Tests Testability ▪ Layered or “componentized” ❑ APIs, Ports and Adapters ▪ Test database access, updates ▪ Business logic and presentation separated ▪ Isolate tests ❑ allows isolating problems ▪ Internal quality ▪ Infrastructure 7 Quadrant OneTest Benefits ▪ Go faster, do more ❑ Unit tests provide safety net ❑ Refactoring support ▪ Improve design & maintainability without changing functionality ▪ Quality, not speed, is goal ▪ Courage ▪ Confidence in design 8 What, Who, When 9 ▪ Unit Tests ❑ Developer intent – program design ❑ Small piece of code does what it should ▪ Component Tests ❑ Architect intent – system design ❑ Components work together correctly ▪ Programmer tests/codes ▪ Continually refactor ▪ Run in CI If Your Team Doesn't Do These … ▪ It's a team problem ▪ Find areas of greatest pain ▪ Testers writing unit tests isn't the answer ▪ Managers must provide time to learn ▪ Without Quadrant One, ❑ the other quadrants will be much harder 1 0 Quadrant One Toolkit ▪ Source code management ❑ Version control ❑ Know what has been changed, by whom ❑ Be able to restore earlier version ▪ Integrated development environment ❑ compile, debug, build GUI, refactor ❑ eg. Eclipse, IntelliJ Idea, NetBeans ▪ Build/CI tools ❑ eg. CruiseControl, Hudson. TeamCity ▪ Unit test tools ❑ xUnit ❑ Mocking tools 1 1 Quadrant 2 1 2 Purpose of Quadrant Two ▪ Drive development with business-facing tests ▪ Ask the right questions ▪ Help customers achieve advance clarity ▪ Capture examples, express as executable tests ▪ External quality ▪ Know when we're done 1 3 Who Does Quadrant 2 Tests, When? ▪ Testers have special expertise ▪ Collaboration with customers ▪ Team responsibility ❑ Programmers ❑ DBAs, analysts, ... ▪ Start of iteration ❑ Business-facing tests drive development ▪ Throughout iteration ❑ No story done until tested 1 4 Toolkit – Eliciting Requirements ▪ Checklists ▪ Mind maps ❑ brainstorming ❑ words, ideas, tasks ▪ Mockups / paper prototypes ❑ User-centered design ▪ Flow diagrams ▪ Whiteboards (physical and virtual) ▪ Thin slice/steel thread exercise ▪ Behavior-driven development 1 5 quadrant-3 Evaluating the Product ▪ Recreate actual user experiences ▪ Realistic use ▪ Learn as you test ▪ Context ❑ What works for your situation ❑ “It depends” ❑ A tool, not a rule ▪ Constructive

Demos with Customers ▪ Iteration reviews ❑ Builds confidence ❑ Quick feedback loop ▪ Informal demos ❑ Pair exploratory testing with customer ❑ Even on unfinished code

Exploratory Testing ▪ Simultaneous learning, test design, test execution [source: James Bach] ❑ “Doing” reveals more than “thinking” ▪ Careful observation ▪ Critical thinking ▪ Diverse ideas ▪ Rich resources ❑ Tools, data, people [source: Jon Hagar]

Other Types of Testing ▪ Scenario testing ❑ Process flows ❑ Realistic data ❑ Soap opera testing (Hans Buwalda) ▪ Usability testing ❑ Personas ❑ Navigation ❑ Observing users ▪ Don't forget documents, reports, help text

Behind the GUI ▪ API testing ❑ Inputs and outputs ❑ Sequence of API calls ❑ Checking log files ❑ Example: Test parsing of upload file ❑ Example: Test shipping cost calculation ❑ States and transitions ▪ Web Services ❑ External customers ❑ Levels of service ❑ Validate definitions against profiles ❑ Validate requests and responses Feedback to Tests that Support Team ▪ Discuss with technical, customer team ▪ Turn what you learn into tests that drive new features ▪ Change process as needed 2 3 Who Does Quadrant 3 Tests, When ▪ Requires good skills, experience, intuition, critical thinking ▪ Involve the customers ▪ Programmers help with tools to facilitate ▪ Do as early as possible 2 4 Quadrant Three Toolkit Tool Strategy ▪ Consider who uses tests, who writes and maintains tests ▪ Quadrant 2 tools may apply ▪ Take time to research, experiment 2 5 Tools for Exploratory Testing ▪ Test scenario setup ❑ eg. Watir/Watij scripts ▪ Generate test data ❑ eg. PerlClip, Ruby script ▪ Simulators ❑ Simulate data, feed to app over time ▪ Monitors ❑ Watch log files ▪ Emulators ❑ Duplicate system behavior ▪ eg. mobile devices 2 6 Quadrant 4

Quadrant Four Tests ▪ Performance ❑ How fast? Identify bottlenecks ▪ Stability ❑ How long? ▪ Reliability ❑ How often? ▪ Scalability ❑ How much? ▪ Maintainability,compatibility, installability…

More Quadrant Four Tests ▪ Memory management ❑ Issues such as leaks ▪ Data migration ❑ Conversion scripts ▪ Recovery ❑ Failover testing ▪ Test environments ❑ Independent, production-like ▪ Baselines ▪ Write stories for these types of tests Converted Data Old Data

Who Does Quadrant 4 Tests, When?

▪ Depends on priorities ▪ May need from start ▪ May need to test scalability early ▪ It pays to get a baseline ▪ Programmers can write multiple-thread harnesses at unit level ▪ Plan for specialists as needed ▪ Team responsibility Quadrant Four Automation

▪ Write stories to evaluate tools ▪ Specialists? ▪ Training in Quadrant Four testing skills Quadrant Four Automation ▪ Write stories to evaluate tools ▪ Specialists? ▪ Training in Quadrant Four testing skills

Quadrant Four Automation ▪ Native database tools ❑ SQL, data import tools ▪ Shell scripting ▪ Monitoring tools examples ❑ jConsole ▪ Application bottlenecks, memory leaks ❑ jProfiler ▪ Database and bean usage

More Quadrant Four Tools ▪ Commercial load test tools ❑ Loadrunner ❑ Silk Performer ▪ Open source test tools ❑ jMeter ❑ The Grinder ❑ jUnitPerf ▪ Performance test providers ❑ Multiple sites

Doneness ▪ No story is done until testing complete ▪ Automated regression tests ▪ Customer requirements captured as passing tests ▪ Delivers value ▪ Doneness in all quadrants

Planning Your Test Strategy ▪ Scope ▪ Priorities, risks ▪ Tools that solve the problem ▪ Customers ▪ Document only what is useful ▪ Consider all four quadrants ▪ Use lessons learned to improve

Advantages of Agile Testing Agile Testing Saves Time and Money Less Documentation Regular feedback from the end user Daily meetings can help to determine the issues well in advance

Challenges in Agile Testing Requirement changes and updates are inherent in an agile method, becoming the biggest challenge for QA Testers are often required to play a semi-developer role New features are introduced quickly, which reduces the available time for test teams to identify whether the latest features are according to the requirement and does it truly address the business suits Test execution cycles are highly compressed Very less time to prepare test plan For regression testing, they will have minimal timing

Software Design • Software design encompasses the set of principles, concepts, and practices that lead to the development of a high-quality system or product • Design principles establish an overriding philosophy that guides the design work you must perform • Design concepts must be understood before the mechanics of design practice are applied, and design practice itself leads to the creation of various representations of the software that serve as a guide for the construction activity that follows • Design is pivotal to successful software engineering

Data/Class design • The data/class design transforms class models (Chapter 10) into design class realizations and the requisite data structures required to implement the software • The objects and relationships defined in the CRC diagram and the detailed data content depicted by class attributes and other notation provide the basis for the data design activity • Part of class design may occur in conjunction with the design of software architecture • More detailed class design occurs as each software component is designed

Architectural design • The architectural design defines the relationship between major structural elements of the software, the architectural styles and patterns (Chapter 13 that can be used to achieve the requirements defined for the system, and the constraints that affect the way in which architecture can be implemented • The architectural design representation—the framework of a computer-based system—is derived from the requirements model

Interface Design • The interface design describes how the software communicates with systems that interoperate with it, and with humans who use it • An interface implies a flow of information (e.g., data and/or control) and a specific type of behavior • Therefore, usage scenarios and behavioral models provide much of the information required for interface design

The design process • Software design is an iterative process through which requirements are translated into a “blueprint” for constructing the software • Initially, the blueprint depicts a holistic view of software

Software Quality Guidelines and Attributes Three characteristics of a good design • The design should implement all of the explicit requirements contained in the requirements model, and it must accommodate all of the implicit requirements desired by stakeholders • The design should be a readable, understandable guide for those who generate code and for those who test and subsequently support the software • The design should provide a complete picture of the software, addressing the data, functional, and behavioral domains from an implementation perspective

Quality Guidelines • In order to evaluate the quality of a design representation, you and other members of the software team must establish technical criteria for good design

1. A design should exhibit an architecture that (1) has been created using recognizable architectural styles or patterns, (2) is composed of components that exhibit good design characteristics, and (3) can be implemented in an evolutionary fashion, thereby facilitating implementation and testing
2. A design should be modular; that is, the software should be logically partitioned into elements or subsystems
3. A design should contain distinct representations of data, architecture, interfaces, and components A design should lead to data structures that are appropriate for the classes to be implemented and are drawn from recognizable data patterns
4. A design should lead to components that exhibit independent functional characteristics
5. A design should lead to interfaces that reduce the complexity of connections between components and with the external environment
6. A design should be derived using a repeatable method that is driven by information obtained during software requirements analysis
7. A design should be represented using a notation that effectively communicates its meaning

Quality Attributes - FURPS Functionality • is assessed byvaluating thefeature set andcapabilities ofthe program,the generalityof the functions that are delivered, andthe security ofthe overallsystem. Usability • is assessed byconsideringhuman factors,overallaesthetics,consistency,anddocumentation Reliability • is evaluated bymeasuring thefrequency andseverity offailure, theaccuracy ofoutput results,the mean-timeto-failure(MTTF), theability torecover fromfailure, and the predictability ofthe program Performance • is measuredusingprocessing speed,response time,resource consumption, throughput,and efficiency Supportability • combines extensibility,adaptability,and serviceabilitY

DEVOPS DevOps is a software development methodology that combines software development (Dev) and IT operations (Ops) in order to improve the speed, quality, and reliability of software delivery. DevOps is a cultural shift towards collaboration and communication between development and operations teams, with the goal of building, testing, and releasing software more quickly and efficiently.

The DevOps approach emphasizes automation and continuous delivery, with the goal of reducing manual processes and minimizing errors. DevOps teams typically use a range of tools and technologies to automate the software delivery process, including version control systems, continuous integration and continuous delivery (CI/CD) pipelines, configuration management tools, and monitoring and logging systems.

DevOps also emphasizes a focus on delivering value to users and stakeholders, with a strong emphasis on feedback and collaboration. DevOps teams work closely with product owners and other stakeholders to ensure that software is delivered on time, meets user needs, and is of high quality.

Some key benefits of the DevOps approach include:

Faster time-to-market: By automating the software delivery process and breaking down silos between development and operations teams, DevOps can help organizations deliver software more quickly and efficiently.

Improved quality: By emphasizing automation and continuous delivery, DevOps can help organizations improve the quality and reliability of their software.

Greater agility: By emphasizing collaboration and feedback, DevOps can help organizations be more responsive to changing business needs and user requirements.

Increased efficiency: By automating manual processes and eliminating wasteful activities, DevOps can help organizations reduce costs and increase efficiency.

5things devops is not DevOps is not simply combining Development & Operations teams

1. DevOps is not a separate team
2. DevOps is not a tool
3. DevOps is not a one-size-fits-all strategy
4. DevOps is not just automation

devops and software development life cycle  The DevOps Lifecycle Looks Like This:  Check in code  Pull code changes for build  Run tests (continuous integration server to generate builds & arrange releases): Test individual models, run integration tests, and run user acceptance tests.  Store artifacts and build repository (repository for storing artifacts, results & releases)  Deploy and release (release automation product to deploy apps)  Configure environment  Update databases  Update apps  Push to users – who receive tested app updates frequently and without interruption  Application & Network Performance Monitoring (preventive safeguard)  Rinse and repeat

main objectives of devops  Installation of server hardware and OS  Configuration of servers, networks, storage, etc…  Monitoring of servers  Continous Integration  Continous Delivery  Respond to outages  IT security  Change control  Backup and disaster recovery planning  Automating and Orchestrating entire process

CI/CD  Continuous integration (CI) is a software engineering practice in which isolated changes are immediately tested and reported on when they are added to a larger code base. The goal of CI is to provide rapid feedback so that if a defect is introduced into the code base, it can be identified and corrected as soon as possible.  Time frames are crucial. Integration should be divided into three steps:  commit new functionality and build new application  run unit tests  run Integration/System tests

Continuos release and deployment The relevant terms here are “Continuous Integration” and “Continuous Deployment”, often used together and abbreviated as CI/CD . Originally Continuous Integration means that you run your “integration tests” at every code change while Continuous Delivery means that you automatically deploy every change that passes your tests.  The Software Development Pipeline  From a high level, a CI/CD pipeline usually consists of the following discrete steps:  Commit. When a developer finishes a change to an application, he or she commits it to a central source code repository.  Build. The change is checked out from the repository and the software is built so that it can be run by a computer. This steps depends a lot on what language is used and for interpreted languages this step can even be absent.  Automated tests. This is where the meat of the CI/CD pipeline is. The change is tested from multiple angles to ensure it works and that it doesn’t break anything else.  Deploy. The built version is deployed to production.

tools in devops Ansible  Scripting  Jenkins  Chef  Git  Nexus  Docker  Nagios  Vagrant….et C

Overall, the DevOps approach is a powerful tool for organizations looking to improve the speed, quality, and reliability of their software delivery, while also fostering a culture of collaboration and continuous improvement.

Story-Based Burn Down Scrum Team Pains 2 Scrum team (at least our team) pains:  Obsesses over sprint planning, especially the task estimates – Long, boring meetings – Big estimation gap between different individuals  Hard to measure true feature completion using burn down chart – # of hours spent cannot represent # of feature completed – Team still concerned about the commitment even if the burn down looks good  Hard to measure the true velocity – If we measure velocity by actual hours the team spent, possibly the velocity will be pretty high even if nothing is done – Hours look more like “capacity” rather than velocity

Situation in our Sprint # 1

1. We spent 6 hours in our 2-week sprint’s planning meeting
2. The burn down chart looked good but the feature completion was bad
3. We used actual hours to measure velocity, but by the end of sprint # 1 we were not confident to use that velocity to make the commitment for the next sprint

Key Problems we want to resolve Seems all the problems are related to “hours”:  Hard to use hours to size the stories/tasks  Hard to associate the time-based burn down with the feature completion – Using actual hours to measure true velocity is not the best solution – how many hours the team can burn has no direct relationship between the commitment which is the final delivery

Our solution for resolving the issues  Story Points is a better choice for the estimates – Published research shows that humans are terrible at estimating anything in hours ¹ – The tasks which one team/individual can finish in 10 hours may cost 100 hours for another team/individual – Story Points is the absolute value for the size of the Product Backlog Items (PBIs)  Task-based burn down could represent true completion – No direct relationship between time spent and feature completion – Task-based burn down should be more meaningful – Key dependency is a clear “definition of done” for the tasks  Use story points to measure the velocity – Velocity is how many story points a team can deliver in one sprint, not how many hours a team can burn – We can use “Story Points finished/sprint” as the formula when calculating the velocity

Improvement Plan for Sprint # 2 6 Action Description

1. Improve the Task Break Down 1) Decompose the final deliverables into small pieces, create one task for each small piece 1) Define "done" for each task - ensure the completion of each task means the delivery of one potential shippable product
2. Use Story Points for estimation 1) Select one “ template story” as the standard criteria 1) Get the story points for each task based on the comparison with the template story, but still estimate how many ideal hours we need to finish that task
3. Use the Task-based Burn Down for tracking 1) Start to use Task-based Burn Down (still use Time-based burn down at the same time) 1) Record the actual hours we spent to finish the tasks
4. Measure velocity using story points finished/sprint 1) After we get stable team velocity, then stop doing estimates using hours, stop using actual hours to measure velocity

Benefit of using new approach  In general, the task-based burn down matches the feature completion – Which means the chart can represent the real velocity

New challenges  Task-based burn down is still not good enough to represent feature completion – Tasks are too detailed - the # of tasks kept increasing, it’s hard (and not necessary) to identify all the related tasks to complete one story in sprint planning meeting – Too many dependencies between different tasks and also we had lots of generic tasks like document writing/regression testing

Our solution for the new challenges 10  Use story-based burn down instead of task-based burn down – Tasks are so detailed that sometimes we can not say they are potential shippable deliverables – But obviously one story is a potential shippable deliverable – Story-based burn down and task-based burn down are essentially the same

Continue to improve the Task Break Down

1. Scale down the size of each story, use “stories” to represent small features, e.g., small enough for one developer to finish in 2-3 days
2. Need a clear definition of done for each story – usually use passing FT/ UAT without Major and above defects as the exit criteria Estimate the size of each story • Use story points to estimate the size of each story, but DON’T do estimate for all the tasks Use the story-based Burn Down for tracking
3. Still use 2 burn down charts at the same time: story-based burn down and effort-based burn down. • Only deduct the estimated story points when the corresponding stories are Completely Finished – meet the definition of done
4. Record the actual hours only for reference, and calculate how many stories (or story points) team can finish in one sprint as the true velocity

What Is Velocity?  Velocity is the amountof workcompletedeach sprint.  It is measured by adding the sizes of the PBIs that are completed by the end of the sprint.  A PBI is eitherdoneorit’s notdone.  The product owner doesn’t get any value from undone items, so velocity does not include the size numbers of partiallycompleted PBIs. What Is Velocity? (contd.)  Velocity measures output (the size of what was delivered), not outcome (thevalue of what was delivered).  We assume that if the product owner has agreed that the team should work on a PBI, it must have some value to him.  However, completing a PBI of size 8 doesn’t necessarily deliver more business value than completing a PBI of size 3.  Perhaps the PBI of size 3 is high value and therefore we work on it early (because it is high value and low cost), and we work on the PBI of size 8 later (because it is lower value and highercost). What Is Velocity? (contd.)  Velocity isused fortwo importantpurposes.  First, it is an essential conceptfor Scrum planning.  For release-level planning, we divide the size of the release by the team’s average velocity to calculate the numberof sprints necessary tocomplete the release.  Additionally, at sprint planning, a team’s velocity is used as one input to help determine its capacity to commit to work during the upcoming sprint.

Misusing Velocity  Velocity is used as a planning tool and as a team diagnostic metric.  It should not be used as a performance metric in an attemptto judge teamproductivity.  When misused in this way, velocity can motivate wasteful and dangerousbehavior.  For example, say We have decided to give the largest bonus to the team that has the highestvelocity.  Superficially this idea might seem sensible; the team with the highest velocity must be getting the most work done each sprint, right? So, why not reward that behavior? Misusing Velocity (contd.)  Well, if I’m comparing teams that aren’t sizing their PBIs using a common baseline (which is very likely true), comparing the numberswould make no sense.  Let’s say that team A assigns a value of 5 to a PBI, whereas team B assignsa valueof 50 to the same PBI.  Team A doesn’t really want me to compare its velocity againstteam B’ svelocity.  Team B’s velocity will be ten times that of team A, even if both teams actually get about the same quantity of workcompletedeachsprint. Misusing Velocity (contd.)  Once team A sees the problem, its members will start to game the system to ensure that their velocity numbers are higher.  The easy way to do this is to just change the scale the team uses to estimate PBIs.  So, team A now sizes the same item (the one it originally sized a 5) to be a 500.  We call this behavior point inflation, and it serves no purpose other than to align a team’s behavior with a misguided measurement system.  Don ’t do this. Misusing Velocity (contd.)  If we set up the reward system to favor bigger numbers, that’s exactly what I’ll get—bigger numbers (point inflation).  Even worse than point inflation is when teams cut corners to get more “done” in an effort to achieve higher, more desirable velocities. Doing so leads to increasingly greater levels of technical debt.  At the end of the day, we should judge velocity on how well it assists us with performing accurate planning and how well it helps a team to internally improveitself.  Anyother uses will likely promote thewrong behavior.