NFR to test

Everyone knows about functional requirements & testing. However, Non Functional Requirements (NFR) appear to be the key to lovable products, especially if there are competitors on the market.

Culture influences a lot NFRs [Moustier 2019-1] [Moustier 2020]:

Banking would promote security a lot

Gaming is focussed a lot on the User Experience

Medical devices would also provide a bunch of NFR, should it be on materials, ways of working, robustness, safety to people, …

As a reference, the [ISO 25010 2011] is commonly cited. This standard is based on eight “characteristics”:

Portability : Adaptability ; Installability ; Replaceability

Maintainability : Modularity ; Reusability ; Analyzability ; Modifiability ; Testability

Security : Confidentiality ; Integrity ; Non-repudiation ; Accountability ; Authenticity

Reliability : Maturity ; Availability ; Fault Tolerance ; Recoverability

Functional Suitability : Functional Completeness, Correctness, Appropriateness

Performance Efficiency : Time Behavior ; Resource Utilization ; Capacity

Compatibility : Co-existence ; Interoperability

Usability : Appropriateness Recognisability ; Learnability ; Operability ; User Error Protection ; User Interface Aesthetics ; Accessibility

There are different kinds of NFR to test that can be expanded with more characteristics. It takes some time to take care of NFRs. Google’s SRE, DevOps vision of Google forecasts that Developers should not spend more than 50% of their time on new features [Beyer 2016]. The rest of their time is assigned to help Ops, that is to say operability and maintainability matters. To help gouvernance, SRE introduces the “ Error Budget ” to help the Product Owner (PO) to balance new features with technical debt.

NFR also strongly impacts architecture. These so-called “Architecturally Significant Requirements” (ASR) [Chen 2013] should then be known as soon as possible to limit deep refactorings.

Discovering NRF is a matter of:

culture of groups and individuals

experience on the business and End Users’ groundfield that can be gained via Gemba walks or Design Thinking

Actually, NFRs strongly represent the culture of the organization; for instance, companies that would care a lot about providing “green products” as their baseline should include testing how much a given part should also be green. It would be a shame if some part would reject tons of CO2 or would involve highly toxic materials in their process...

Handling NFR is something that should impact every activity in the Software Development Life Cycle. In an agile configuration, NFR should not be treated in a staggered approach after the whole release is ready. This matter should be handled “continuously”, that is to say it should included during any activity; for instance,

SAFe introduces NFR as part of the description of an epic [SAFe 2021-34]

Architecture should take ASRs into account as early as possible [Moustier 2019-1]

The four quadrants of testing includes two quadrants on NFR [Crispin 2021]

This is why NFR impacts both DoR and DoD ; however, handling any kind of NFR at once is merely impossible from the start and would make no sense for a MVP to support 1M Users from the start if the product is not to be insanely viral on the mass market. Fortunately, it appears that NFR are usually progressive[Larman 2010]. This property enables a progressive adoption (say, moving from 1k Users to 10k Users) which is usually aligned with products newly deployed on the market. With the time being, reinforcing a NFR should be reflected in both DoR and DoD updates.

By Agilitest, they know how crucial NFR are. This is why the product can be plugged to a performance measuring tool such as Octoperf [ Agilitest 2020 ].

‍

To discover the whole set of practices, click here .

‍