SW Testing: 64 Essential Test Metrics For Measuring Software Quality, Testing & More

Should you measure things while testing? Sure. In order to be able to manage the phase (and the resources available to execute it) it is mandatory... What should you measure? Here is an interesting article on the topic:

• 64 Essential Test Metrics For Measuring Software Quality, Testing & More!

Quoting:

"Software testing metrics are a way to measure and monitor your test activities. More importantly, they give insights into your team’s test progress, productivity, and the quality of the system under test. When we ask ourselves “What have we tested?”, metrics will give us better answers than just “we have tested it.” Different teams measure various aspects depending on what they want to track and control or improve.

A metric usually conveys a result or a prediction based on the combination of data.

Result Metrics:  metrics that are mostly an absolute measure of an activity/process completed.

Example: Time taken to run a set of test cases in a suite

Predictive Metrics:  metrics that are derivatives and act as early warning signs of an unfavorable result.

Example: Defects created vs. Resolved chart shows the rate of defect fixing.  This grabs the team’s attention if this rate is slower than the rate desired.

Are you using the right software test metrics? Get answers, best practices and strategies in our 14-page guide to improving your software quality through data and analytics.

Why Test Metrics? Why Should You Care?

The aim of collecting test metrics is to use the data for improving the test process, rather than to just show fancy reports. This includes finding tangible answers to the questions:

• How long will it take to test?
• How much money will it take to test?
• How bad are the bugs?
• How many bugs found were fixed? reopened? closed? deferred?
• How many bugs did the test team did not find?
• How much of the software was tested?
• Will testing be done on time? Can the software be shipped on time?
• How good were the tests? Are we using low-value test cases?
• What is the cost of testing?
• Was the test effort adequate? Could we have fit more testing in this release?

Good answers to these questions need measurement. This post includes 64 of the absolute, derivative, result, and predictive metrics that testers and QA managers use the most often.

The Fundamental Metrics 

As a tester, your road to metric creation has to start somewhere.  Fundamental QA metrics are a combination of absolute numbers that can then be used to produce derivative metrics.

Absolute Numbers:

• Total number of test cases
• Number of test cases passed
• Number of test cases failed
• Number of test cases blocked
• Number of defects found
• Number of defects accepted
• Number of defects rejected
• Number of defects deferred
• Number of critical defects
• Number of planned test hours
• Number of actual test hours
• Number of bugs found after shipping"

For the derivative and other type of metrics, follow the link above.

SW Construction: Angular CLI (web app generator tutorial)

http://www.dotnetcurry.com/angularjs/1409/angular-cli-tutorial

Citando:
"Introducing Angular CLI

As many of you would know, building a modern web application using a front-end framework like Angular is no more as simple as developing few html pages and showing them on the browser. To work with modern JavaScript frameworks, you now need an environment to be set on the machine, as well as an environment for the application inside the folder that has to contain the code.

The following list shows the required tools to build an Angular application at a higher level:

• The system to be used for development should have Node.js installed on it. Node.jsis a server platform that supports building web servers using JavaScript. It is also widely used as a platform to build web applications, as it is very easy to spin up a server and start it instantly on Node.js.
• A Node.js based package manager like npm or yarn is needed to install the libraries to be used in the application.
• The application needs a setup to install a set of required packages containing the library files, tools to compile the TypeScript code, bundle the files together, run unit tests, start a web server and build files to deploy the application. While setting up these tasks has become easier with tools like Grunt, Gulp and Webpack, one still needs to spend time in writing the scripts for these tasks in every new project.

The third point of the above list takes a lot of time and effort.

To reduce this pain, the Angular team created Angular CLI. Angular CLI is a generator that makes the process of building an Angular application easier."

SW Design: Enterprise Application Patterns for Xamarin.Forms

https://developer.xamarin.com/guides/xamarin-forms/enterprise-application-patterns/

PDF: https://aka.ms/xamarinpatternsebook

Quoting:
"Architectural guidance for developing adaptable, maintainable, and testable Xamarin.Forms enterprise applications

This eBook provides guidance on how to implement the Model-View-ViewModel (MVVM) pattern, dependency injection, navigation, validation, and configuration management, while maintaining loose coupling. In addition, there's also guidance on performing authentication and authorization with IdentityServer, accessing data from containerized microservices, and unit testing."

SW Design: .NET application architectures

https://www.microsoft.com/net/learn/architecture

Rating software?

Yes. Your software could have to be rated like films in certain countries. How? Using institutions like International Age Rating Coallition (IARC):

https://www.globalratings.com/about.aspx: "A ground-breaking global rating and age classification system for digitally delivered games and apps that reflects the unique cultural differences among nations and regions."

For developers (Source: https://www.globalratings.com/for-developers.aspx):
"How developers can get their games and apps rated with IARC?

IARC allows developers to obtain ratings for multiple territories and storefronts at once via a single streamlined system. The system is available to any developer that is distributing their game or app through a participating digital storefront.
How much does it cost to use IARC?
Obtaining ratings via IARC is free of cost to developers.

How long does the IARC rating process take?
The IARC rating process is designed to be as quick and easy as possible. Once a developer completes the questionnaire, the ratings are issued immediately. The questionnaire is programmed with unique algorithms that assign ratings for each of the participating rating authorities which reflect their own distinct local standards about the age appropriateness of various content featured in the app. The length of time it takes to complete the questionnaire depends on how much ratings-pertinent content is in the game or app. For a game or app that contains very little pertinent content the questionnaire can be finished in a few minutes. If a game or app has more content - like violence, profanity, sexual content, nudity, gambling or other pertinent material - the questionnaire may take a few more minutes to complete.
Why can I only use IARC for products that are distributed by digital means?
In order to handle the high volume of digital games and apps, IARC utilizes an automated process which assigns ratings instantly without rating authorities' review of the product prior to its release. As a result, certain rating assignments may require post-release modification, which is far easier to accomplish in a digital environment than it would be with physical product sold in packages. Adjusting rating assignments after a boxed product has hit store shelves is a complicated and expensive proposition, which is why an automated rating process like IARC is only being deployed for digitally delivered product.
What do I do if the rating my product has been assigned seems incorrect?
Developers can request a rating check for any of the ratings their game or app has been assigned by the IARC system. Developers will have to include a brief explanation of why the rating should be different and may be asked to provide supporting evidence – such as video clips or a build of the game or app – so that the rating check request can be evaluated by the rating authority(s) in question.
Who makes up IARC and how are the rating criteria decided?
IARC is composed of representatives from numerous ratings authorities around the world, each of which plays a role in developing the questionnaire and programming the logic in the rating tool that developers complete when obtaining a rating. That logic is based on the distinct content standards used by each rating authority, and the system generates a unique rating for each territory based on those localized standards.
How can I get my game or app rated using IARC?
Developers can only access the IARC system when submitting their game or app to any of the system's participating storefronts.
(...)"

TOOL: Free Pascal (compliler) and Lazarus (IDE)

Free Pascal is still being used actively to teach programming skills to youngsters, I have recently learned. It is free as the name implies and includes a text-based IDE (for those who remember what that was:  Borland Turbo Pascal).

For download and documentation see:

• Free Pascal documentation index: https://www.freepascal.org/docs.var
• If interested also make sure to read about Lazarus (a free GUI-based IDE): http://wiki.freepascal.org/Lazarus_Documentation
• ... and Virtual Pascal: https://en.wikipedia.org/wiki/Virtual_Pascal

Turbo Pascal is nowadays freely downloadable in the Embarcadero site (which bought Borland) and still releases Delphi (an evolution of the Turbo Pascal IDE and the Object Pascal language for Graphical User Interfaces:

• http://edn.embarcadero.com/article/20792  
• "Freeware release: Borland released several versions of Turbo Pascal as freeware after they became "antique software" (abandonware) in 2002. The newest released international version was TP 5.5, while the newer French TP 7.01 was also released as freeware.The downloads are still available on the successor website of Embarcadero Technologies." (Source: Wikipedia)

SW Design: Recursos sobre e para usabilidade (de sítios Web)

Usability resources (in PT), for PT public sector web sites, FFR: 
https://usabilidade.gov.pt/

Quoting:
"Recursos e ferramentas para melhorar a experiência de utilização de sítios públicos.

Boas práticas, código, design e templates abertos para descarregar e implementar."

Usability Check list (PT, quoted from above, interesting even though some items -e.g. see the Design section- are not completely specific):

1. DESEMPENHO

•  
  A página tem um tempo de carregamento adequado.
•  
  O contraste entre o fundo e a fonte do texto é adequado.
•  
  O tamanho e estilo da tipografia são nítidos e fáceis de ler.
•  
  O sítio está preparado para gerar páginas de erro perceptíveis. (i.e. 404 - Página não encontrada, etc.)
•  
  Os conteúdos estão otimizados para dispositivos móveis.

2. NAVEGABILIDADE

•  
  Existem ajudas visuais que identifiquem a localização na estrutura do sitio. (i.e. Breadcrumbs, etc.)
•  
  Os serviços disponíveis do sitio estão devidamente assinaladas e identificadas.
•  
  Os links são consistentes e fáceis de serem identificados.
•  
  Os links são claros e identificam o destino.
•  
  As páginas do sitio estão identificadas e nomeadas. (i.e. nome da página que surge no navegador)
•  
  O sitio está preparado para compatibilidade com todos os navegadores. (i.e.Cross browser)
•  
  O url da página é claro e facilmente memorizável.
•  
  É possível aceder diretamente à página principal em qualquer zona do sitio. (i.e. homepage link)
•  
  O botão Voltar está sempre presente nas páginas.
•  
  Os elementos clicáveis estão claramente identificados como passíveis de ser acionados.

3. DESIGN

•  
  O design é intuitivo e apelativo.
•  
  O design está preparado para todos os dispositivos de navegação. (i.e. web design responsivo)

4. FORMULÁRIOS E MENSAGENS

•  
  As mensagens de erro são claras e percetíveis.
•  
  Os formulários funcionam corretamente (São apresentadas mensagens de submissão com sucesso ou de erro).
•  
  Os campos obrigatórios estão devidamente assinalados.

5. ORGANIZAÇÃO DA INFORMAÇÃO

•  
  A organização das páginas do sitio é consistente através de toda a estrutura. (i.e. estilo de navegação e arrumação do conteúdo).
•  
  Os ícones representam claramente as suas ações e têm a propriedade ALT definida.
•  
  A informação relevante encontra-se destacada.
•  
  O espaçamento entre linhas é adequado.
•  
  Os textos estão redigidos de acordo com a perspectiva do cidadão destinatário do serviço.
•  
  Os títulos são claros e coerentes com o conteúdo.
•  
  Não existem anúncios ou pop-ups intrusivos.

6. DISPONIBILIZAÇÃO DE SERVIÇOS

•  
  Existe informação sobre as várias fases do processo.
•  
  A ajuda é clara e está perfeitamente visivel.
•  
  A informação é simples e objetiva.

Java evolution

Some changes highlighted in the Java One conference:
http://www.eweek.com/development/java-moving-forward-with-faster-pace-release-schedule-modular-system

Highlights:
- Java 9 is (in principle) backwards compatible with Java8
- Java 10 will be Java 18.3 (WTF!), out in March 2018 (and from then on new features shall be released on a 6 month cadence as they are ready)
- JEE will be moved to the Apache Foundation (with more companies other than Oracle supporting it) and called EE4J
New features include:
- Modular Java (Project jigsaw)
- Java shell (jshell)
- Serverless Java (FaaS)

SW Construction: Languages to know in 2018...

... based on job offers of a specific site in 2017:

• http://www.eweek.com/development/coding-languages-every-developer-should-learn-in-2018

And the list includes:

1. Java

2. Python

3. JavaScript

4. C++

5. C#

6. PHP

7. Perl

Other: Swift (iOS), R (big data analysis), Rust

Other technologies: SQL, .NET framework, Node.js (running JavaScript on the server side), MEAN stack (MongoDB, ExpressJS, AngularJS and Node.js)

Quoting:
"One of the world’s largest coding bootcamps, Coding Dojo, has released an objective analysis of the most in-demand programing languages of 2018.

Coding Dojo came to its findings by analyzing the hundreds of thousands of job postings that contained the name of a programming language on job search engine Indeed.com. It found that Java is the most in-demand followed by Python and JavaScript.

“Software development is a dynamic field,” (...)"

Product Design: Planned obsolescence (as a feature?)

There are good features and there are bad ones. If true, Apple is using planned obsolescence features in its iPhones since 2008:
http://www.nzherald.co.nz/business/news/article.cfm?c_id=3&objectid=11963122

Apple explains the "feature":
https://www.theverge.com/2017/12/20/16800058/apple-iphone-slow-fix-battery-life-capacity

Quoting:
""I believe, as do others, that Apple introduced a change to limit performance when battery condition decreases past a certain point.

"If the performance drop is due to the 'sudden shutdown' fix, users will experience reduced performance without notification.

"Users expect either full performance, or reduced performance with a notification that their phone is in low-power mode.

"This fix creates a third, unexpected state."

Without this warning Poole believes that the fix will cause users to believe their phone is slow so they should replace it, rather than the battery which is actually at fault.

"This will likely feed into the 'planned obsolescence' narrative," he added.

Every time Apple launches a new handset people seemingly flock to Google to ask why their current iPhone or iOS is slow.

This trend has been seen every year since Apple released the iPhone 3G in 2008.

Explanations for the slowness range from Apple's iOS causing problems on older devices to the firm deliberately slowing down old phones to make people buy the new handsets.

This latter explanation is known as planned, or built-in, obsolescence.

This is the idea that manufacturer's deliberately make their products in such a way that they become out-of-date sooner.

By doing this, the companies can encourage customers to buy the latest model of a certain product."

Updated: 2017-12-08, Apple "explains".

TOOLS: Mocking Frameworks (and testing tools) for Java programmers (LIST)

The 8 Useful Java Testing tools (& Frameworks) for Programmers, Developers and Coders

Listed above, 2 mocking frameworks for Java:

"Mockito

Mockito is designed as a open source testing framework for Java which is available under a MIT License. Mockito allows programmers to create and test double objects (mock objects) in automated unit tests for the purpose of Test-driven Development (TDD) or Behavior Driven Development (BDD).

Powermock

PowerMock is a Java Framework for unit testing of source code and It runs as an extension of other Mocking frameworks like Mockito or EasyMock but comes with more powerful capabilities. PowerMock utilizes a custom classloader and bytecode manipulator to enable mocking of static methods, removal of static initializes, constructors, final classes and methods and private methods. It as been primarily designed to extend the existing API’s with a small number of methods and annotations to enable the extra features.
It is available under an open source Apache License 2.0 and is available in Powermock Google Code site."

BOOK: C++ Coding Standards: 101 Rules, Guidelines, and Best Practices (Herb Sutter, Andrei Alexandrescu)

Amazon.com: C++ Coding Standards: 101 Rules, Guidelines, and Best Practices (9780321113580): Herb Sutter, Andrei Alexandrescu: Books

Quoting:
"Consistent, high-quality coding standards improve software quality, reduce time-to-market, promote teamwork, eliminate time wasted on inconsequential matters, and simplify maintenance. Now, two of the world's most respected C++ experts distill the rich collective experience of the global C++ community into a set of coding standards that every developer and development team can understand and use as a basis for their own coding standards.

The authors cover virtually every facet of C++ programming: design and coding style, functions, operators, class design, inheritance, construction/destruction, copying, assignment, namespaces, modules, templates, genericity, exceptions, STL containers and algorithms, and more. Each standard is described concisely, with practical examples. From type definition to error handling, this book presents C++ best practices, including some that have only recently been identified and standardized-techniques you may not know even if you've used C++ for years. Along the way, you'll find answers to questions like

• What's worth standardizing--and what isn't?
• What are the best ways to code for scalability?
• What are the elements of a rational error handling policy?
• How (and why) do you avoid unnecessary initialization, cyclic, and definitional dependencies?
• When (and how) should you use static and dynamic polymorphism together?
• How do you practice "safe" overriding?
• When should you provide a no-fail swap?
• Why and how should you prevent exceptions from propagating across module boundaries?
• Why shouldn't you write namespace declarations or directives in a header file?
• Why should you use STL vector and string instead of arrays?
• How do you choose the right STL search or sort algorithm?
• What rules should you follow to ensure type-safe code?
• Whether you're working alone or with others, C++ Coding Standards will help you write cleaner code--and write it faster, with fewer hassles and less frustration."

Scope Management: How to scope bespoke SW Dev Projects

Interesting article on Scope Management throughout the SW Development lifecycle (waterfall):
https://www.forbes.com/sites/quora/2017/12/14/what-are-some-best-practices-for-scoping-software-development-projects/#127b01df1bca

There are tips / items for several project phases above. For example:
Quoting:
"One of the most important things to do during this time is to define very specific goals for the project. For example, instead of “improve X to be more scalable and performant”, a better and more specific goal might be to “improve X by adding unit tests, supporting 20K queries per second, and reducing capped mean of user latency to <= 200ms”. Having very specific goals allows you to ruthlessly cut anything that does not contribute to these goals, so you don’t suffer from feature creep. Along these lines, you might also consider explicitly defining anti-goals, and separating must-haves and nice-to-haves.

Minimize the batch size of the project by (1) setting up clear milestones and checkpoints for the project and (2) making it easy to launch only part of the project. Not only does this help break down the project, it will also allow the team to pause or cut the project early, should another higher priority task come up.

De-risk the project as soon as possible. Two common ways of de-risking a project include (1) working on the riskiest parts upfront and (2) prototype the riskiest parts using dummy data and/or scaffolding. Whenever a new open-source system or external service is used, that usually represents a big risk.

Don’t optimize for total amount of work done, instead, optimize for total amount of impact over time. Once you’ve gotten the riskiest part out of the way, prioritize working on the part of the project that would result in the highest amount of impact immediately."

Roles: CTOs are not developers

https://moderncto.io/youre-developer-not-cto-yet/

Quoting:
"Developers are not CTOs. I remember an argument I heard from a single developer CTO. He’d say stuff like, “Well I made the product, and I was here when it started so I’m the best fit for CTO.” That was me, a decade ago. Wow, was I ignorant.

What if we extend that logic to other parts of life? “Well, I was standing next to him when he had a heart attack, and I make medical equipment, so I’m the best one to do the triple bypass.” Laughable. If I rest on past achievements I’ll never grow."

BOOK: Art of Unit Testing, Second Edition (Roy Osherove)

To see the TOC:
https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890

2 free chapters:
https://livebook.manning.com/#!/book/the-art-of-unit-testing-second-edition/chapter-4
https://livebook.manning.com/#!/book/the-art-of-unit-testing-second-edition/chapter-1/45

Quoting:
"About the book
You know you should be unit testing, so why aren't you doing it? If you're new to unit testing, if you find unit testing tedious, or if you're just not getting enough payoff for the effort you put into it, keep reading.

The Art of Unit Testing, Second Edition guides you step by step from writing your first simple unit tests to building complete test sets that are maintainable, readable, and trustworthy. You'll move quickly to more complicated subjects like mocks and stubs, while learning to use isolation (mocking) frameworks like Moq, FakeItEasy, and Typemock Isolator. You'll explore test patterns and organization, refactor code applications, and learn how to test "untestable" code. Along the way, you'll learn about integration testing and techniques for testing with databases.
(...)
The Art of Unit Testing, Second Edition guides you step by step from writing your first simple tests to developing robust test sets that are maintainable, readable, and trustworthy. You'll master the foundational ideas and quickly move to high-value subjects like mocks, stubs, and isolation, including frameworks such as Moq, FakeItEasy and Typemock Isolator. You'll explore test patterns and organization, working with legacy code, and even "untestable" code. Along the way, you'll learn about integration testing and techniques and tools for testing databases and other technologies."

ISBN: 978-1-933988-27-6

CONFERENCE: GDG DevFest Season 2017 (PT)

GDG DevFest Season 2017 - Coimbra (PT), 2017-12-09

Quoting:
"The biggest Google tech conference in Portugal: All about Android, Web, Cloud and Security from the world experts"

Additional info:
Day 1 agenda: https://devfest.gdgcoimbra.xyz/schedule/day1
About the Google Developers Group (Coimbra city): http://gdgcoimbra.xyz/

Learning: movimento código portugal (PT, 2nd Edition)

Codemove is a PT movement to teach kids programming. A good step into the future (in PT):

• https://www.codemove.pt/ 
• https://www.publico.pt/2017/12/04/sociedade/noticia/de-pequenino-se-aprende-a-codificar-1794553

More on google:
movimento codigo portugal - Pesquisa Google

Learning: Advent of code?

https://adventofcode.com/

Quoting:
"Advent of Code is a series of small programming puzzles for a variety of skill levels. They are self-contained and are just as appropriate for an expert who wants to stay sharp as they are for a beginner who is just learning to code. Each puzzle calls upon different skills and has two parts that build on a theme."

Security: Apple is revising its SW Development Process

We should ALWAYS take into account security issues in SW Development, even if there are no explicit requirements on that. Do you agree?

What do you think the developers of these features (and the rest of the team, of course) heard after Apple felt obliged to do this public (shaming) announcement?

https://www.itproportal.com/news/apple-to-review-software-development-after-login-flaw-was-discovered/

Quoting:
"The vulnerability allows users of macOS 10.13 to gain admin rights, or log in as root, simply by clicking a login box several times. Apple issued a fix within 24 hours, and both the US and German governments issued alerts advising Mac users to patch up.

But besides patching up, Apple is now looking to review how it does code.

“We greatly regret this error and we apologize to all Mac users,” Apple said in a statement. “Our customers deserve better. We are auditing our development processes to help prevent this from happening again.”

“Security is a top priority for every Apple product, and regrettably we stumbled with this release of Mac OS,” Apple said in its statement.

To exploit the vulnerability, users simply needed to bring up the authentication dialog box, which is often done if needing to configure privacy or network settings. Once the dialog box appears, type in 'root' as a username. Leave the password blank, press Enter, then click 'unlock' a couple of times and voila – access granted.

Fortunately, the workaround to the flaw appears simple – all users need to do is configure a root password and the flaw will no longer work."