CT: Continuous Testing for the testing bottleneck?

https://www.forbes.com/sites/forbestechcouncil/2017/10/30/how-to-pass-the-software-development-test/#13501252f019

Citando:
"A quick history lesson on why testing has become a bottleneck. Agile methodologies, DevOps and continuous delivery tool chains are modern practices that have been adopted and implemented to drive the speed of applications released into production. But modernizing the testing practices within the software development lifecycle hasn’t occurred at the same pace.

Why is that? Too many testing processes have been manual, slowing down the process and introducing the opportunity for human error. Additionally, a lack of visibility into the progression, status and results of QA testing has made it difficult for DevOps teams to determine if the code they built is truly aligned with the business and customer requirements. A telling stat is that 63% of organizations that implemented DevOps practices report that Testing/QA practices are a bottleneck."

Standards: Safety Critical what?

About safety critical and the most common standards to comply with:
"The Safety Critical technical area addresses the analysis, development, verification and validation of software and systems that are classified as either safety- or mission-critical.

Systems classified as safety-critical are those where failure may lead to personal injuries, fatalities or harmful effects to the environment. Airborne systems in aircraft and signalling systems in railway are traditionally the best-known cases of safety-critical systems. Systems controlling industrial processes in chemical, petrochemical and nuclear plants are also in this category.
Mission-critical systems are those where failure may lead to either loss or major degradation of the mission performance. Traditionally the best known cases of mission-critical systems are unmanned space missions such as telecommunications and scientific satellites, and the energy production and distribution infrastructure.
Enterprise systems in banking, retail and other industries may also be considered mission-critical though we prefer to designate them ‘business-critical’.

The services delivered by the Safety Critical technical area aggregate a set of disciplines and technology that is closely related to international safety standards. These standards are a daily reference in the work performed by the Safety Critical technical area. The most common standards used are for aerospace: ARP-4754, ARP-4761, DO-178B and DO-178C, and DO-254; for railway: EN-501216, EN-50128 and EN-50129; for space ECSS-E-ST-40C and ECSS-Q-ST-80C; for automotive the ISO 26262 and for industrial systems in general IEC 61508.

The work performed by the Safety Critical technical area is characterised by the rigour of the engineering processes. The main challenge faced is to comply with that rigour in the most economically efficient way. The projects and activities developed by the Safety Critical technical area fall into the following categories:

1. • Development of real-time embedded software in either full life-cycle, from requirements to final acceptance, or in partial life-cycle. Often the development entails close co-operation with the customer and in some cases members of the project team may work at the customer’s premises while interacting with the project team at home. The SW will typically be developed in either C or Ada and requires knowledge of real-time operating systems, microprocessor architectures and low-level communication protocols. 
2. • Verification and validation of real-time embedded software, from requirements and code reviews, to unit tests, integration and system tests. A full life-cycle embedded software development also includes V&V but this type of service is often delivered as specific projects, where we verify and validate software developed by the customer or a third party. The V&V services include not only testing software but also testing with hardware-in-the-loop. Typical tools used include VectorCAST, LDRA and bespoke validation facilities. 
3. • Development of integrated electronic systems comprising the software and the hardware that supports it. This requires competencies in both software and electronics, including the ability to procure and integrate COTS components. Typical systems developed include validation facilities with HW-in-the-loop and solutions for military C&C systems. 
4. • Functional safety assessment and reliability prediction (known as RAMS). This entails analysing the failure modes of systems using techniques such as Fault Tree Analysis (FTA) and Failure Mode Effects Analysis (FMEA). This analysis may contribute to both allocate criticalities to components and to reformulate the system design to minimise the number of critical components. 
5. Certification support to help our customers comply with safety-critical standards and training on those standards. This implies a wide body of knowledge in the technical, normative and application domain. 

Typical projects include certification support for railway signalling systems, support for airborne software and hardware certification and training on standards such as DO-178C. The work in the Safety Critical technical area requires interest in safety aspects, the ability to analyse systems as a whole, to know that there is more than just software development and, of course, good reading and writing skills."

[INTERNAL] Source: CSW-QMS-2000-CPD-0174, VERSION: 13, 2017-08-09, Annex B.

REPORT: 2017 State of DevOps report

https://puppet.com/resources/whitepaper/state-of-devops-report

Quoting:
"Over the past six years and more than 27,000 State of DevOps survey responses, we’ve found clear evidence that DevOps practices yield remarkable results for IT teams and organizations.

This year we also discovered new findings about transformational leadership, automation practices, continuous delivery, lean product management, and DevOps in not-for-profits and organizations that use off-the-shelf software.

Here are a few things you’ll learn in this year's report:

- How DevOps practices affect deployment frequency, lead time, change failure rate and MTTR
- The influence leadership has on DevOps transformations
- How high- and low-performing teams automate differently
- The impact of architecture and team structure on IT performance
- How DevOps helps organizations reach both their financial and non-financial goals."

Autonomous Vehicles: Open sourced? (Apollo 1.0 self-driving car software)

https://github.com/ApolloAuto

Source:
https://www.cnet.com/roadshow/news/open-source-apollo-speeds-up-baidus-self-driving-software-development/

Quoting:
"In July, Chinese technology company Baidu made its Apollo 1.0 self-driving car software available as open source on Github, using the Apache/BSD license. By Day 4 of the release, it was the most downloaded C++ software on the site.

At an Apollo meetup hosted by Baidu at its Sunnyvale, California, offices, company president Ya-Qin Zhang announced Apollo 1.5, a major iteration of the software, just three months after the initial release."

Security: Sonar (Open Source MS Tool)

https://thenextweb.com/apps/2017/10/26/microsoft-launches-sonar-to-test-your-sites-performance-and-security/

Quoting:
"Enter your project’s URL, and Sonar will comb through it for accessibility, interoperability, performance, security and progressive web app-related issues. Once it’s done scanning, it’ll list the errors it’s found and do its best to explain what’s going wrong, highlighting the errant code snippets and offering possible solutions.
(...) Sonar improves on the capabilities of other linting tools by executing your website code in a container instead of simply performing a static analysis for more accurate results, as well as allowing for integration with other services. And if you don’t care to use Sonar in your browser, you can also invoke its command-line interface."

BOOK: "The Road Map to Software Engineering: A Standards-Based Guide"

Another Software Development / Software Engineering book FFR:
The Road Map to Software Engineering: A Standards-Based Guide: 9780471683629: Computer Science Books @ Amazon.com

Quoting:
"The Road Map to Software Engineering: A Standards-Based Guide organizes relevant IEEE software and systems standards using two frameworks: the SWEBOK Guide's topical knowledge areas and the widely used IEEE/EIA 12207 standard. This useful guide is endorsed and recommended by the Software and Systems Engineering Standards Committee of the IEEE Computer Society for both practitioners and students."

BOOK: Software Engineering Standards: C. Mazza, J. Fairclough, B. Melton, D. De Pablo, A. Scheffer

A very good summary (the book is thin because topics are handled at a high level perspective and assume you are already a software engineer) of the ESA/ECSS standards:

Software Engineering Standards: C. Mazza, J. Fairclough, B. Melton, D. De Pablo, A. Scheffer: 9780131065680: Amazon.com: Books

Quoting:
"These standards define the essential aspects of software development and have been produced by removing all space-specific aspects from the European Space Agency's software engineering standards. They cover the whole software engineering process at a uniform level of abstraction."

Reuse: Global Mapper SDK (4DMapper)

4DMapper is a geospatial cloud-based SDK (referred in a press release):
https://www.suasnews.com/2017/10/4dmapper-chooses-global-mapper-software-development-kit-sdk-online-analytics/

More details, quoted:
"Global Mapper Software Development Kit (SDK) as the engine behind its online data analytics services. The company’s innovative cloud-based approach to geospatial data creation and distribution reduces the need for their customers to maintain high-end processing hardware and software and it simplifies and streamlines access to the processed data. By making use of a collection of Global Mapper’s powerful 3D data manipulation tools, 4DMapper users can quickly and easily convert a point cloud into contours, generate an elevation grid, and much more."
(...)
"Among the tools available to 4DMapper customers are LiDAR ground point identification and reclassification; DEM creation from a point cloud; custom contour generation; and delineation of ridgelines. Each of these workflows involves the uploading of the relevant source data and results in the creation of a derivative file that can be quickly and easily shared with a customer or client on a web browser."

BOOK: "Extreme Programming Explained: Embrace Change, 2nd Edition" (The XP Series) - Kent Beck

The definitive guide to XP is the book by the author (Kent Beck):

https://www.amazon.com/gp/product/0321278658?ie=UTF8

Extreme Programming Explained: Embrace Change, 2nd Edition (The XP Series)

2nd Edition, ISBN-13: 978-0321278654, ISBN-10: 0321278658

PS. Additional info about XP, considered the first of the agile methodologies (long before Scrum):
https://martinfowler.com/bliki/ExtremeProgramming.html

PT: Upyourstrategy (quality-related event at FEUP)

Just to let you know that a quality-related event (PT) will be hosted in FEUP in the next days, with respectable speakers:
upyourstrategy (https://www.upyourquality.com/)

BOOK: "Software Engineering: A Practitioner's Approach" (AKA SEPA)

Another bible on the subject of software engineering is:
Software Engineering: A Practitioner's Approach: Roger S. Pressman, Bruce Maxim: 9780078022128: Amazon.com: Books

Quoting:
"For almost three decades, Roger Pressman's Software Engineering: A Practitioner's Approach has been the world's leading textbook in software engineering. The new edition represents a major restructuring and update of previous editions, solidifying the book's position as the most comprehensive guide to this important subject. The chapter structure will return to a more linear presentation of software engineering topics with a direct emphasis on the major activities that are part of a generic software process. Content will focus on widely used software engineering methods and will de-emphasize or completely eliminate discussion of secondary methods, tools and techniques. The intent is to provide a more targeted, prescriptive, and focused approach, while attempting to maintain SEPA's reputation as a comprehensive guide to software engineering. The 39 chapters of this edition are organized into five parts - Process, Modeling, Quality Management, Managing Software Projects, and Advanced Topics. The book has been revised and restructured to improve pedagogical flow and emphasize new and important software engineering processes and practices."

PS. Kudos to M. Serra for pointing this out (used at ISEL, PT)
PS2. This is also the book being used at ISEC, PT here.

Some info about RUP (Rational Unified Process)

The Short Story

A summary on RUP (an iterative software development process) can be found here (FFR):

• RUP ( Rational Unified Process ) | have dreams and make it happen

The typical summary image can be found in the site above:

• https://theexplorationofmyworlds.files.wordpress.com/2012/07/rational-unified-process.jpg

The Long Story - The Rational Unified Process

Quoting the "Object Oriented Analysis and Design with Applications" book :

"In its simplest form, RUP consists of some fundamental workflows:

1. Business Engineering. Understanding the needs of the business.

2. Requirements. Translating business need into the behaviours of an automated system.

3. Analysis and Design. Translating requirements into a software architecture.

4. Implementation. Creating software that fits within the architecture and has the required behaviours.

5. Test. Ensuring that the required behaviours are correct, and that all required behaviours are present.

6. Configuration and change management. Keeping track of all the different versions of all the work products.

7. Project Management. Managing schedules and resources.

8. Environment. Setting up and maintaining the development environment.

9. Deployment. Everything needed to roll out the project.

These activities are not separated in time. Rather, they are executed concurrently throughout the lifetime of the project. As we see in [the RUP typical image process representation], not much code gets written early in the project lifecycle; but the amount is not zero. Late in the project, most of the requirements are known, but some new ones are still identified.

(…)

Iteration

A project governed by a RUP process moves forward in increments called iterations.

The goal of each iteration is to develop some working software that can be demonstrated to all the stakeholders, and that the stakeholders will find meaningful. The software developed by an iteration should cut through all or most of the major subsystems of the project.

(…)"

Source: [Object Oriented Analysis and Design with Applications, 2d. ed., Grady Booch, Robert C, Martin, James Newkirk]

RUP defends iterations and additional mandatory workflows (management ones for instance): Configuration and change management. Project Management. Environment is also an important issue (deployment deals with the change to production environment). There is an evident overlapping of Requirements and Design in the Elaboration phase (identification of architectural requirements).

TOOLS: Pencil (rapid Prototyping tools - for mockups creation as well as live feature navigation)

A tool has been pointed out to me (free at the time of posting) and requiring installation (as opposed to some similar web-based tools):
https://pencil.evolus.vn/

Quoting:
"An open-source GUI prototyping tool that's available for ALL platforms.
Pencil is built for the purpose of providing a free and open-source GUI prototyping tool that people can easily install and use to create mockups in popular desktop platforms.
The latest stable version of Pencil is 3.0.4 which contains stability fixes and features a visual stencil builder. More details can be found in the release notes."

PS. About rapid prototyping (tools):
On Software Development (or "O Sr. SiIva disse..."): TOOLS: Rapid prototyping tools (for mockups creation as well as live feature navigation)

PS2. Also make sure to look at the MS tool (Sketchup in MS Expression Studio):
https://msdn.microsoft.com/en-us/library/ee341458(v=expression.40).aspx

https://msdn.microsoft.com/en-us/library/ee341449(v=expression.40).aspx

PS. Kudos to Vera Moreira for pointing this one out.

Books on Requirements?

For everything in life, there's a book. I even found some book for good friends of mine who ride the bicycle (that went something like "how to not ruin your sex life riding bikes"). Unfortunately a search for it in the Amazon store today does not yield results.

And then, there are good books on writing good requirements.

Some of them are pretty much old but they are still good books on the subject (this is an "old" art):

• https://www.amazon.co.uk/Mastering-Requirements-Process-Suzanne-Robertson/dp/0201360462/ref=sr_1_2?s=books&ie=UTF8
• https://www.amazon.co.uk/Mastering-Requirements-Process-2nd-second/dp/B008IT2WJW/ref=sr_1_3 (the 2nd Edition)
• https://www.amazon.co.uk/Managing-Software-Requirements-Approach-Technology/dp/0201615932/ref=sr_1_4?s=books&ie=UTF8
• https://www.amazon.co.uk/Managing-Software-Requirements-Addison-Wesley-Technology/dp/032112247X/ref=sr_1_5?s=books&ie=UTF8
• Writing Better Requirements: Amazon.co.uk: Ian Alexander, Richard Stevens: 9780321131638: Books

PS: Some classical papers on requirements:

• http://homepages.laas.fr/kader/Hooks.pdf

• http://www.win.tue.nl/~wstomv/edu/2ip30/references/smart-requirements.pdf

Additional references (sources for the INTERNAL QMS ENG02 Process) 

The main conceptual references for the process description, back in 2003, were:

Guide to the user requirements definition phase, ESA PSS-05-02 Issue 1 Revision 1 (1995).

Guide to the software requirements definition phase, ESA PSS-05-03 Issue 1 Revision 1 (1995).

Space engineering – Software, ESA ECSS-E-40B Draft 1 (2002)

Some additional references used were:

Managing Software Requirements: a unified approach, Dean Leffingwell, Don Widrig, 2000, Addison-Wesley

Mastering the Requirements Process, Suzanne and James Robertson, 1999, Addison-Wesley

Modelling for requirement engineering, Dr. Eric Conquet, 2002, ESA / ESTEC TOS- EME

Specifying Good Requirements, Donald Firesmith, in Journal of Object Technology, vol. 2, no. 4, July-August 2003, pp. 77-87. http://www.jot.fm/issues/issue_2003_07/column7

@2017-11, additional sources for QMS PCS. 

BOOK: Essentials Of Software Engineering: Frank Tsui, Orlando Karam, Barbara Bernal

Essentials Of Software Engineering: Frank Tsui, Orlando Karam, Barbara Bernal: 9781284106008: Amazon.com: Books

Quoting:
"Written for the undergraduate, one-term course, Essentials of Software Engineering, Fourth Edition provides students with a systematic engineering approach to software engineering principles and methodologies. Comprehensive, yet concise, the Fourth Edition includes new information on areas of high interest to computer scientists, including Big Data and developing in the cloud.

In-depth coverage of key issues, combined with a strong focus on software quality, makes Essentials of Software Engineering, Fourth Edition the perfect text for students entering the fast-growing and lucrative field of software development. The text includes thorough overviews of programming concepts, system analysis and design, principles of software engineering, development and support processes, methodologies, software testing and quality, and product management, while incorporating real-world examples throughout."

PS. Kudos to G. Silva for pointing this out (being used at ISEC).

LIST: RIA and Mobile Web Based frameworks

https://en.m.wikipedia.org/wiki/Multiple_phone_web-based_application_framework

A broader category list:
https://en.m.wikipedia.org/wiki/List_of_rich_Internet_application_frameworks

Mobile: Apache Cordova

https://en.m.wikipedia.org/wiki/Apache_Cordova

Quoting:
"Apache Cordova (formerly PhoneGap) is a mobile application development framework originally created by Nitobi. Adobe Systems purchased Nitobi in 2011, rebranded it as PhoneGap, and later released an open source version of the software called Apache Cordova.^[3] Apache Cordova enables software programmers to build applications for mobile devices using CSS3, HTML5, and JavaScript instead of relying on platform-specific APIs like those in Android, iOS, or Windows Phone.^[4] It enables wrapping up of CSS, HTML, and JavaScript code depending upon the platform of the device. It extends the features of HTML and JavaScript to work with the device. The resulting applications are hybrid, meaning that they are neither truly native mobile application (because all layout rendering is done via Web views instead of the platform's native UI framework) nor purely Web-based (because they are not just Web apps, but are packaged as apps for distribution and have access to native device APIs). Mixing native and hybrid code snippets has been possible since version 1.9."

Mobile: Ionic framework

https://en.m.wikipedia.org/wiki/Ionic_(mobile_app_framework)

Quoting:
"Ionic is a complete open-source SDK for hybrid mobile app development.^[4] The original version was released in 2013 and built on top of AngularJSand Apache Cordova. The more recent releases, known as Ionic 2 or simply "Ionic", are built on Angular. Ionic provides tools and services for developing hybrid mobile apps using Web technologies like CSS, HTML5, and Sass. Apps can be built with these Web technologies and then distributed through native app stores to be installed on devices by leveraging Cordova. Ionic was created by Max Lynch, Ben Sperry, and Adam Bradley of Drifty Co. in 2013.^[5]"

See also: [Microsoft] Xamarin

Standards: NASA Technical Standards

And the family of standards for NASA - systems engineering, i.e. not only software development - can be found below (a similar roadmap to ESA ECSS standards can be found here).

https://standards.nasa.gov/nasa-technical-standards (following some links to the standards inside this link might require login)

Quoting:
"Each NASA Technical Standard is assigned to a Technical Discipline. Please select the respective link to access that discipline's standards.

0000 - Documentation and Configuration

1000 - Systems Engineering and Integration, Aerospace Environments, Celestial Mechanics

2000 - Computer Systems, Software, Information Systems

3000 - Human Factors and Health

4000 - Electrical and Electronics Systems,  Avionics/Control Systems, Optics

5000 - Structures/Mechanical Systems, Fluid Dynamics, Thermal, Propulsion, Aerodynamics

6000 - Materials and Processes, Parts

7000 - System and Subsystem Test, Analysis, Modeling, Evaluation

8000 - Safety, Quality, Reliability, Maintainability

9000 - Operations, Command, Control, Telemetry/Data Systems, Communications

Any questions on the above standards can be directed to the NTSS Curator at standards@msfc.nasa.gov"

And the relevant links are...

Maintenance categories (applicable to SW development, of course):

• https://en.wikipedia.org/wiki/Software_maintenance#Categories_of_maintenance_in_ISO.2FIEC_14764
• https://en.wikipedia.org/wiki/Change_request (and the various names it might get to be called: CR, SCR, CCN, ...)
• Note: ISO/IEC 14764 does not mention the term "Evolutionary Maintenance" that in a SW Maintenance contract means that there is a budget (in hours) for developing new features to the product being maintained (avoiding explicit change requests and additional contract management activities). "Predictive maintenance"is also relevant for HUMS (Health and Usage Maintenance [management] Systems)

About SDP and life cycles (some not widely used nowadays):

• https://en.wikipedia.org/wiki/Software_development_process
• https://en.wikipedia.org/wiki/Spiral_model
• https://en.wikipedia.org/wiki/Extreme_programming
• https://en.wikipedia.org/wiki/V-Model_(software_development)
• Make sure you also read about TDD, RUP, etc.

ENG QMS Process example (for TOC analysis, INTERNAL):

• https://quality.critical.pt/qmsunified/ENG-engineering/ENG01-requirements-analysis/CSW-QMS-2005-PCS-3424-requirements-analysis.pdf#search=eng01