quinta-feira, 20 de outubro de 2016

Software Construction: Scripting with Python

Every programmer shall have in its tricks bag competencies in some kind of scripting language. Along the way you'll have to know a way of automating repetitive tasks to save you and your team time (while building, deploying, testing, etc.).

Good candidates include Python (do you agree on that?): It's said to be simple and fast to learn as well as free:

  1. https://www.python.org/
  2. http://www.astro.up.pt/~sousasag/Python_For_Astronomers/Python_qr.pdf (cheat sheet)
  3. https://perso.limsi.fr/pointal/_media/python:cours:mementopython3-english.pdf (cheat sheet)
  4. http://lifehacker.com/keep-this-python-cheat-sheet-on-hand-when-learning-to-c-1655521825 (cheat sheet)
  5. http://www.tecmint.com/the-truth-of-python-and-perl-features-pros-and-cons-discussed/ (Perl vs Python)


BOOK: Clean Code | Robert C. Martin

So, another good book on how to write good (clean) code to keep in mind is Robert C. Martin's "Clean Code: A Handbook of Agile Software Craftsmanship":

https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882

Quoting:
"Even bad code can function. But if code isn’t clean, it can bring a development organization to its knees. Every year, countless hours and significant resources are lost because of poorly written code. But it doesn’t have to be that way.

Noted software expert Robert C. Martin presents a revolutionary paradigm with Clean Code: A Handbook of Agile Software Craftsmanship . Martin has teamed up with his colleagues from Object Mentor to distill their best agile practice of cleaning code “on the fly” into a book that will instill within you the values of a software craftsman and make you a better programmer—but only if you work at it.

What kind of work will you be doing? You’ll be reading code—lots of code. And you will be challenged to think about what’s right about that code, and what’s wrong with it. More importantly, you will be challenged to reassess your professional values and your commitment to your craft.

Clean Code is divided into three parts. The first describes the principles, patterns, and practices of writing clean code. The second part consists of several case studies of increasing complexity. Each case study is an exercise in cleaning up code—of transforming a code base that has some problems into one that is sound and efficient. The third part is the payoff: a single chapter containing a list of heuristics and “smells” gathered while creating the case studies. The result is a knowledge base that describes the way we think when we write, read, and clean code.

Readers will come away from this book understanding
How to tell the difference between good and bad code
How to write good code and how to transform bad code into good code
How to create good names, good functions, good objects, and good classes
How to format code for maximum readability
How to implement complete error handling without obscuring code logic
How to unit test and practice test-driven development
This book is a must for any developer, software engineer, project manager, team lead, or systems analyst with an interest in producing better code."

Table of Contents 

Clean code
Meaningful names
Functions
Comments
Formatting
Objects and data structures
Error handling
Boundaries
Unit tests
Classes
Systems
Emergence
Concurrency
Successive refinement
JUnit internals
Refactoring serialdate
Smells and heuristics
Appendix A: Concurrency II
Appendix B: Org.jfree.date.serialdate
Appendix C: Cross references of heuristics
Epilogue.

segunda-feira, 17 de outubro de 2016

DIGEST: Agile/Scrum links

Agile / Scrum recent relevant links include:
Additional resources related to Agile / Scrum:
The Tools Dept.: 
(Updated: 2016-10-20, PT BR link; minor rephrases; 2016-11-24: link to summary and minor rephrases; 2017-06-06: actionable book)


TOOL: Online Task Boards - Trello Tour

An online task board implementation so that we can leave those post-its/magnetic cards off the wall/board (when doing Agile / Scrum or even Kan Ban). Use a tool to support you on that such as Trello:



PS: Kudos to André Martins for pointing this out. Also make sure to check additional tools for remote teams collaboration here and here (sococo).




quinta-feira, 13 de outubro de 2016

Intermission: Git (the slang and the GBCS tool)

As per Wikipedia:
"Git is a mild pejorative with origins in British English for an unpleasant, silly, incompetent, stupid, annoying, senile, elderly or childish person. It is usually an insult, more severe than twit or idiot but less severe than wanker, arsehole or twat.
The word git first appeared in print in 1946, but is undoubtedly older. [citation needed] It is originally an alteration of the word get, dating back to the 14th century.[citation needed] A shortening of beget, get insinuates that the recipient is someone's misbegotten offspring and therefore a bastard. In parts of northern England, Northern Ireland and Scotland get is still used in preference to git; the get form is used in the Beatles song "I'm So Tired". [citation needed]
The word has been ruled by the Speaker of the House of Commons to be unparliamentary language.
The word was used self-deprecatingly by Linus Torvalds in naming the git source control package."
Source: https://en.m.wikipedia.org/wiki/Git_(slang)


I also have some (fun) words to say on Git:
The GB smart metering initiative is called Great Britain Companion Specification (GBCS) and there's a well known Portuguese company heavily involved in it (although that is not much publicized). One of the tools was called GFI (Git For Industry) and this is a quote from "Smart Metering for Dummies - CGI" that refers to it, as well as to the unfortunate choice of name.
But if Linus did it, why not other? 

Quoting:
"In an attempt to accelerate development of devices in the absence of a DCC environment, the DCC has commissioned Critical Software to modify a tool previously developed to validate GBCS (the unfortunately named GBCS Interface Testing, or GIT for short). The modified tool, GIT for Industry (GFI) will allow device manufacturers to generate ‘gold standard’ GBCS commands on a ZigBee HAN to which they can connect and test their devices. By the time you read this, the first version of this tool should have been released and we should be well into a series of seven GBCS Test Events organised by the DCC for budding device manufacturers to come along and test out their devices against GIT."

Quoting:
"Parse and Correlate Provider: The Parse and Correlate (P&C) Provider is responsible for
developing the software application used by DCC Users to convert GBCS (the protocol used by the DCC to talk to smart devices) into DUIS (the protocol used by DCC Users to talk to the DCC). P&C also checks that the DSP has done its job correctly when translating critical commands (really important ones) into GBCS (the correlate bit of Parse and Correlate).
This will all make much more sense when you’ve read Chapter 4 (trust me). Critical Software was appointed P&C Provider on 29 April 2014 under a three‐year contract with the option of two one‐year extensions."

The PDF can be found here (requires registration): https://www.cgi-group.co.uk/smart-metering-for-dummies

About GBCS: https://www.gov.uk/government/consultations/smart-metering-implementation-programme-the-process-to-finalise-the-great-britain-companion-specification

(@2016-10: additional text, quote)

INDEX: About Project Plans - The Hitchhiker's Guide to the Galaxy (of QMS / SDP Project Plans)

The Short Story


Plans, plans, plans… 

Q: What kind of initial QMS / SDP-related documentation should it be created at project start (also called Kick-off)?
A: "Some" plans. 

Plans, allow us to plan (which by the way everyone should do) and after approval, allow us to monitor and control the project execution against those same plans. PMs are responsible to approve them even though sometimes they will delegate their creation and maintenance to the TM and/or the most experienced SPAE (Software Product Assurance Engineer) in the team.

What plans are we talking about? The project Plan (and the related detailed plan), the QAP, the CMP, …
You'll hear of even more plans (depending on the standards we are being compliant with). Mainly it has to do with the fact that for some standards we have to plan even further (and deliver those plans so that the contractor knows how we plan - and later on, double check the execution of tasks in the plan, through the inspection of produced evidences). Examples of such standards are the ESA standards (ECSS, for space development), DO-178 (for airborne systems), CENELEC 50126 (for railway applications), and so on. 
Examples of those include:
  • Software Verification Plan (SVP)
  • Software Validation Plan or (the mix of both):
  • Software Verification and Validation Plan (SVVP)
Another PA-related artifact is the PA Log book (sometimes called SPAE Logbook). 
And Agile/Scrum projects use yet another log book: The sprint retro logbook.

What is the purpose of those main SDP plans? 
What are their goals , when are they created, when should they be updated?

Before starting, one important thing to remember is the fact that there should be a public list of QMS Process Owners (PO) and some backups for that role (called Process Representatives - PRs).

The (Very) Long Story


In order to know more about each artifact, the following "sequence of steps" is recommended:
  • Read one entry in the final table below; from there you get to know the QMS process to where it belongs;
  • Open the QMS Process and browse (at least) the main activities there described (typically there is only a single table/slide per process activity).
  • The plan/artifact in question should be an artifact of this process you've just opened and if not, it is because it probably is just updated by this process. It could be an AD of the process or not - you can check it at the end of the process document.
  • Open the template for the artifact (or the provided samples - if any), and scan read its contents. If some concrete examples are available also scan them. Google can also find them.
  • Read the associated guidebooks (or the instruction sheets in the spreadsheet files).
  • During this process, remember to list your questions / open issues so that in the end you can always go to the PO (or as a last resource the QD) and solve them.
  • The one million dollar final questions to answer are:
    • Am I able to create these documents from scratch?
    • Am I able to understand them and extract the needed information for the activities/tasks I will be performing for the project?
    • Do I know who to talk to in case of doubt (who is the doc owner? Who are the PO/PRs)?

INDEX: Plan / Artifact
Templates / Examples / Additional Relevant Docs
Templates
(Docs from MT)
<<master template -- Quality Assurance Plan.docx>>
<<master template -- Product Assurance Report.docx>>
<<master template -- Int. Configuration Management Plan.docx>>
<<master template -- Configuration Management Plan.docx>>

Templates
(Spreadsheets)
<<internal QAP pre-tailored -- CSW-QMS-2013-TPL-00997-rd-pre-tailored-qap.xltx>>
<<internal QAP -- PA Process -- CSW-QMS-2009-TPL-04681-quantitative-qap.xltx>>
<<CSW-QMS-2014-TPL-01088-sprint-retrospective-workbook.xltm>>
<<CSW-QMS-2007-TPL-0112-configuration-items-list.xltm>>
<<CSW-QMS-2006-TPL-1126-spae-log-book.xltm>>

GBKs
<<CSW-QMS-2014-GBK-01269-agile-guidebook.pdf>>
<<CSW-QMS-2012-GBK-04055-crucible-guidebook.pdf>>
<<CSW-QMS-2012-GBK-02170-research-development-projects-tailoring.pdf>>
<<CSW-QMS-2010-GBK-04838-review-report-macro.pdf>>
<<CSW-QMS-2009-GBK-04120-wise-project-module-guidebook.pdf>>
<<CSW-QMS-2009-GBK-03894-project-milestones-meetings.pdf>>
<<CSW-QMS-2009-GBK-03591-qap-how-to.pdf>>

Relevant Trainings
<<project-management-practices-at-csw-internal-training.pptx>>
<<CSW-QMS-2016-PRS-00605-QMSVerificationProcess2016DocumentandCodeR.pptx>>
<<CSW-2015-PRS-00614-managing-baselines-and-change-control-tracking.pptx>>
<<CSW-QMS-2015-TPL-00368-excel-project-tips-tricks.pdf>>




Artifact / Work Product (Plan)
Who is mostly involved
QMS Process / Doc. Level
Main Purpose
Artifact Doc Format / Tool
Generated by MT?
Example Doc. / Link
Associated Template and/or GBK / HOW-TO
More info (when to create, update and AOB)
AOB
Master Project Plan
PM
PM Process (MAN.1)
High-level plan (Gantt chart with the main WBS of the project (similar to the one produced for the estimation sum-up sheet if created during proposal elaboration);

Relevant topics and uses for this artifact include: high level view of ptoject activities (overall view); end project date determination;  resource balancing; critical path analysis; effort and cost estimation (use of PERT estimation techniques  -Best Case Scenario, Expected Case Sc., Worst Case Sc.)…
MS Project (.mpp, etc.)



For each project, the WISE export can be opened in MS Project: https://wise.critical.pt/wiseweb/projects/ExportProjectFile.action?id=7439
For MS Project planning tool info, tips and tricks, see training: https://my.critical.pt/humanresources/Documents/CSW-QMS-2015-TPL-00368-excel-project-tips-tricks.pdf
Should be always kept up to date (acc. To CRs/CCNs, approved changes to schedule, …); uploaded to "WISE Projects"; V1 ready at KOM (based on info from PRL, if existent)
Previously: There were ECS and ASD master plan templates as QMS artifacts.

A guide for the "WISE Project Module" (covering EVM indicators, managing baselines and more) can be found here:
Detailed Project Plan
Typically: PM/TM
PM Process (MAN.1)
Used for detailed planning (day to day tasks, concrete resource allocations, % progress…)
MS Project (.mpp)

Optional: Created if a finer planning and control is needed (by the TM): details the Master Plan (and could be used to report %s progress for each task and calculate overal % progress, etc.)
N/A
Updated weekly (with progress)

External QAP
SPAE (PM typically delegates)
Software Product Assurance Process (MAN.8)
A written statement on how we intend to perform QA activities (i.e. how will the QMS apply to the project). 
For instance: Describes the QA aspects of verification including
  • Review activities to be performed
  • Transition criteria for review activities

The external QAP is a doc; The internal QAP is a XLS (see entry in this table).
MS Word (Master template, template ID = TBD - empty at MT at the moment, reported PROCIMP-819);
Yes
See template TOC.
See template and "fill in the blanks";

Optional: If DL, it is created at KO; tailored with the QD (in case of doubt)
Note: This document could not be requested by the customer as deliverable (DL).
Internal QAP
SPAE
Software Product Assurance Process (MAN.8)
XLS that includes all process tailorings: e.g. Verification Process tailoring

Includes a record of all tailoring decisions e.g. How will verification be performed (e.g. will the project do code reviews?)? Validation? Reqs, Design and SW Construction?
Spreadsheet (see template ids at right)
No
See spreadhseet contents.
See XLS template:


Also see the QAP GBK (link above)
Same as Ext. QAP
Old Ids:
CSW-QMS-2003-TPL-2570 (EN),
CSW-QMS-2004-TPL-2755 (PT)
CMP (External Config. Mg. Plan)
Typically: CM (Project Role; PM assigns this role to a team member)
CM Process (SUP.2)
A written statement on how we intend to do  Config. Management (activities).
This document is optional but if done typically is to be delivered to the customer; this doc  consolidates info in a single document some information already stated in other internal docs such as the ICMP, QAP, CIL, …

MS Word (Master template, template ID = CSW-QMS-2004-TPL-0234)
Yes
See template contents; check list contents
See template(s) and "fill in the blanks";



Optional: If DL, it is created at KO and kept up-to-date during the project (CRs/CCNs could impact this DL);
Note: This document could not be requested by the customer as deliverable (DL).

Template Variants: External and Internal; pre-tailored for certain life cycles: See doc list.

EN
[CSW-QMS-2004-TPL-0234]
{CMP}
Configuration Management Plan (CMP)
EN
[CSW-QMS-2006-TPL-3196]
{CMP}
Internal CM Plan (ICMP)
EN
[CSW-QMS-2013-TPL-01989]
{CMP}
Internal Maintenance Pre-Tailored CM Plan (ICMP)
EN
[CSW-QMS-2013-TPL-00995]
{CMP}
Internal R&D Pre-Tailored CM Plan (ICMP)

Internal CMP (ICMP)
CM (project role)
CM Process
The ICMP is a simpler document (when compared to the CMP) that includes the CM process tailoring decisions (what activities in the CM process are to be performed and what are not and with what formality and frequence) and the CM strategies (such as check-up strategy, build, baseline, tagging and branching strategies); it specifies the conventioned text for the TAGs to be applied at certain moments to the Version Control System (VCS).
MS Word (Master template, template ID = CSW-QMS-2006-TPL-3196; pre-tailored templates: CSW-QMS-2013-TPL-01989 (Maintenance); CSW-QMS-2013-TPL-00995 (R&D)
)

Yes
See template TOC and pre-tailored templates.
See above

As seen above, there are 3 ICMP templates being generated by the MT.
CIL
Typically: CM (PM approves)
Verification Process
Contains the review strategy (usually internal). It list internal and external configuration items under configuration (built from the proposal and including CFIs received from the customer - be it software or hardware, external input docs, etc.) and for the internal, how we plan to do the reviews.
Excel Template: CSW-QMS-2007-TPL-0112
No
See spreadsheet contents; see CM check list contents;


At KO; aligned with the DLs (and outputs) indicated in the PRL;

Typically the CM check list shall be ran before the end of each phase.
Sometimes the CIL info is included in the SVP (Software Verification Plan):
The SVP might be provided by the customer or be required by regulatory standards such as ECSS with the detailed review plan (i.e. while te CIL is typically internal, we could have one additional deliverable, the SVP)
More Project Documents









SPAE Log Book
Most experienced SPAE
SPA Process (MAN.8)
A guide for the performance of Product Assurance activities, including verification check lists before milestones
Excel macro-enabled template:  CSW-QMS-2006-TPL-1226
No

See Milestone meeting prep. GBK:
Created at KO, updated during phases and at least once per milestone

PAR - External Product Assurance Report
SPAE (PM Approves)
SPA Process (MAN.8)
A report with several metrics collected regarding the project execution. Could be requested by the customer as a deliverable.

In case of milestone meetings, the detailed quality verification is performed and the software product assurance report (PAR) is produced, reporting a sum-up of the results. Actions must be created whenever problems are identified and/or when a potential problem is identified. The PAR shall be released for all interested parties (PM and project team, and other departments such as the QD).
 MS Word (Master Template, template ID = CSW-QMS-2003-TPL-1860)
No
See template TOC
See template and "fill in the blanks"
If required: Create one new PAR document per milestone (except KOM).

A guide for milestone meetings (where the PAR contents could be presented) can be found here:

Sprint Retro Log book
Scrum Master / SPAE
Agile Guidebook (to be split in Scrum/Agile Development SDP)
A book to support sprint retrospectives (which is one of the last events to take place in a Scrum/Agile sprint), with some macro integrations to speed up the collection and display of agile metrics, as well as sheets to collect lessons learned and perform action management from them...
Excel template: CSW-QMS-2014-TPL-01088
No
See spreadsheet contents
See template, namely the instructions sheet: