Software Construction: Static Analysis - why is it important and a possible list of candidate tools

Static analysis is an important part of software construction and you as a developer should know some of them well (and use them).

• It'll give you an extra insight on your code (as well as others code).
• Good tools will collect some measures automatically and point you to good candidates for rework / refactoring 
• Automatically collected data include can include Lines of Code / LOC per unit, cyclomatic complexity, etc. 
• Good tools will also allow you to configure the measures and KPIs you want to analyse (e.g. Allow you to configure the KPI "cyclomatic complexity <=5" which could be synonym of "good code" for this particular project)
• Kiviat charts are useful for analysing the source code modules, typically one chart per source code file
• For some markets the use of these tools is not optional. For instance, we could not be allowed to deliver code to the integration and system test phases that has a cyclomatic complexity greater than X (rational: more complex code is harder to test, namely to achieve 100% source code coverage by tests, MC-DC techniques, etc.)
• For peer reviews (be it formal code reviews or other), these tools will help you cleaning up your code as well as respecting coding standards (if you configure them properly with parts of those coding standards). Run them before submitting your code for peer review.
• The WWW will show you what measures are more used by the industry. Examples: LOC, % commentary lines, method length, cyclomatic complexity, subroutine depth, and more.
• These tools can be configured as a task in the continuous integration server (so that for each build the reports are generated automatically for later analysis, by the TM, by the developer, etc). 

Some tools:

• https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis
• https://en.wikipedia.org/wiki/SonarQube - locally installable, free, as most good things in life and multi-programming-language (C, C++, Java, C#).

Reuse: Statistical Data Processing

When in need of statistical data analysis, we will need to implement or reuse system components into our system.
Microsoft has been making available the (free until a certain point) open-source R statistical programming language (and servers):
http://www.infoworld.com/article/3156544/big-data/microsofts-r-tools-bring-data-science-to-the-masses.html


Quoting (the article focus on a new MS acquisition but provides us with an overview of the R programming language and the Development environment to support developers and Data Scientists):


"One of Microsoft’s more interesting recent acquisitions was Revolution Analytics, a company that built tools for working with big data problems using the open source statistical programming language R. Mixing an open source model with commercial tools, Revolution Analytics offered a range of tools supporting academic and personal use, alongside software that took advantage of massive amounts of data–including Hadoop. Under Microsoft’s stewardship, the now-renamed R Server has become a bridge between on-premises and cloud data.
Two years on, Microsoft has announced a set of major updates to its R tools. The R programming language has become an important part of its data strategy, with support in Azure and SQL Server—and, more important, in its Azure Machine Learning service, where it can be used to preprocess data before delivering it to a machine learning pipeline. It’s also one of Microsoft’s key cross-platform server products, with versions for both Red Hat Linux and Suse Linux." 

Software Construction: Scripting with Python

Every programmer shall have in its tricks bag competencies in some kind of scripting language. Along the way you'll have to know a way of automating repetitive tasks to save you and your team time (while building, deploying, testing, etc.).

Good candidates include Python (do you agree on that?): It's said to be simple and fast to learn as well as free:

1. https://www.python.org/
2. http://www.astro.up.pt/~sousasag/Python_For_Astronomers/Python_qr.pdf (cheat sheet)
3. https://perso.limsi.fr/pointal/_media/python:cours:mementopython3-english.pdf (cheat sheet)
4. http://lifehacker.com/keep-this-python-cheat-sheet-on-hand-when-learning-to-c-1655521825 (cheat sheet)
5. http://www.tecmint.com/the-truth-of-python-and-perl-features-pros-and-cons-discussed/ (Perl vs Python)

SW Design, SW Construction: Security as a Design Quality Attribute ("5 lines" summary)

Some topics on Security (the "5 lines" summary with a little bit more than 5 lines): As a systems analyst, system architect, software developer and tester (SPA Engineer) you must:

- Read on security related to the technologies you'll be using for producing the final information system being developed (i.e. the solution):

• C++, C, Java, ... - there are full books on the security topic 
• JEE, .Net and other enterprise application frameworks
• RDBMS and other entity persistence means 
• Other common core system components like...
• App. Servers (JBoss and the like)
• Web Servers (Apache, IIS and the like)
• Operating Systems 
• Anything else (that your system reuses: drivers, hardware, ...). 

- Make sure everything above is patched (regularly) and that whatever you program (and configure) is secure in the light of what you (regularly) read.

Remember to:
- Subscribe newsletters on the topic (of follow the proper newfeeds / groups) to help you keeping updated
- Use the proper frameworks for helping you on security as a process, for example:

The OWASP Application Security Verification Standard (ASVS) Project provides a basis for testing web application technical security controls and also provides developers with a list of requirements for secure development.

- Use the proper tools to help you (executing security assessments) before release. Raise all relevant [code] issues found.
- Reuse the invaluable resources (with guidelines and concrete checklists for specific technologies) like the following (take your time exploring them properly and use whatever suits you):

• https://ics-cert.us-cert.gov/Assessments
• http://www.ussecurityawareness.org/highres/infosec-auditing.html
• https://web.nvd.nist.gov/view/ncp/repository

The bottom line: 
Security is a process: You change a line of code (or a system component) and you could have changed the security characteristics of your system as a whole. Remember that a system is as strong as its weakest link (and you don't want to have been working on that weakest part).

(@2016-11-15: minor rephrases, formatting)

TOOLS: Source Monitor (Static Analysis - Multi-language)

Static analysis refers to analysis of the code statically (without running it).
There are several tools embedded in modern IDEs (as plugins or even natively).
There are also some standalone multi-language tools.
A simple (and free) example of a tool like this is the multi-language tool named Source Monitor:

• http://www.campwoodsw.com/sourcemonitor.html

Quoting:
"The freeware program SourceMonitor lets you see inside your software source code to find out how much code you have and to identify the relative complexity of your modules. For example, you can use SourceMonitor to identify the code that is most likely to contain defects and thus warrants formal review. SourceMonitor, written in C++, runs through your code at high speed, typically at least 10,000 lines of code per second. SourceMonitor provides the following:

• Collects metrics in a fast, single pass through source files.
• Measures metrics for source code written in C++, C, C#, VB.NET, Java, Delphi, Visual Basic (VB6) or HTML.
• Includes method and function level metrics for C++, C, C#, VB.NET, Java, and Delphi.
• Offers Modified Complexity metric option.
• Saves metrics in checkpoints for comparison during software development projects.
• Displays and prints metrics in tables and charts, including Kiviat diagrams.
• Operates within a standard Windows GUI or inside your scripts using XML command files.
• Exports metrics to XML or CSV (comma-separated-value) files for further processing with other tools."

What do you do static analysis for? 
Some reasons include:
- For quickly gaining insight on some new code (to you) when entering a project
- For spotting difficult parts of the code (in terms of complexity for example), spotting refactoring candidates
- For quickly analysing some quality attributes of the code being produced by the team (as a Technical Manager or mini-team leader, for instance % of comments when compared to LOC)
- For presenting reports to the customer on specific quality attributes of the code (Kiviat Charts with KPIs and an optimal region where we would like or are obliged to have the code).

History: Added details, quote 2017-11; 2017-12-04: Minor rephrases;