Static vs dynamic code analysis: Advantages and Disadvantages

static-vs-dynamic-code-analysis-advantages-and-disadvantages

What are the advantages and limitations of static and dynamic software code analysis? Maj. Michael Kleffman of the Air Force’s Application Software Assurance Center of Excellence spelled it out.

Static code analysis advantages:

  1. It can find weaknesses in the code at the exact location.
  2. It can be conducted by trained software assurance developers who fully understand the code.
  3. It allows a quicker turn around for fixes.
  4. It is relatively fast if automated tools are used.
  5. Automated tools can scan the entire code base.
  6. Automated tools can provide mitigation recommendations, reducing the research time.
  7. It permits weaknesses to be found earlier in the development life cycle, reducing the cost to fix.

Static code analysis limitations:

  1. It is time consuming if conducted manually.
  2. Automated tools do not support all programming languages.
  3. Automated tools produce false positives and false negatives.
  4. There are not enough trained personnel to thoroughly conduct static code analysis.
  5. Automated tools can provide a false sense of security that everything is being addressed.
  6. Automated tools only as good as the rules they are using to scan with.
  7. It does not find vulnerabilities introduced in the runtime environment.

Dynamic code analysis advantages:

  1. It identifies vulnerabilities in a runtime environment.
  2. Automated tools provide flexibility on what to scan for.
  3. It allows for analysis of applications in which you do not have access to the actual code.
  4. It identifies vulnerabilities that might have been false negatives in the static code analysis.
  5. It permits you to validate static code analysis findings.
  6. It can be conducted against any application.

Dynamic code analysis limitations:

  1. Automated tools provide a false sense of security that everything is being addressed.
  2. Automated tools produce false positives and false negatives.
  3. Automated tools are only as good as the rules they are using to scan with.
  4. There are not enough trained personnel to thoroughly conduct dynamic code analysis [as with static analysis].
  5. It is more difficult to trace the vulnerability back to the exact location in the code, taking longer to fix the problem.
Tagged : / / / / / / / / / / / / / / /

Comparison Between UCC, CLOC, POWERSOFTWARE,EZ-Metrics and Metrixware

ucc-vs-cloc-vs-powersoftware-vs-ez-metrics-vs-metrixware

Unified CodeCount (UCC) CLOC POWERSOFTWARE EZ-Metrics Metrixware
Languages
C/C++, C#, Java, SQL, Ada, Perl, ASP, ASP.NET, JSP, CSS, HTML, JavaScript, VB, and VbScript YES – All ADA, Assembly, ASP, C#, C/C++, CSS, Fortran, IDL, HTML, Java, JavaScript, JSP, Perl, PHP, PL/SQL, PowerBuilder, Python, Ruby, ShellScript, Textfiles, VB6 / VB.NET / VBScript, VHDL, Windows Batch and XML YES ALL
Platforms
Windows & Linux Linux 2.6.9, Unix, Mac OS X, Windows 9x/Me/XP/Vista, Solaris BOTH Windows, Linux planned but no date Both
Baselines comparison
How the tool manages folder hierarchy changes? The tool tries to match files between two baselines using filenames. As such, two files having the same name in different folder structures can be matched. The tool also detects to match and compare files if the folder is changed while filenames of the files contained in the folder are kept the same. NA No information NA
How the tool manages files which are renamed? Currently, the tool does not handle files renamed. However, if the file is renamed but its content does not change, the tool considers it as a duplicate. NA No information NA
How the tool manages files or block swapping? We have not handled swapping blocks of code yet. If the code is copied from one place to another, it is considered as deleted and added. If files are swapped and its filename does not change, the tool can match and compare them. Available No information Available
What is the algorithm used for line change detection? For comparing between lines, we detect the number of common characters between them and determine whether they are modified or deleted using a threshold. This threshold can be specified through a parameter named –t. For detecting bulk of changed or added code, we implemented our own algorithm for detecting longest common sequences. I am sorry, it is quite complex to be described in this email. We are documenting it in detail, and if you are interested I can send you a copy after it is completed. SLOC, PERL Mod No information NA
Miscellaneous
GUI & CLI CLI CLI Both but separate products GUI
CSV & XML Output Only TXT XML HTML, CSV, RAW XML data YES
Provide Qualitative metrics? No. The tool is focused on software size metrics. NO yes but separate product YES
Price Open Source Open Source KEPM (which includes EPM) costs 1,995 USD for a single license or 4,995 for a 5-user license Commericial
Frequency of the releases No information in net Regular One minor/Major release per month or 2 months.
Date of last release December,2009 Apr-10 16-Mar-10
Press on the net Not many reviews available in net Nope
Integration with quality platform Provides different language source for the integration. Nope
Recommend NO Yes No No
Algorithm confidence The total sizing of analyzed source code files in terms the SLOC count contains the highest degree of confidence. However, the sizing information pertaining to the sub classifications (compiler directives, data lines, executable lines) has a somewhat lower level of confidence associated with them.

Misclassifications of the sub classifications of SLOC may occur due to:

(1) user modifications to the UCC tool,
(2) syntax and semantic enhancements to the parsed programming language,
(3) exotic usage of the parsed programming language, and
(4) integrity of the host platform execution environment.

SLOC algorithm with perl string handling features and SPAN mdoules NA
Advantages / Drawbacks / Comments Output not according to our need.
Limited Output Format
Delta is not useful
Low Processing speed
Output according to our need.
Output in many form(CSV, XML, TXT and Mysql)
Delta is useful according to our needs
Fast processing
I tried 30 days trial version. They given web based account/dashboard to add src file and generate output. Which was not functional and could not test it functionality in details. Basic functionality is not working.
Tagged : / / / / / / / / / / / / / / / /

SLOC Tools Comparison | SLOC Tools Differences | SLOC Tools Comparison Table

sloc-tools-comparison

Tool Open Source – Commercial URL
CAST Commercial www.castsoftware.com
Sonar Open Source www.sonarsource.org
SLOC Open Source -> Commercial http://www.dwheeler.com/sloccount/
RSM Commercial www.msquaredtechnologies.com
LocMetrics Commercial www.locmetrics.com
EZ-Metrics Commercial http://www.jamesheiresconsulting.com/Products.htm
Metrixware Commercial www.metrixware.com
Parasoft (Jtest) Commercial www.parasoft.com/
Squale Open Source www.squale.org/
KODERS Commercial www.koders.com
PRACTILINE www.practiline.com
POWERSOFTWARE Commercial http://www.powersoftware.com/
CLOC Open Source http://cloc.sourceforge.net/
Unified CodeCount (UCC) Open Source http://sunset.usc.edu/research/CODECOUNT/
Tagged : / / / / / / / / / / / / / / /

What is Code Coverage and Why Code Coverage?

code-coverage

What is Code Coverage
Code Coverage is an important measurement in Software Quality Engineering. While Software testing ensures correctness of the applications, a metric is required to track the What is Code Coverage Code Coverage is an important measurement in Software Quality Engineering. While Software testing ensures correctness of the applications, a metric is required to track the completeness and effectiveness of the testing undertaken. Code Coverage helps achieve reliable quality through identifying untested areas of the application.

Why Code Coverage
Software testing is a challenging function. The testers need to ensure complete functional and non-functional correctness of the product. Considering the complex workflows and use cases of modern day applications, the number of unique cases that the software can be used often run into millions, which is not feasible to be covered under testing exercise. The testers thus need to
– While Planning Tests
o Ensure covering all workflows in terms of decision trees in the code
o Ensure covering all data values – by identifying patterns rather covering millions of values
– While testing
o Ensuring the testing is completely exercising the whole application with planned and exploratory tests.

At the end of testing, the decision to stop testing and release the product still remains subjective, based on the presence or absence of bugs, inflow of new bugs, success rate of each test cycle, confidence rating of the testers or users, etc. Whereas the definitive metric of quantifying how much of the application was really tested, is missed.

Code Coverage is measured as quantification of application code exercised by the testing activities. Code Coverage can be measured at various levels – in terms of programming language constructs – Packages, Classes, Methods, Branches or in terms of physical artifacts – Folders, Files and Lines. For Eg. A Line Coverage metric of 67% means the testing exercised 67% of all executable statements of the application. A Code Coverage metric usually is accompanied by Code Coverage Analysis Report – which helps identify the un-tested part of the application code, thereby giving the testers early inputs for complete testing.

Benefits of Code Coverage

  • Objective Indicator of Test Coverage of application code
  • Pointers to uncovered Packages / Classes / Methods / Branches
  • Pointers to uncovered Folders / Files / Lines
  • Drill down to untested part of source code and devise new tests
  • Early Indicator for Testing Quality and Fixing it by adding new tests.
  • Remove redundancy in testing
  • Increased Confidence for Releases

Test Your Test

Typical Emotional Storyboard

  • Write Some code! Happy!
  • Does it work? Sad!
  • Write some test! Happy!
  • Do they really test the code? Sad!
  • Measure the Code Coverage! Happy!

Coverage Measurement

  1. Shows Which line of code are executed
  2. How much of your code is covered by your tests?
  3. Your tests test your product
  4. Coverage testing tests your tests

Goal

  • 100%
  • Coverage Ideal
  • Not Always possible
  • Can be expensive to achieve
  • Design for testability

Good: Write more tests
Only way to truly increase code coverage

Bad
Excluding Code to boost Coverage

Types of Coverage

  1. Statement Coverage
  2. Branch Coverage
  3. Path Coverage
  4. Loop Path Coverage
  5. Data – driven Code
  6. Complex Conditionals
  7. Hidden Branches

How; – Coverage Tools

  1. Clover
  2. Cobertura
  3. Emma
Tagged : / / / / / / / / / / / / /

Benefits of CVSNT, What are the advantages of CVSNT over CVS ?

advantages-of-cvsnt-over-cvs

Advantages of CVSNT over CVS

Feature CVSNT CVS
Server
Supports authentication via Microsoft Active Directory or SSH (windows only) YES NO
Set enforced protocols allow server to lock out clients connecting over insecure protocols or using insecure/inefficient options. YES NO
Easily remove protocols (without recompile) YES NO
Branch ACLs can be used to restrict access YES NO
LockServer provides file level locking YES NO
More sophisticated / extra triggers available e.g. postcommit. Triggers also available via COM/DLL/.so interfaces YES NO
Supports Unicode files with additional keyword expansion switches YES NO
Efficient storage of binary files using binary deltas YES NO
Extended modules functionality using the modules2 file YES NO
Advanced Reserved Edits and checked commits (supercedes exclusive locking concept) YES NO
Server-side default options (cvsrc) YES NO
CVSROOT/config scripts etc. YES YES
Repository browsing via cvs ls command YES YES
Pluggable server-side diff programs YES NO
Supports Unicode files with additional keyword expansion switches YES NO
Server-side default options (cvsrc) YES NO
UTF-8 (Unicode) Server. YES NO
Multi Lingual filenames suport. YES NO
Rendevous Support YES NO
Binary availability for Windows, Mac OS X, Linux, Solaris, HPUX YES YES
Client support for IBM iSeries (AS/400) OS/400 YES NO
Windows Server
Supports encrypted authentication via SSL (all platforms) YES NO
Configurable with Windows Control Panel YES NO
Compatible with NTFS ACL’s for using permissions based on Windows username or group. YES NO
Triggers also available via COM and DLL interfaces YES NO
Cshdump handler YES NO
Native file access YES NO
Native MSI Installer YES NO
Client
Smart Merge using MergePoint YES NO
Supports Unicode files with additional keyword expansion switches YES NO
“Import-and-go” by optionally turning freshly imported trees into a new sandbox automatically. No more need to purge and do a fresh checkout first YES NO
Version OSX resource fork extensions keyword expansion switches YES NO

Source: Related Website & http://www.cvsnt.org

Tagged : / / / / / / / / / / / / / /