This is a wiki page that help seccom and TSC to define a Quality gates inside SonarCloud to improve the quality of code.
Conditions on New Code
Conditions on New Code apply to all branches and to Pull Requests.
Gold (code mature)
| Metric | Operator | Value |
|---|---|---|
| Test coverage | greater than | 80% |
| Duplicated Lines | less than | 3% |
| Maintainability Rating | equal | A |
| Reliability Rating | equal | A |
| Security Hotspots | equal | A |
| Security Rating | equal | A |
Silver (in progress to mature project)
| Metric | Operator | Value |
|---|---|---|
| Test coverage | greater than | 60% |
| Duplicated Lines | less than | 6% |
| Maintainability Rating | equal | B |
| Reliability Rating | equal | B |
| Security Hotspots | equal | A |
| Security Rating | equal | A |
Bronze (mature project not at all)
| Metric | Operator | Value |
|---|---|---|
| Test coverage | greater than | 55% |
| Security Hotspots | equal | A |
| Security Rating | equal | A |
source of metrics chapter https://docs.sonarqube.org/latest/user-guide/quality-gates/
Tests
Coverage (coverage)
It is a mix of Line coverage and Condition coverage. Its goal is to provide an even more accurate answer to the following question: How much of the source code has been covered by the unit tests?
Coverage = (CT + CF + LC)/(2*B + EL)
where
CT = conditions that have been evaluated to 'true' at least once
CF = conditions that have been evaluated to 'false' at least once
LC = covered lines = linestocover - uncovered_lines
B = total number of conditions
EL = total number of executable lines (lines_to_cover)
Coverage on new code (new_coverage)
Identical to Coverage but restricted to new / updated source code.
Line coverage (line_coverage)
On a given line of code, Line coverage simply answers the following question: Has this line of code been executed during the execution of the unit tests?. It is the density of covered lines by unit tests:
Line coverage = LC / EL
where
LC = covered lines (lines_to_cover - uncovered_lines)
EL = total number of executable lines (lines_to_cover)
Line coverage on new code (new_line_coverage)
Identical to Line coverage but restricted to new / updated source code.
Line coverage hits (coverage_line_hits_data)
List of covered lines.
Lines to cover (lines_to_cover)
Number of lines of code which could be covered by unit tests (for example, blank lines or full comments lines are not considered as lines to cover).
Lines to cover on new code (new_lines_to_cover)
Identical to Lines to cover but restricted to new / updated source code.
Duplications
Duplicated blocks (duplicated_blocks)
Number of duplicated blocks of lines.
Language-specific details
Duplicated files (duplicated_files)
Number of files involved in duplications.
Duplicated lines (duplicated_lines)
Number of lines involved in duplications.
Duplicated lines (%) (duplicated_lines_density)
= duplicated_lines / lines * 100
Maintainability
Maintainability Rating (sqale_rating)
(Formerly the SQALE rating.) Rating given to your project related to the value of your Technical Debt Ratio. The default Maintainability Rating grid is:
A=0-0.05, B=0.06-0.1, C=0.11-0.20, D=0.21-0.5, E=0.51-1
The Maintainability Rating scale can be alternately stated by saying that if the outstanding remediation cost is:
- <=5% of the time that has already gone into the application, the rating is A
- between 6 to 10% the rating is a B
- between 11 to 20% the rating is a C
- between 21 to 50% the rating is a D
- anything over 50% is an E
Reliability
Bugs (bugs)
Number of bug issues.
New Bugs (new_bugs)
Number of new bug issues.
Reliability Rating (reliability_rating)
A = 0 Bugs
B = at least 1 Minor Bug
C = at least 1 Major Bug
D = at least 1 Critical Bug
E = at least 1 Blocker Bug
Reliability remediation effort (reliability_remediation_effort)
Effort to fix all bug issues. The measure is stored in minutes in the DB. An 8-hour day is assumed when values are shown in days.
Reliability remediation effort on new code (new_reliability_remediation_effort)
Same as Reliability remediation effort but on the code changed on New Code.
Security
Vulnerabilities (vulnerabilities)
Number of vulnerability issues.
Vulnerabilities on new code (new_vulnerabilities)
Number of new vulnerability issues.
Security Rating (security_rating)
A = 0 Vulnerabilities
B = at least 1 Minor Vulnerability
C = at least 1 Major Vulnerability
D = at least 1 Critical Vulnerability
E = at least 1 Blocker Vulnerability
Security remediation effort (security_remediation_effort)
Effort to fix all vulnerability issues. The measure is stored in minutes in the DB. An 8-hour day is assumed when values are shown in days.
Security remediation effort on new code (new_security_remediation_effort)
Same as Security remediation effort but on the code changed on New Code.
Security Hotspots (security_hotspots) Number of Security Hotspots
Security Hotspots on new code (new_security_hotspots) Number of new Security Hotspots on New Code.
Security Review Rating (security_review_rating)
The Security Review Rating is a letter grade based on the percentage of Reviewed (Fixed or Safe) Security Hotspots.
A = >= 80%
B = >= 70% and <80%
C = >= 50% and <70%
D = >= 30% and <50%
E = < 30%
security Review Rating on new code (new_security_review_rating)
Security Review Rating for New Code.
Security Hotspots Reviewed (security_hotspots_reviewed)
Percentage of Reviewed (Fixed or Safe) Security Hotspots.
Ratio Formula: Number of Reviewed (Fixed or Safe) Hotspots x 100 / (To_Review Hotspots + Reviewed Hotspots)
New Security Hotspots Reviewed
Percentage of Reviewed Security Hotspots (Fixed or Safe) on New Code.