README.md 6.3 KB
Newer Older
1
2
3
4
5
6
<!--
SPDX-FileCopyrightText: 2021 OW2

SPDX-License-Identifier: Apache-2.0
-->

7
8
9
10
11
12
13
14
# What is MRL?

MRL (Market Readiness Levels) refers to a project assessment methodology initiated by OW2, and described here: https://www.ow2.org/view/MRL .

MRL includes, but is not limited to, a technical layer that collects data from different sources (issue trackers including Gitlab/Github/Jira, SonarQube, Scancode...), aggregates them as significant metrics (like the number of unanswered bugs, or the ratio of source files without license), then allows to visualize them.

The current repository contains the technical framework related to data collection, aggregation and visualization.

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
15
# MRL data collection
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
16

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
17
Move to data-collection/ directory, then build with maven:
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
18
19
20
21
22
23
24

```
mvn clean install
```

## Quick start

25
#### Phase 1 (data collecting script generation, using sample configuration in src/test/resources/ow2_projects.cfg):
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
26
27

```
28
java -jar target/data-collection.jar src/test/resources/ow2_projects.cfg 1
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
29
```
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
30
A /tmp/MRL directory should be created, with a script inside.
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
31

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
32
33
34
If MRL metadata endpoints are defined, MRL metadata files may also be present (name starting with MRL_).
Currently, only thresholds are supported as metadata (requires "MRL.thresholds" property pointing on MRL thresholds endpoint in ow2_projects.cfg), and collected in file "MRL_thresholds.csv".

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
35
#### Phase 2 (launch generated data collecting script):
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
36
37
38
39

```
/tmp/MRL/fetch_statistics.sh
```
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
40
The script should collect data for a few minutes, and save them as a bunch of .json files in /tmp/MRL.
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
41

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
42
#### Phase 3 (parse data and generate CSV file):
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
43
44

```
45
java -jar target/data-collection.jar src/test/resources/ow2_projects.cfg 3
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
46
47
```

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
48
Aggregated OMM data should be now available in CSV format, in file /tmp/MRL/metric_values.csv .
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
49
50
51

## Processing details

52
The package contains some sample data (those that can't be collected online, like OMM and scancode, supposed to be computed by batch), and a sample configuration file (ow2_projects.cfg), in src/test/resources .
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
53
54
55
The configuration file specifies:
- The list of OW2 projects to scan
- The output directory for data (default /tmp/MRL)
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
56
- For each scanned project, properties necessary to reach the project's Git repo, issue tracker, Sonarqube, scancode output, and OMM questionnaire data.
57
- Optionally, for a given project, overwrite value(s) for specified metric(s) (projectName.overwrite.metricName).
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
58

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
59
The data collection process is divided in 3 phases:
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
60
61
62
63
64
65
66
- Phase 1: generate a data collecting script for all projects (fetch_statistics.sh)
- Phase 2: run the data collecting script (can more efficiently be ran by hand, the script being executable).
- Phase 3: parse the data and generate an aggregated CSV file.

Each phase can be launched as follows:

```
67
java -jar target/data-collection.jar src/test/resources/ow2_projects.cfg <phase-number(s)>
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
68
69
70
71
72
```

Valid phase numbers are 1 (generate data collecting script), 2 (run script) or 3 (parse collected data).
Multiple phases can be chained, using comma-separated phase numbers (eg. 1,2,3).

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
73
74
# MRL Webapp

75
The MRL webapp (webapp/ directory) is packaged as a .war archive, and requires a MySQL database. 
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
76
77
78

## Initialize MySQL

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
79
Run the mysql_schema.sql SQL script located in src/main/resources (creates a OW2_MRL database if not exists, then populates it with table schemas and some data).
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
80
81

```
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
82
$ mysql -u username -p < src/main/resources/mysql_schema.sql
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
83
84
```

85
A test dataset is provided in src/test/resources/metric_values.csv .
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
86
87
88
It can be added to the OW2_MRL database as follows:

```
89
$ mysql -u username -p OW2_MRL -Bse "LOAD DATA LOCAL INFILE 'src/test/resources/metric_values.csv' INTO TABLE RawData FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';"
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
90
91
```

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
### Insert real data and metadata

Real data (after data collection as described above) can be added to the OW2_MRL database as follows, supposing that data collection was issued in /tmp/MRL:

1) Insert metric values:

```
$ mysql -u username -p OW2_MRL -Bse "LOAD DATA LOCAL INFILE '/tmp/MRL/metric_values.csv' INTO TABLE RawData FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';"
```

2) Insert MRL thresholds metadata (if any):

First, it is recommended to save the current Metric values (in case of failure):

```
mysql -u root -p OW2_MRL -Bse "DROP TABLE IF EXISTS SpareMetric;"
mysql -u root -p OW2_MRL -Bse "CREATE TABLE SpareMetric as select * from Metric;"
```

Then, to insert threshold values in Metric table:

```
mysql -u username -p OW2_MRL -Bse "LOAD DATA LOCAL INFILE '/tmp/MRL/MRL_thresholds.csv' REPLACE INTO TABLE Metric FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (MetricName,Threshold1,Threshold2,Threshold3,Threshold4,Threshold5,Reverse);"
```

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
117
118
119
120
121
122
123
124
125
126
127
128
## Setup the webapp

- Update src/main/resources/mysql.properties with MySQL credentials
- Build the webapp or copy mysql.properties in webapp WEB-INF/classes (then reload).

Webapp build and execution:

```
$ mvn clean install
$ mvn jetty:run
```

129
The webapp is then available at http://localhost:8080/index.jsp .
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
130
131
132
133
134
135
136

## Automate data collection and database feeding

A collectdata.sh script is provided in frontend-webapp.

First edit collectdata.sh, to:
- Update MYSQL_USER, MYSQL_PASSWORD and MYSQL_DB
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
137
138
- Update MRL_ROOT (.. by default), if needed to run script from another directory or a crontab.
- Eventually change paths to configuration files or executable jars (supposed to be in $MRL_ROOT/data-collection)
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
139
140
141

Before *first* execution of the script, remove test data from MySQL, if any:
```
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
142
mysql> use OW2_MRL;
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
143
144
145
146
147
148
149
150
151
152
153
mysql> delete from RawData;
```

Run the script (can be done from a crontab):

```
$ ./collectdata.sh
```

Check the data is inserted, when no error occurs: using the webapp and/or the mysql shell.

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
154
### Add script to crontab (automation)
Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
155

Pierre-Yves Gibello's avatar
Pierre-Yves Gibello committed
156
157
158
159
160
161
162
163
164
165
166
167
The script is intended to be run daily (at most), as data have a daily timestamp formatted as dd-MM-yyyy .
The date is included in the primary key, so if the script runs twice the same day, only the 1st data set will be included in the database (unless you delete it manually from MySQL).

To add it in the crontab:
- Make sure collectdata.sh is executable, and has MRL_ROOT set to an absolute path (so the cron will find commands).
- Edit the crontab to insert the script, for example using "crontab -e" (for user crontab) or editing /etc/crontab (system-wide).

Note: the following pattern would run the script every day a 4:30 AM:

```
30 4 * * * /path/to/mrl/collectdata.sh >> /tmp/mrl_collect.log
```