require(kaiaulu)
require(data.table)
require(JirAgileR)
require(knitr, quietly = T)
require(dplyr, quietly = T)
require(jsonlite)
Introduction
This example is adapted from the JirAgileR package README.md.
As usual, the first step is to load the project configuration file.
Project Configuration File
In this notebook, we will obtain data from the issue tracker JIRA. We will use Apache’s Geronimo open source project. Refer to the conf/
folder on Kaiaulu’s git repository for Geronimo and other project configuration files. It is in this project configuration file we specify where Kaiaulu can find the jira sources from Geronimo. We will use the issue_tracker -> jira fields only. In regards to the “issues” and “issue_comments” fields, these should be set to paths where you want to store the jira data. Then, you can access this jira data later using these same paths.
conf <- yaml::read_yaml("../conf/geronimo.yml")
issue_tracker_domain <- conf[["issue_tracker"]][["jira"]][["domain"]]
issue_tracker_project_key <- conf[["issue_tracker"]][["jira"]][["project_key"]]
save_path_issue_tracker_issues <- conf[["issue_tracker"]][["jira"]][["issues"]]
save_path_issue_tracker_issue_comments <- conf[["issue_tracker"]][["jira"]][["issue_comments"]]
Specifying the JIRA credentials
In the project configuration file, we define the domain which JIRA is hosted. The example configuration file uses the domain for Apache Software Foundation. We can use this information to queue the domain to identify all project keys available in its JIRA:
# Save credentials to pass them only one time
save_jira_credentials(domain = issue_tracker_domain)
# Get full list of projects in domain
apache_jira_projects <- data.table(get_jira_projects()) #%>%
#select(key, name)
You can load apache_jira_projects
for the list of projects available. However, this step is optional.
Download Issues (Without Comments)
In the project configuration file, we also specify the project configuration key (you can also type it directly for the function call). Here, we used the key GERONIMO
. You can also manually explore the existing issues from a given project by visiting it on your browser. Please see the function documentation warning for get_jira_issues
on respecting the number of function calls to the server to avoid being IP blocked.
The following function will download all the available issues. For more details, see the associated package documentation. This data can be used to obtain the bug_count
metric. See the associated R notebook bug_count.Rmd
for details.
We will download the data on verbose
, and as json (as.data.frame = FALSE
), as the turning the later appears to cause column names to not match values, ignores the requested fields, and gives inconsistent column names. Kaiaulu implements its own parser for the json file, so you will still be able to obtain a table in the end without the associated issues.
# Retrieve the issues from a single project - in this case the project QTWB from bugreports.qt.io. See documentation to define which fields to see
json_issues <- get_jira_issues(jql_query = paste0("project='",issue_tracker_project_key,"'"),
fields = c("summary",
"description",
"creator",
"assignee",
"reporter",
"issuetype",
"status",
"resolution",
"components",
"created",
"updated",
"resolutiondate"),
verbose=TRUE,
as.data.frame = FALSE)
jsonlite::write_json(json_issues,save_path_issue_tracker_issues)
The json will be downloaded on the path specified in the project configuration file, which by default is kaiaulu/kaiaulu/rawdata/issue_tracker
. We can then use Kaiaulu’s function to parse the data into a tabular format. Since our request did not include the comment
field, only the issues table will be available. A few rows of the json issues is shown next:
jira_issues_list <- parse_jira(save_path_issue_tracker_issues)
jira_issues <- jira_issues_list[["issues"]]
jira_comments <- jira_issues_list[["comments"]]
kable(jira_issues[7:8])
issue_key | issue_summary | issue_type | issue_status | issue_resolution | issue_components | issue_description | issue_created_datetimetz | issue_updated_datetimetz | issue_resolution_datetimetz | issue_creator_id | issue_creator_name | issue_creator_timezone | issue_reporter_id | issue_reporter_name | issue_reporter_timezone | issue_assignee_id | issue_assignee_name | issue_assignee_timezone |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
GERONIMO-782 | ejb ws deployment system does not use gbean builder references | Bug | Closed | Fixed | deployment | The ejb builder uses static methods on AxisServiceBuilder rather than the gbean interface exposed by AxisBuilder. Fixing this in a reasonable amount of time will require removing xfire support. David Blevins has indicated that the xfire support needs to be completely rewritten anyway so I will proceed with this. | 2005-07-20T09:36:40.000+0000 | 2006-08-07T15:05:51.000+0000 | 2005-07-23T14:56:54.000+0000 | djencks | David Jencks | America/Los_Angeles | djencks | David Jencks | America/Los_Angeles | djencks | David Jencks | America/Los_Angeles |
GERONIMO-781 | TomcatModuleBuilderTest busted | Bug | Closed | Fixed | Tomcat | As a result of the recent changes to the Tomcat GBeans, the TomcatModuleBuilderTest blows up. It creates an entire Tomcat GBean structure by hand, and now sets invalid properties on GBeans (engine on host not hosts on engine) and when I tried to fix it I was stuck on complaints that “localhost is not a legal virtual host” or something to that effect. I’ve commented out the entire test for now, but if someone who understands the new GBean structure could take a peek that would be great. | 2005-07-20T07:38:23.000+0000 | 2006-08-07T15:05:51.000+0000 | 2005-07-20T08:49:34.000+0000 | ammulder | Aaron Mulder | Etc/UTC | ammulder | Aaron Mulder | Etc/UTC | jgenender | Jeff Genender | America/Denver |
Download Issue with Comments
In the same manner as before, we can perform the same function call, but including the field comment
. This will result in the same table being generated but with the additional comment information per issue (if an issue has more than one comment, the issue id is repeated for each different comment). The comment is shown on the column comment_comments_id
.
The data of this table can be used to calculate social smell metrics
, as it represents a form of developer communication. A notebook discussing how to use JIRA data as communication network and/or combining to mailing list data will be made available in the future.
json_issue_comments <- get_jira_issues(jql_query = paste0("project='",issue_tracker_project_key,"'"),
fields = c("summary",
"description",
"creator",
"assignee",
"reporter",
"issuetype",
"status",
"resolution",
"components",
"created",
"updated",
"resolutiondate",
"comment"),
verbose=TRUE,
as.data.frame = FALSE)
jsonlite::write_json(json_issue_comments,save_path_issue_tracker_issue_comments)
Since this time around we requested the issue data and comments, when using the parse_jira
function, both the issues and comments table will be available from the parser. Since the issue table was already displayed, the following show a few rows of the issue comments table:
jira_issue_comments <- parse_jira(save_path_issue_tracker_issue_comments)
jira_issues <- jira_issue_comments[["issues"]]
jira_comments <- jira_issue_comments[["comments"]]
kable(jira_comments[55:56])
issue_key | comment_id | comment_created_datetimetz | comment_updated_datetimetz | comment_author_id | comment_author_name | comment_author_timezone | comment_author_update_id | comment_author_update_name | comment_author_update_timezone | comment_body |
---|---|---|---|---|---|---|---|---|---|---|
GERONIMO-767 | 12320101 | 2005-08-26T17:09:27.000+0000 | 2005-08-26T17:09:27.000+0000 | djencks | David Jencks | America/Los_Angeles | djencks | David Jencks | America/Los_Angeles | duplicate of 892 |
GERONIMO-766 | 12426359 | 2006-08-07T23:08:55.000+0000 | 2006-08-07T23:08:55.000+0000 | ammulder | Aaron Mulder | Etc/UTC | ammulder | Aaron Mulder | Etc/UTC | Can use “login” command to the deployer to save credentials and then call the tool from a script. |