[election] RESPONSE NEEDED / Error in merge data used to determine active contributor list

David McBride
 

TSC members,

This is to inform you that I've found that the data from one of the sources (merges) used to determine the active contributor list is in error.  Obviously, this is a problem for the election, but please bear in mind the following:
  • Only one of the six data sources (merges) is in error.  The other data sources are not affected.
  • I've analyzed the effect of either removing the 1 data source, or replacing it with the correct data.  In either case, less than 10% of the list is affected (more details below).
  • None of the nominators or nominees, so far, would be disqualified by either disregarding the data, or replacing it with correct data.
Background
Recently, I was contacted by a member of the community who told me that he thought that the number of contributions reported for him from merges was too high.  I had retrieved the merge data from a Bitergia dashboard in August.  So, I went back to the dashboard and re-ran the same query with the same 1 year time period.  Surprisingly, I was unable to reproduce the data.  I tried varying the time period and the dates to see if I had somehow made a mistake.  However, I was unable to reproduce the results.  So, I contacted Bitergia and explained the issue to them and supplied them the CSV file that I had exported from the query in August.  They also attempted to reproduce the results, but were unable to do so.  They also did spot checks on the current results and determined that they appear to be accurate.  They also checked to see whether there had been any changes to the code since I ran the query in August, but there were none.  I am at a loss to explain the discrepancy.  

Options
I've discussed the situation with Tim Irnich and we came up with three options for handling the error:

Option 1:  Disregard the merge data
In this case, I would recalculate the total contributions without using the merge data.  The analysis that I did shows that this would remove about 15 of the 225 people currently on the active contributor list.  Of course, no one would be added to the list.  None of the current nominators or nominees would be affected.

Option 2:  Replace the merge data
In this case, I would recalculate the total contributions using data newly retrieved from the same dashboard that the original data was retrieved from. Bitergia believes that the dashboard is working correctly and they have done spot checks for verification. The best case is that the problem was a user error on my part and that the data is accurate.  Of course, the downside risk is that there may be a problem that Bitergia has not yet identified and that there are errors in the data.  The analysis shows that this option would remove about 8 of the 225 people currently on the active contributor list.  None of the current nominators or nominees would be affected.

Option 3:  Do Nothing
Based on the analysis of the first two options, less than 10% of the active contributor list is affected by the error in the merge data.  In addition, I've found that none of those that would be affected are either current nominators or nominees.  So, this suggests that another option is to simply do nothing.  This would mean that a handful of people would be on the active contributor list that did not meet the 20 contribution threshold.  Of course, it also means that a small number of people would not be on the list that should be.  

Summary of options

#

Description

Are current nominators or nominees affected?

Number of individuals added to the current list

Number of individuals removed from current list

1

Disregard the merge data

No

0

15 / 225

2

Replace the merge data 

No

Unknown but appears to be low

8 / 225

3

Do nothing

No

0

0 / 225


I'd like to start a discussion with this email that leads to a decision by the TSC on how to proceed, so please respond to this mail with your thoughts or questions.  Thanks.

David

--
David McBride
Release Manager, OPNFV
Email/Google Talk: dmcbride@...
Skype: davidjmcbride1
IRC: dmcbride

Join opnfv-tsc@lists.opnfv.org to automatically receive all group messages.