Weighted Context in Collabberry

Problem

In the last RnDAO Collabberry epoch, issues arose due to differing interpretations of whether someone had worked with a peer or not. Some team members felt it was unfair to be assessed by individuals who don’t work directly with them and, therefore, may not have the full picture of their contributions.

Context is not a binary concept where one either has it or does not; people have varying levels of context with their teammates. This variability can lead to feelings of unfairness, particularly when someone with very little context gives scores that negatively impact a peer's overall evaluation, despite not fully understanding the peer's work.

Solution

Introduce a self-stated level of context that each contributor declares about the peer they are assessing. This context weight will influence the final compensation formula, ensuring that evaluations are more reflective of actual understanding and familiarity with the peer's contributions.

Implementation

UI

The P2P assessment form now will include one more input for each contributor - Context Range. 

Algorithm

Context Weight: Each contributor will assign a context weight (from 1 to 5) when evaluating their peers. This weight represents how well the contributor knows the work of the peer they are reviewing.

Weight 1: Very little context

Weight 2: Some context

Weight 3: Moderate context

Weight 4: Good context

Weight 5: Excellent context

Step 1: Calculate Weighted Average Score

Instead of using a simple average of the scores received by each contributor, calculate a Weighted Average Score that accounts for the context weight. This can be done by multiplying each performance score by its corresponding context weight and then dividing by the sum of the context weights.

Where:

WAS - Weighted Average Score

S_i - is the score given by contributor i

W_i is the context weight assigned by contributor i

Step 2: Calculate the Salary Adjustment Multiplier

The Salary Adjustment Multiplier (SAM) will now be calculated using the Weighted Average Score:

This step remains the same as in your original algorithm, but it now uses the Weighted Average Score instead of a simple average.

Step 3: Calculate Individual Adjusted Salary

The adjusted salary is then calculated based on the updated SAM:

Example

Person A receives the following evaluations:

Contributor 1: Score = 4, Context Weight = 3

Contributor 2: Score = 5, Context Weight = 2

Contributor 3: Score = 3, Context Weight = 4

Calculate Weighted Average Score:

Weighted Average Score=(4×3)+(5×2)+(3×4)/(3+2+4)=3.78

SAM = (3.78−3)×0.1=0.078

Assuming Person A has a Base Salary of $2000:

Adjusted Salary=2000×(1+0.078)=2000×1.078=$2156

Weighted Context in Collabberry

Problem

In the last RnDAO Collabberry epoch, issues arose due to differing interpretations of whether someone had worked with a peer or not. Some team members felt it was unfair to be assessed by individuals who don’t work directly with them and, therefore, may not have the full picture of their contributions.

Context is not a binary concept where one either has it or does not; people have varying levels of context with their teammates. This variability can lead to feelings of unfairness, particularly when someone with very little context gives scores that negatively impact a peer's overall evaluation, despite not fully understanding the peer's work.

Solution

Introduce a self-stated level of context that each contributor declares about the peer they are assessing. This context weight will influence the final compensation formula, ensuring that evaluations are more reflective of actual understanding and familiarity with the peer's contributions.

Implementation

UI

The P2P assessment form now will include one more input for each contributor - Context Range. 

Algorithm

Context Weight: Each contributor will assign a context weight (from 1 to 5) when evaluating their peers. This weight represents how well the contributor knows the work of the peer they are reviewing.

Weight 1: Very little context

Weight 2: Some context

Weight 3: Moderate context

Weight 4: Good context

Weight 5: Excellent context

Step 1: Calculate Weighted Average Score

Instead of using a simple average of the scores received by each contributor, calculate a Weighted Average Score that accounts for the context weight. This can be done by multiplying each performance score by its corresponding context weight and then dividing by the sum of the context weights.

Where:

WAS - Weighted Average Score

S_i - is the score given by contributor i

W_i is the context weight assigned by contributor i

Step 2: Calculate the Salary Adjustment Multiplier

The Salary Adjustment Multiplier (SAM) will now be calculated using the Weighted Average Score:

This step remains the same as in your original algorithm, but it now uses the Weighted Average Score instead of a simple average.

Step 3: Calculate Individual Adjusted Salary

The adjusted salary is then calculated based on the updated SAM:

Example

Person A receives the following evaluations:

Contributor 1: Score = 4, Context Weight = 3

Contributor 2: Score = 5, Context Weight = 2

Contributor 3: Score = 3, Context Weight = 4

Calculate Weighted Average Score:

Weighted Average Score=(4×3)+(5×2)+(3×4)/(3+2+4)=3.78

SAM = (3.78−3)×0.1=0.078

Assuming Person A has a Base Salary of $2000:

Adjusted Salary=2000×(1+0.078)=2000×1.078=$2156