How do I proceed with non-custodial Grant CLR payout?

 
This document serves as the bible on how to go about making payouts once the round has ended. At a high level the flow , once the matching distribution (CLR results) have been validated by DAO + internally, it falls onto engineering to ensure the contract is loaded with the funds and grant owners would be able to claim these. NOTE: The reason this is called the non custodial payout is cause we do not send the funds to the grant owners. Instead we uploading a mapping of the address to matching amount onto a newly deployed contract and allow users to claim them
STEP 1 : Ensure Distribution is finalized
Drop an update on #proj-grants on slack and ensure all the contributions which need to be ignored and the matching distribution on the database is exactly what we would want to payout. This is nothing more than asking internally if all analysis is completed and we are good to proceed for payouts
 
STEP 2 : Check if DB has the latest results
Note: Ensure the CLR Rounds are set to active in admin.
notion image
 
We need to ensure the distribution is generated right and stored in the DB before we proceed. There are 2 ways to go about this :
OPTION A (use results from last time CLR was run)
In case if the final distribution was generated right but someone ended up resetting the curve / if you are confident that the distribution from last time the algorithm was run , you can skip recalculating the entire curve by doing the following steps.
  1. reset all CLR calculations by heading over to Grant CLR and by clicking Reset all CLR calculations for all rounds
notion image
  1. Run this on the shell to set all the data in GrantCLRCalculation Table to active
clr_pks = [133,134,135,136,137,138,139] # TODO UPDATE 


# GET CLR
from grants.models import ( Grant, GrantCLR, GrantCLRCalculation)
all_calculations = GrantCLRCalculation.objects.all()
for clr_pk in clr_pks:
    # GET CLR
    clr_round = GrantCLR.objects.get(pk=clr_pk)
    # GET GRANTS
    grant_filters = clr_round.grant_filters
    grants = Grant.objects.filter(**grant_filters)
		
		# RUN THIS TO ENSURE GRANTS IGNORED AREN'T CONSIDERED
		grants = grants.filter(network='mainnet', hidden=False, active=True, is_clr_eligible=True, link_to_new_grant=None)

    clr_calculations = all_calculations.filter(grantclr=clr_pk)
    for grant in grants:
        cal = clr_calculations.filter(grant=grant.pk).order_by('-created_on').first()
        if cal and not cal.latest:
            cal.latest = True
            cal.save()
 
  1. Update the grants CLR prediction curve by running
./manage.py estimate_clr mainnet <CLR-ID-1> slim true && 
./manage.py estimate_clr mainnet <CLR-ID-2> slim true
 
=========================
OPTION B (regenerate results )
If you would like to run the estimate clr again. You can do so by running A or B in the code snippet bellow
 
# A - RUNNING IT ASYNC
./manage.py estimate_clr mainnet <CLR-ID-1> final false &&
./manage.py estimate_clr mainnet <CLR-ID-2> final false

# B - RUNNING IT AS SYNC (RECOMMENDED)
screen  # create a new screen
./manage.py estimate_clr mainnet <CLR-ID-1> final true && 
./manage.py estimate_clr mainnet <CLR-ID-2> final true
=========================
Once you've populated the data, you need to ensure whatever you did worked and you can do that by heading to the following links and ensuring :
notion image
  • grant (check random grants to ensure the clr prediction curve is populated)
notion image
 
STEP 3 : Deploying Payout Contract
Each time we end a round (this might contain multiple CLR rounds within in), we deploy an instance of the matching_contracts hosted at https://github.com/gitcoinco/matching_contracts
 
Follow the instructions on the README.md to how to set up the repo. Things to remember :
  • Ensure the .env is populated with the right variable
 
Deploying & Verifying the contract
# ---- RINKEBY ----
## DEPLOY
yarn deploy:rinkeby
## VERIFY
yarn hardhat verify --constructor-args inputs-rinkeby.js --network rinkeby <RINKEBY_CONTRACT_ADDRESS>

# ---- MAINNET ----
## DEPLOY
yarn deploy:mainnet 
## VERIFY
yarn hardhat verify --constructor-args inputs-mainnet.js --network mainnet <MAINNET_CONTRACT_ADDRESS>
Once you've deployed the contract, verify the contract is verified by ensuring these URL load
notion image
 
Raise a PR against the repo updating the readme with the new contracts. Example: https://github.com/gitcoinco/matching_contracts/pull/7
 
 
STEP 4 : UPDATING web Repo
 
This step involve taking the data we generated in STEP 2 and populating the CLRMatch table
  • finalize populates the CLRMatch with the payouts which needs to happen
  • prepare_final_payout → sets it ready to ready state
# finalize
./manage.py payout_round_noncustodial finalize mainnet --clr_pks=CLR-ID-1,CLR-ID-1 þgm<roundNumber>
# continue? (y/n) -> enter y

# prepare_final_payout
./manage.py payout_round_noncustodial prepare_final_payout mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>
# continue? (y/n) -> enter y
 
STEP 6: Uploading Distribution To Contract
 
 
This step is the most expensive step so make sure you have enough ETH in your address before running set_payouts This address deployed the contract is what will be allowed to upload this data onto the contract.
IMPORTANT The set_payouts doesn't actually upload the distribution to the contract as we have an older web3.py library. So what this steps does is
  • consume the data we added in STEP5 from CLR Match table
  • generates chunks of that data (this is cause the distribution are too large to be uploaded in one go / block )
  • and prints the chunks onto console. (Check attached file)
  • waits for you to manually invoke the setPayouts and upload the chunks (1 transaction per chunk). CHECK VIDEO HERE
 
# set_payouts

./manage.py payout_round_noncustodial set_payouts mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>
# continue? (y/n) -> enter y
# THIS IS A REAL PAYOUT FOR mainnet DAI.
# ARE YOU DOUBLE SECRET SUPER SURE? (y/n)? (y/n) -> enter y

# chunks get generated. Copy this and save it locally
# Head over to your contract -> click write contract tab -> and paste the chunk in
# Refer the video above
 
🔵 NOTE:
  • Save the txnId of the final chunk you uploaded as you will need that in the FINAL STEP
  • Wait until all transactions have completed before proceeding to the next step.
STEP 7: Verifying the Distribution on Chain
 
  • Grab a grants admin address from CLR Match table
  • Select payout and query with the address
  • Compare to ensure the value we have in our DB is the same matching amount as what is on chain.
Repeat this for a few random grants to ensure everything went right
 
notion image
 
 
Now we check to ensure what the total match amount is and compare it with funds available in Contract.
# verify
./manage.py payout_round_noncustodial verify mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>

# this will error out with the following message as contract does not have funds

#* * * * * * * * * * * * * * * * * * * * * * * *
# Contract DAI balance is insufficient
#  Required balance:  651249.414275245880061539
#  Current balance:   0
#  Extra DAI needed:  651249.414275245880061539 Contract needs another 651249.414275245880061539 DAI
#* * * * * * * * * * * * * * * * * * * * * * * * *
 
STEP 8: Finalize Contract
 
🔴 NOTE: Once Finalized , YOU WILL NOT BE TO INVOKE setPayouts and add more chunks
 
  • Head over to contract's write tab
  • Select Finalize and sign the transaction NOTE:
notion image
 
 
STEP 9: Transfer Funds From Multisig To Contract
 
Draft a mail with the contract details send it over to multisig owners capturing
  • where the contract was deployed
  • how much funds needs to be transferred to it
Reach out to Kyle/Kevin on whom to send out the mail to.
 
🔵 Note: you can get the funds needed to be loaded from Step 7
STEP 10: Verify If Contract Has Enough Funds
 
Once we reach this step, the funds are now present on contract and we validate this running the verify command like we did in STEP 7.
 
# verify
./manage.py payout_round_noncustodial verify mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>

# If sufficent funds are present -> you'd see the below message on the console
# Contract balance of {dai_balance} DAI is exactly equal to the required amount
 
STEP 11: Enable Payouts
 
  • Head over to contract's write tab
  • Select enablePayouts and sign the transaction
notion image
 
Once this step is complete, users should be able to claim funds via the web UI OR on the contract write tab by clicking on claimPayout
 
STEP 12: SAVE CONTRIBUTION + SEND EMAIL
 
🔴 NOTE: EMAIL ARE SENT OUT TO USERS AT THE END OF THIS STEP.
In this step:
  • contributions are created and saved in DB based on CLRMatch
  • emails are sent out to the grant owners
  • activity are created
 
By running this step you should have :
  • Uploaded the distribution (make sure you have the final chunks txn ID) (step 6)
  • Contract has been finalized (step 8)
  • Funds have been transferred into the contract (step 9)
  • Verify that the contract has enough funds to payout (step 10)
  • Payouts have been enabled (step 11)
 
# notify_users
./manage.py payout_round_noncustodial notify_users mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>

# enter a txn id -> Enter the last txn id after uploading the chunk in step 6
STEP 13: WRAP UP
 
  • Do ensure you drop this on the channel and update the progress so that folks are aware.
  • Keep track of all the transactions / contract addresses
  • Having the chunks in a file makes it easier as opposed to copy pasting it from the console for each chunk
This the template I had used for round 10 and I'd recommend using this to keep an update on the progress : https://gitcoincore.slack.com/archives/CBDTKB59A/p1627017455146600
 
 
 

How do I proceed with non-custodial Grant CLR payout?

 
This document serves as the bible on how to go about making payouts once the round has ended. At a high level the flow , once the matching distribution (CLR results) have been validated by DAO + internally, it falls onto engineering to ensure the contract is loaded with the funds and grant owners would be able to claim these. NOTE: The reason this is called the non custodial payout is cause we do not send the funds to the grant owners. Instead we uploading a mapping of the address to matching amount onto a newly deployed contract and allow users to claim them
STEP 1 : Ensure Distribution is finalized
Drop an update on #proj-grants on slack and ensure all the contributions which need to be ignored and the matching distribution on the database is exactly what we would want to payout. This is nothing more than asking internally if all analysis is completed and we are good to proceed for payouts
 
STEP 2 : Check if DB has the latest results
Note: Ensure the CLR Rounds are set to active in admin.
notion image
 
We need to ensure the distribution is generated right and stored in the DB before we proceed. There are 2 ways to go about this :
OPTION A (use results from last time CLR was run)
In case if the final distribution was generated right but someone ended up resetting the curve / if you are confident that the distribution from last time the algorithm was run , you can skip recalculating the entire curve by doing the following steps.
  1. reset all CLR calculations by heading over to Grant CLR and by clicking Reset all CLR calculations for all rounds
notion image
  1. Run this on the shell to set all the data in GrantCLRCalculation Table to active
clr_pks = [133,134,135,136,137,138,139] # TODO UPDATE 


# GET CLR
from grants.models import ( Grant, GrantCLR, GrantCLRCalculation)
all_calculations = GrantCLRCalculation.objects.all()
for clr_pk in clr_pks:
    # GET CLR
    clr_round = GrantCLR.objects.get(pk=clr_pk)
    # GET GRANTS
    grant_filters = clr_round.grant_filters
    grants = Grant.objects.filter(**grant_filters)
		
		# RUN THIS TO ENSURE GRANTS IGNORED AREN'T CONSIDERED
		grants = grants.filter(network='mainnet', hidden=False, active=True, is_clr_eligible=True, link_to_new_grant=None)

    clr_calculations = all_calculations.filter(grantclr=clr_pk)
    for grant in grants:
        cal = clr_calculations.filter(grant=grant.pk).order_by('-created_on').first()
        if cal and not cal.latest:
            cal.latest = True
            cal.save()
 
  1. Update the grants CLR prediction curve by running
./manage.py estimate_clr mainnet <CLR-ID-1> slim true && 
./manage.py estimate_clr mainnet <CLR-ID-2> slim true
 
=========================
OPTION B (regenerate results )
If you would like to run the estimate clr again. You can do so by running A or B in the code snippet bellow
 
# A - RUNNING IT ASYNC
./manage.py estimate_clr mainnet <CLR-ID-1> final false &&
./manage.py estimate_clr mainnet <CLR-ID-2> final false

# B - RUNNING IT AS SYNC (RECOMMENDED)
screen  # create a new screen
./manage.py estimate_clr mainnet <CLR-ID-1> final true && 
./manage.py estimate_clr mainnet <CLR-ID-2> final true
=========================
Once you've populated the data, you need to ensure whatever you did worked and you can do that by heading to the following links and ensuring :
notion image
  • grant (check random grants to ensure the clr prediction curve is populated)
notion image
 
STEP 3 : Deploying Payout Contract
Each time we end a round (this might contain multiple CLR rounds within in), we deploy an instance of the matching_contracts hosted at https://github.com/gitcoinco/matching_contracts
 
Follow the instructions on the README.md to how to set up the repo. Things to remember :
  • Ensure the .env is populated with the right variable
 
Deploying & Verifying the contract
# ---- RINKEBY ----
## DEPLOY
yarn deploy:rinkeby
## VERIFY
yarn hardhat verify --constructor-args inputs-rinkeby.js --network rinkeby <RINKEBY_CONTRACT_ADDRESS>

# ---- MAINNET ----
## DEPLOY
yarn deploy:mainnet 
## VERIFY
yarn hardhat verify --constructor-args inputs-mainnet.js --network mainnet <MAINNET_CONTRACT_ADDRESS>
Once you've deployed the contract, verify the contract is verified by ensuring these URL load
notion image
 
Raise a PR against the repo updating the readme with the new contracts. Example: https://github.com/gitcoinco/matching_contracts/pull/7
 
 
STEP 4 : UPDATING web Repo
 
This step involve taking the data we generated in STEP 2 and populating the CLRMatch table
  • finalize populates the CLRMatch with the payouts which needs to happen
  • prepare_final_payout → sets it ready to ready state
# finalize
./manage.py payout_round_noncustodial finalize mainnet --clr_pks=CLR-ID-1,CLR-ID-1 þgm<roundNumber>
# continue? (y/n) -> enter y

# prepare_final_payout
./manage.py payout_round_noncustodial prepare_final_payout mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>
# continue? (y/n) -> enter y
 
STEP 6: Uploading Distribution To Contract
 
 
This step is the most expensive step so make sure you have enough ETH in your address before running set_payouts This address deployed the contract is what will be allowed to upload this data onto the contract.
IMPORTANT The set_payouts doesn't actually upload the distribution to the contract as we have an older web3.py library. So what this steps does is
  • consume the data we added in STEP5 from CLR Match table
  • generates chunks of that data (this is cause the distribution are too large to be uploaded in one go / block )
  • and prints the chunks onto console. (Check attached file)
  • waits for you to manually invoke the setPayouts and upload the chunks (1 transaction per chunk). CHECK VIDEO HERE
 
# set_payouts

./manage.py payout_round_noncustodial set_payouts mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>
# continue? (y/n) -> enter y
# THIS IS A REAL PAYOUT FOR mainnet DAI.
# ARE YOU DOUBLE SECRET SUPER SURE? (y/n)? (y/n) -> enter y

# chunks get generated. Copy this and save it locally
# Head over to your contract -> click write contract tab -> and paste the chunk in
# Refer the video above
 
🔵 NOTE:
  • Save the txnId of the final chunk you uploaded as you will need that in the FINAL STEP
  • Wait until all transactions have completed before proceeding to the next step.
STEP 7: Verifying the Distribution on Chain
 
  • Grab a grants admin address from CLR Match table
  • Select payout and query with the address
  • Compare to ensure the value we have in our DB is the same matching amount as what is on chain.
Repeat this for a few random grants to ensure everything went right
 
notion image
 
 
Now we check to ensure what the total match amount is and compare it with funds available in Contract.
# verify
./manage.py payout_round_noncustodial verify mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>

# this will error out with the following message as contract does not have funds

#* * * * * * * * * * * * * * * * * * * * * * * *
# Contract DAI balance is insufficient
#  Required balance:  651249.414275245880061539
#  Current balance:   0
#  Extra DAI needed:  651249.414275245880061539 Contract needs another 651249.414275245880061539 DAI
#* * * * * * * * * * * * * * * * * * * * * * * * *
 
STEP 8: Finalize Contract
 
🔴 NOTE: Once Finalized , YOU WILL NOT BE TO INVOKE setPayouts and add more chunks
 
  • Head over to contract's write tab
  • Select Finalize and sign the transaction NOTE:
notion image
 
 
STEP 9: Transfer Funds From Multisig To Contract
 
Draft a mail with the contract details send it over to multisig owners capturing
  • where the contract was deployed
  • how much funds needs to be transferred to it
Reach out to Kyle/Kevin on whom to send out the mail to.
 
🔵 Note: you can get the funds needed to be loaded from Step 7
STEP 10: Verify If Contract Has Enough Funds
 
Once we reach this step, the funds are now present on contract and we validate this running the verify command like we did in STEP 7.
 
# verify
./manage.py payout_round_noncustodial verify mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>

# If sufficent funds are present -> you'd see the below message on the console
# Contract balance of {dai_balance} DAI is exactly equal to the required amount
 
STEP 11: Enable Payouts
 
  • Head over to contract's write tab
  • Select enablePayouts and sign the transaction
notion image
 
Once this step is complete, users should be able to claim funds via the web UI OR on the contract write tab by clicking on claimPayout
 
STEP 12: SAVE CONTRIBUTION + SEND EMAIL
 
🔴 NOTE: EMAIL ARE SENT OUT TO USERS AT THE END OF THIS STEP.
In this step:
  • contributions are created and saved in DB based on CLRMatch
  • emails are sent out to the grant owners
  • activity are created
 
By running this step you should have :
  • Uploaded the distribution (make sure you have the final chunks txn ID) (step 6)
  • Contract has been finalized (step 8)
  • Funds have been transferred into the contract (step 9)
  • Verify that the contract has enough funds to payout (step 10)
  • Payouts have been enabled (step 11)
 
# notify_users
./manage.py payout_round_noncustodial notify_users mainnet --clr_pks=CLR-ID-1,CLR-ID-1 --clr_round=<roundNumber>

# enter a txn id -> Enter the last txn id after uploading the chunk in step 6
STEP 13: WRAP UP
 
  • Do ensure you drop this on the channel and update the progress so that folks are aware.
  • Keep track of all the transactions / contract addresses
  • Having the chunks in a file makes it easier as opposed to copy pasting it from the console for each chunk
This the template I had used for round 10 and I'd recommend using this to keep an update on the progress : https://gitcoincore.slack.com/archives/CBDTKB59A/p1627017455146600