When there is damage to a hole, try to remember these three steps and you’ll always get it right under the Rules.
Step 1 – Is the damage a ball mark? If yes, fix it and it’s over. If it’s not a ball mark, go to the next step.
Step 2 – Does the damage materially change the dimensions of the hole? If no, play without fixing the damage. It would be a sign of good etiquette to fix the damage once all have finished the hole. If yes, go to the next step.
Step 3 – Is a member of the committee readily available? If yes, ask them to fix the damage. If not, you may fix the damage.
You will find a detailed explanation of these three steps in Decision 16-1a/6. In the example you cited, unless the damage to the hole was significant, you would probably have to play without being able to fix the damage. By all means, please fix the damage after all in your group have finished the hole.