Maybe check how anchor test does it? Pretty sure it spins up a local validator.

Marking this as the answer since it's more general (and are steps 4-6 in Dhrumil's answer), but both are great, thanks :)

What if you typo/mis-copy the new upgrade authority, or your wallet keypair is corrupted/mis-saved as you do these upgrades? I guess there's a small window where your program can become 100% bricked. That small window multiply by the # of times you may be doing this (probably manually) can magnify the risk of human error.