This isn't perfect, but a solution I've used when running into this has been running test-validator separately, and using --clone-upgradeable-program (need to be using 1.18.10) with IDs like Metaplex.

Additionally, if you want to automate this with ```anchor test```, you can grab the program binary with ```solana program dump``` like you have been, and use ```[[test.genesis]]``` with the programId and binary to load the program at genesis rather than cloning from mainnet.

Example:
```
# Wormhole, like your example
[[test.genesis]]
address = "worm2ZoG2kUd4vFXhvjh93UUH596ayRfgQ2MgjNMTth"
program = "wormhole.so"
```

For more information about why this may be the case, this answer from Jon C details it: https://solana.stackexchange.com/questions/13041/transaction-confirmation-expires-on-test-validator