Note: I'm not asking about the max depth which is 4.
If it is truly unlimited, can we theoretically have as many CPI calls as we want considering we remain under the compute limit?
there is actually a limit!
The max instruction trace length. It is limited to 64. So max you can do is 63 CPIs.
You will get a "Max instruction trace length exceeded" if you go over ;)
See this feature for details: https://github.com/solana-labs/solana/pull/27938