How can I monitor my validator, other than using validator logs?

Question

I want to monitor my validator for development purpose. I want to see debug logs

debug!("CUDA ECDSA for {}", valid_packet_count);
debug!("allocating out..");

And this trace logs

trace!("Starting verify num packets: {}", num_packets);
trace!("elem len: {}", elems.len() as u32);
trace!("packet sizeof: {}", size_of::<Packet>() as u32);
trace!("len offset: {}", PACKET_DATA_SIZE as u32);

and various performance measures and metrics such as

let (result, recv_time_us) = measure_us!({

like this measure_us! macro , to get performace stats for each section

cause in the solana-validator-logs I can only find which is logged by info macro

Answer 1

Enable various log levels like so:

export RUST_LOG=DEBUG

You can also specify different log levels for a comma-separated list of modules:

export RUST_LOG=solana_bpf_loader=debug,solana_runtime::system_instruction_processor=trace

More info: https://docs.rs/env_logger/latest/env_logger/#enabling-logging