starting a devnet validator with geyser plugin - solana version 1.13.3:
exec solana-validator \
--identity ~/validator-keypair-devnet.json \
--vote-account ~/vote-account-keypair-devnet.json \
--entrypoint entrypoint.devnet.solana.com:8001 \
--entrypoint entrypoint2.devnet.solana.com:8001 \
--entrypoint entrypoint3.devnet.solana.com:8001 \
--ledger /mnt/solana-ledger \
--dynamic-port-range 8000-8020 \
--no-voting \
--rpc-port 8899 \
--rpc-bind-address 127.0.0.1 \
--private-rpc \
--full-rpc-api \
--wal-recovery-mode skip_any_corrupted_record \
--log /var/log/sol/validator.log \
--accounts /mnt/solana-accounts/accounts \
--minimal-snapshot-download-speed 104857600 \
--geyser-plugin-config /home/sol/geyser-plugin-nats/config.json
validator starts with plugin:
[2023-02-21T23:08:42.464540129Z INFO solana_geyser_plugin_manager::geyser_plugin_service] Starting GeyserPluginService from config files: ["/home/sol/geyser-plugin-nats/config.json"]
[2023-02-21T23:08:43.362028528Z INFO geyser_plugin_nats] Loading "GeyserPluginNats" from config_file "/home/sol/geyser-plugin-nats/config.json"
[2023-02-21T23:08:43.362179237Z INFO solana_geyser_plugin_manager::geyser_plugin_service] Started GeyserPluginService
I'm watching solana-validator --ledger //mnt/solana-ledger monitor and it shows that the validator get caught up and then shortly thereafter exits with:
Feb 21 22:59:09 systemd[1]: sol.service: Main process exited, code=killed, status=11/SEGV
Feb 21 22:59:09 systemd[1]: sol.service: Failed with result 'signal'.
Logs immediately preceding exit:
[2023-02-21T23:14:40.775088863Z INFO solana_poh::poh_recorder] reset poh from: 5UrcWHQCb1CUeABruohqNbahSHNXZEA9hsH5EpA4s3qU,12629413319,197334582 to: 5UrcWHQCb1CUeABruohqNbahSHNXZEA9hsH5EpA4s3qU,197334583
[2023-02-21T23:14:40.775114185Z INFO solana_core::replay_stage] 8PBdtQdTMG8iHehcVR5De6cLec5gonV1fjyD5qopHiQB reset PoH to tick 12629413376 (within slot 197334583). I am not in the leader schedule yet
[2023-02-21T23:14:40.775164038Z INFO solana_core::replay_stage] new fork:197334584 parent:197334583 root:197334571
[2023-02-21T23:14:40.776013954Z INFO solana_metrics::metrics] datapoint: bank-timestamp get_timestamp_estimate_us=601i
[2023-02-21T23:14:40.776022905Z INFO solana_metrics::metrics] datapoint: bank-timestamp-correction slot=197334584i from_genesis=1676014849i corrected=1677020841i ancestor_timestamp=1677020841i
[2023-02-21T23:14:40.776148192Z INFO solana_metrics::metrics] datapoint: bank-new_from_parent-heights slot=197334584i block_height=186293213i parent_slot=197334583i bank_rc_creation_us=3i total_elapsed_us=959i status_cache_rc_us=0i fee_components_us=1i blockhash_queue_us=7i stakes_cache_us=0i epoch_stakes_time_us=0i builtin_programs_us=1i rewards_pool_pubkeys_us=0i cached_executors_us=2i transaction_debug_keys_us=0i transaction_log_collector_config_us=0i feature_set_us=0i ancestors_us=4i update_epoch_us=0i update_sysvars_us=809i fill_sysvar_cache_us=114i
[2023-02-21T23:14:40.776159732Z INFO solana_metrics::metrics] datapoint: bank-executor-cache-stats slot=197334583i hits=266i misses=1i evictions=0i insertions=1i replacements=0i one_hit_wonders=0i
[2023-02-21T23:14:40.809284028Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=2762i slot=197334646i last_index=47i num_repaired=48i num_recovered=0i
[2023-02-21T23:14:40.809299839Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=2963i slot=197334655i last_index=47i num_repaired=48i num_recovered=0i
[2023-02-21T23:14:40.811543192Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=3137i slot=197334661i last_index=29i num_repaired=30i num_recovered=0i
[2023-02-21T23:14:40.813359543Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=2967i slot=197334658i last_index=70i num_repaired=71i num_recovered=0i
[2023-02-21T23:14:46.775909988Z INFO solana_validator] solana-validator 1.13.3 (src:3271b83d; feat:4011803773)
[2023-02-21T23:14:46.775930619Z INFO solana_validator] Starting validator with: ArgsOs {
inner: [
"solana-validator",
How can I get more information on what might be going wrong?
rustup default 1.60, rebuild plugin, restart validator.