Ubuntu 20.04 remove stored SSH private key password

• Open "Passwords and Encryption" app ("seahorse")
• Click on the "arrow left" icon in the top left
• Select "Passwords" / Login
• Click on "Password for unlocking xxx@yyy SSH key
• Delete Password

Error Message in SSH was

• sign_and_send_pubkey: signing failed for RSA "/home/myuser/.ssh/id_rsa" from agent: agent refused operation