Hi again leyakoreya
Sorry, after checking it deeper, I can see this option is already available.
Just hover over the attachment and you will see the delete icon on the top right side:
Just note that, actually, you have to save the product first for this button to be shown. I'm going to fix this.