IIRC it's an extra 80lbs on the car. I've removed the soft top in the past, but only for trunk access through the backseat passthrough.
In my experience, the soft top ends up more damaged sitting in the garage/shop/shed than it does sitting in the car. One hot sunny day will eliminate all the wrinkles.
So it's really up to you, if you're doing it for weight savings it's really not that much total