Correct -- deleting everything that was on the drive as new won't stop it from being used as a regular backup drive. They are just optional applications to make scheduled backups etc. However, if you delete everything you may lose the custom icon for that drive (if it has one) in the My Computer window but that won't make any difference to the drive's operation either. You may wish to copy those files to a new folder on another drive for safekeeping just in case you decide you want to install something from them later, then delete the originals.
