Using only the Contacts tab, there isn't a quick way. You have to select each individual contact and then delete the lot of them. There are ways to do so relatively quickly, however, they are more complex than simply selecting all your contacts and removing them.
In order to avoid accidental deletion, we made the decision not to include a delete all option. Once a contact is deleted, the details are no longer stored, and we can't restore deleted contacts.
By going to the Search Contacts tab, you are able to begin the process. Ensure that the Search Lists dropdown reads "All Lists" and then select Create A New List in the final dropdown. This will dump all your contacts into a delete list. Then go to the Lists tab, select your delete list and view it. From here you can delete all contacts, and you can choose if they are removed from just the list or the database overall.
In part, this was the best way put forward during development to prevent accidental deletes. The process will become more streamlined as Contact Relationship Manager is updated.