You can actually delete a message from the conversation window. You just do a long click on the message and the option to delete will appear on the top of the page (Trash can icon).
To delete a full conversation from your inbox tap the 3 vertical dots to the right of your contacts name/number. Then click "Delete".
Looking for iOS? Click here!