A client just asked this so I thought I’d share my answer 🙂
Categories are how you organize your posts.  For example, on my organizing site I have Orderly@Home, Orderly On the Go, Orderly Resources and so on.
Tags are the keywords that relate to the content of your posts or pages.  So if I do a post on organizing your vehicle, my keywords might be vehicle,organizing and car clutter.  As long as those words appear in my post or page, search engines will see that the tags are relevant to the text and I’d get a good ranking for that post/page.