We automatically create an RSS feed for your blog. It is at yoursite.com/blog/feed/
You would want to use a third party widget that would display your rss feed. Here is an example you can try:
(you may need to sign up first)
After you input your rss url, you can then copy the code they provide and paste it in the source code of where you want to add it to your site!