Favicons are the small image that are associated with a website. They can be seen to the left of the address bar, on the navigation tabs, or in the bookmarks/favorites menu of most browsers.
They are a strictly cosmetic featuire, their main use being to promote stronger brand identity.
This website provides guides on how to implement a favicon, as well as favicon generators and some free favicons.
An example of a favicon being show on the Firefox tab bar-
An example of a favicon in the Internet Explorer bookmarks menu-
An example of a favicon next to the Firefox address bar-