A user typically calls an element a button if it looks like a button. On this site, "Ask Question" is a button, but the "Questions", "Tags", "Users", etc. next to it are links. Users don't care if it's an <a> tag or a <button> tag, they only care if it looks pushable. As far as what graphically indicates a button, it can be as little as a border around some text (i.e. Windows 8 UI), or it can be a gradient background with shadowing - as long as it appears as a block that's slightly raised from the background.
Functionality-wise, the line between buttons and links has blurred lately. It used to be that buttons would always perform a task, and links would always take you somewhere else, but that's not necessarily true anymore. I think the main differentiation now is that a button always is a verb - even if it takes you to another page to perform that action. (i.e. "Ask Question") Links are typically (but not always) labels describing where you'll go if you click on it.
Sometimes links are used as buttons, though, where they have the appearance of a link (colored and/or underlined text), but have a verb label that performs an action on the page. The main purpose there is to de-emphasize the action which would potentially be too visually prominent as a button.
