Listening is a little more complex than most people realize, and there are various misconceptions about the listening process. Here are a few common misconceptions adapted from "Speech Communication, by William D. Brooks.

Listening ability is related to the intelligence of the listener

Research indicates that while there may be a slight link between the intelligence of the listener and listening ability, the relationship is very slight.

Daily use of listening eliminates the need for special training

Listening is a learned skill, and while we are all able to "listen" to some degree, listening skills can be honed and developed to a high level through training and effort. Some listening can occur and develop naturally through interaction, but training is important. That probably fits your experience that people simply aren't good listeners much of the time.

Improving reading ability also improves listening ability

Obviously one has to understand the language of the speaker to be able to listen and comprehend, but research indicates there is no relatonship between reading ability per se and listening ability.

Listening is easy

This is probably the biggest misconception about listening. Most people believe they are listening, when, in fact they are not, or at least are doing so minimally or ineffectively. It requires the focusing of attention, being open and wanting to really understand another person, and putting aside one's own agenda. These things, and the other requirements for effective listening need to be practiced, and learned, and above all, effort needs to be made.