To tell the whole truth

Witnesses in court have been required to tell "the truth, the whole truth and nothing but the truth," or words to that effect. However, when that witness begins to elaborate upon a question asked him, the Court will interrupt him and tell him, "Just answer the question." It seems to me that if the Court does not want witnesses to tell the whole truth, then they should not make them swear to do so.