From the Quick Guide to the Rules of Golf…
If, after playing a shot, you think your ball may be lost outside a water hazard or out of bounds you should play a provisional ball. You must announce that it is a provisional ball and play it before you go forward to search for the original ball.
If the original ball is lost (other than in a water hazard) or out of bounds, you must continue with the provisional ball, under penalty of one stroke. If the original ball is found in bounds within 5 minutes, you must continue play of the hole with it, and must stop playing the provisional ball.
From the Rules of Golf
If a ball may be lost outside a water hazard or may be out of bounds, to save time the player may play another ball provisionally in accordance with Rule 27-1.
Additional Information *
In other words, if it is known or it is virtually certain that the ball is within the margins of a water hazard (or lateral water hazard) then the player may not play a provisional ball. The reason for this is that the relief options for a ball lost in a water hazard (Rule 26-1a) are significantly more advantageous than those for a ball that is lost or out of bounds Rule 27-1).
There are two Decisions that clearly illustrate this
27-2a/2 Provisional Ball Played Solely in Belief Original Ball Might Be in Water Hazard
Q. A player’s tee shot might be in a water hazard, but clearly it is not lost outside a water hazard or out of bounds. The player announces that, since his ball might be in the hazard, he is going to play a provisional ball and he does so. Rule 27-2a seems to prohibit a provisional ball in the circumstances. What is the ruling?
A. The player did not play a provisional ball which, according to the Definition of “Provisional Ball,” is a ball played under Rule 27-2 for a ball which may be lost outside a water hazard or may be out of bounds. The second ball from the tee was in play since it was not a provisional ball.
27-2a/2.2 Possibility That Original Ball Is in Water Hazard May Not Preclude Play of Provisional Ball
Q. If a player’s original ball may have come to rest in a water hazard, is he precluded from playing a provisional ball?
A. No. Even though the original ball may be in a water hazard, the player is entitled to play a provisional ball if the original ball might also be lost outside the water hazard or out of bounds. In such a case, if the original ball is found in the water hazard, the provisional ball must be abandoned — Rule 27-2c (Formerly 27-2c/1)
The point made in the answer to this second Decision is important. Whether a ball may be lost inside or outside of a hazard may depend a lot on the surrounding terrain. If a wide fairway leads straight down to a water hazard then the ball will either be found on the fairway or will be in the water hazard. But if there is long grass and/or trees around the water hazard then the ball could be lost anywhere inside or outside of the hazard because it could be hidden in the deep rough or could have been deflected off trees in any direction.
* Information taken from Barry Rhodes blog.