I've done that a few times. Seen a photo opportunity and gone back later to find the moment has passed!
I am always seeing photo opportunities when I am driving in fact I sometimes think I look at everything through an imaginary camera!
I do this! The best photos always come when my hands are busy!