The brain probe was a device in the Captain Proton holoprogram. It was used as a method of torture in order to gain information.
The Twin Mistresses of Evil, Malicia and Demonica, attempted to use one on Buster Kincaid before being stopped by Captain Proton. (VOY: "Thirty Days")