In 2371, a team of specialists, namely xenobiologists and exopsychologists were planned to be assembled on Starbase 201 awaiting for the arrival of a Jem'Hadar youth discovered by Quark. Odo, who disliked the idea of the youth being "studied like a laboratory specimen," as proposed by Starfleet, offered to raise the youth so as to be treated like a civilized being. (DS9: "The Abandoned")
Like many concepts in the Trek universe, exopsychology is a neologism based on the assumption that a familiar concept (in this case, psychology) will expand beyond the scope of current comprehension in the future.
The prefix exo means "outside," and the derivation of exopsychology appears to have been based on the word exobiology, which the American Heritage Dictionary currently defines as "the branch of biology that deals with the search for extraterrestrial life and the effects of extraterrestrial surroundings on living organisms. Also called astrobiology, space biology." Thus, exopsychology would presumably be the same as "space psychology," although this connection has not been established.