The stress reaction test was one of the requirements prospective cadets needed to pass before they were accepted into Starfleet Academy. The applicant was tested in a holosuite or holodeck to gauge how they dealt with an unexpected situation, and their performance was graded by a senior officer such as the chief of operations.
When Nog applied to the Academy in 2371, Chief O'Brien proctored his stress reaction test in one of Quark's holosuites using Program Delta 5 9. Nog studied hard and was prepared for the test, but he erroneously believed it would be a test on his performance as pilot of a runabout. The actual test put Nog in Deep Space 9's operations center; O'Brien had intentionally led Nog to believe otherwise, as this made it a truly stressful test rather than routine review of something Nog knew well.
On his first try, Nog failed because Quark had sabotaged the holo-program. When his father Rom found out, Rom went straight to Captain Sisko with the problem and was furious with his brother. Nog later passed and was accepted into the Academy. (DS9: "Facets")