A man once tried annoying him by insulting his father, but didn't know that Norvalians don't have fathers.
In 2377, he became romantically involved with Kathryn Janeway, who had been kidnapped and brain washed to think she was a laborer on Quarra. He met her at the power plant and asked her out on a date. Later he would see her at a local bar that was frequented by the workers. He walked her home, and she accepted his invitation for dinner. Janeway decided to make dinner for them, but she burned the dinner. They became romantic with each other, and Jaffen spent the night with her. After they dated for three months, she decided to move in with him.
Later when Chakotay tried to tell Janeway that she was captain of Voyager and had been brainwashed, she told Jaffen the story. He became jealous believing that Chakotay wanted Janeway for himself. He turned Chakotay into the authorities. After he and Janeway came to believe Chakotay's story he helped her break into the power plant so she could contact Voyager using the transponder in the plant. He protected her from the guards while she contacted the ship.
The brainwashing plot was foiled and Janeway and the rest of the crew were restored to their true identity. Janeway decided to return to her former life. She stated that she would have offered Jaffen a job in engineering, but it wouldn't be appropriate. He decided to give her an engineering trinket, to remember him by. (VOY: "Workforce", "Workforce, Part II")