The film begins in San Francisco's Union Square. A couple, Mark and Ann, walk in circles around the square having a somewhat vague conversation, confessing their love for one another, and making a plan for the coming Sunday at a hotel. Harry Caul, a talented, reclusive surveillance expert, follows the couple and surveils their conversation, with the help of his assistant Stan and Paul, a cop who works with the surveillance team. Using cameras and recording devices, the team records the conversation between the couple for a private client, the CEO of a corporation.
Harry goes home after the job to his apartment, which has three locks and an extensive burglar alarm system. It is Harry's birthday, and the woman at the front desk of his building has taken the liberty of delivering Harry's mail directly into his living room, which makes him paranoid. He calls down to tell her that he would like to have the only key to his apartment. The next day, Harry goes to work at his office, which is in a large warehouse, and listens obsessively to the tapes of the couple. Even though the surveillance was just a gig, Harry is beginning to feel personally invested, and wonders what the couple's conversation means.
Later, Harry calls his employer from a payphone, and he makes an appointment for 2:30 the next day. Harry visits his mistress, Amy, that night, who is waiting for him in a small apartment. He brings her wine that he received for his birthday, and it becomes clear that Harry financially supports Amy as he leaves money for rent on her counter. When Amy tries to ask Harry personal questions, he deflects them, and she tells him that she doesn't want to see him anymore.
Harry goes to his appointment at the corporate office of his employer. After waiting a long time, the director never arrives, and his assistant Martin Stett says that he will take the tapes from Harry Caul in the director's absence. Harry insists that he had instructions to bring the tapes directly to the director, and Martin and Harry struggle over the tapes. Harry leaves with the tapes, suspicious of Martin. As Harry leaves, Martin warns him, "Be careful," and Harry sees Mark near an elevator. Then, once on the elevator, he sees Ann. It turns out that Ann and Mark—the couple on the tapes—both work in the office.
Harry becomes all the more obsessed with the tapes, eventually discovering a part of them in which Mark tells Ann, "He'd kill us if he got the chance." At a Catholic church, Harry goes to confession and confesses to the priest that his work has led to the deaths of innocent people in the past.
The following day Harry attends a convention for people in the surveillance industry. In the middle of the convention, he realizes that Martin Stett has followed him there. He also runs into his friend Paul, who introduces Harry to his main professional rival, Bernie Moran. When Harry learns that his former assistant Stan has gone to work for Bernie, he tells Stan that he is working on a complicated case and needs Stan to come back to working for him. Then, he tries to call Amy, but finds that her number has been disconnected. He also runs into Martin Stett, who tells Harry to deliver the tapes to him and the director that coming Sunday.
Harry, Paul, Stan, Bernie and a group of other people from the convention go to Harry's office for an impromptu party after the convention. A woman named Meredith comes along, and she seduces Harry and pulls him aside to have a private chat. During their chat, Harry divulges that he misses Amy and has an uncharacteristically emotionally honest interaction with Meredith. When Harry and Meredith return to the party, Bernie references the case that Harry worked on a few years earlier which led to the brutal murder of three people. Stan then reveals the existence of the tapes that Harry has been obsessed with. When Bernie asks Harry to go into business with him and Harry refuses, Bernie begins playing a tape of the private conversation between Harry and Meredith. Having slipped a surveillance device—in the form of a pen—into Harry's breastpocket earlier, Bernie was able to record the entire conversation. Harry is livid and throws everyone out of the office. Meredith stays with him and they get ready to go to bed.
Harry continues to obsess over the tapes, listening to them before going to bed with Meredith. She eventually coaxes him into bed. That night, Harry dreams that he is following Ann, the woman from the tapes, through a foggy landscape. Amy carries the tapes, and Harry tells her intimate details about his childhood and his relationship to his parents. When Harry wakes up, he finds that Meredith has left and stolen the tapes.
Martin Stett calls Harry at home, informing him that they have been watching Harry all along and that they are in possession of the tapes. He then requests that Harry bring the photographs to the office, so that they can pay him. Harry goes to the office as instructed, where he meets the director, who is listening to the tapes. Harry discovers that Ann is the director's wife, and that she is having an affair with Mark. When Harry tries to find out if they will kill Ann, the director and Martin give him vague answers.
Worried for Ann's well-being, Harry decides to go to the hotel in an attempt to possibly save her. When he cannot reserve room 773, the room that Mark and Ann discussed on the tapes, he gets the room next door. In his room, he makes a hole in the wall and listens in on the conversation in 773, and hears the director and Ann arguing. When he goes out on the balcony, he hears Ann's scream and sees a bloody handprint on the divider. Disturbed by the belief that Ann has been murdered, Harry goes into his room and lies down on the bed, falling asleep.
When he wakes up, Harry breaks into room 773, which he finds curiously clean. There is no sign of any disturbance or violence in the room, but when he flushes the toilet, it fills with blood and floods the room.
Harry goes to the office building, where he tries to speak the director. He quickly learns, however, that the director has allegedly been killed in a car crash. He sees that Ann is still alive and has inherited the director's fortune. Putting the pieces together, Harry realizes that it was Ann and Mark who killed the director, not the other way around. Martin Stett spots Harry amidst the hubbub surrounding the death of the director.
Back at his apartment, Harry receives a call from Martin Stett, who tells him that he knows that Harry knows the details of the director's murder and that they will be watching him. Over the phone, Martin plays a recorded tape of Harry playing the saxophone in his apartment. Consumed with paranoia, Harry rips up his apartment in search of the surveillance devices planted in his apartment. After ripping up floorboards and tearing down wallpaper, Harry sits in the middle of his decimated apartment and plays the saxophone.
