Understanding the social structures between objects, organizing, and selecting them accordingly, is fundamental to social cognition. We report an example that demonstrates the object association learned from social interactions could impact visual attention. Particularly, when two hands approach each other to perform a handshake, they tend to be attended to as a unit because of the cooperative relationship exhibited in the action: even a cue presented on a non-target hand may facilitate a response to the targets that appear on the non-cued hand (Experiment 1), indicating that attentional shift between two hands was facilitated; furthermore, the response to a target on one hand is significantly impaired by a distractor on the other hand (Experiment 2), implying that it is difficult to selectively confine attention to a single hand. These effects were dependent on the existence of the hands when cue and target appeared (Experiment 3); neither perceptual familiarity, or physical fit can explain all the attention effects (Experiment 4). These results have bearings on the perceptual root of social cognition.