The protein kinase TAK1 is activated by binding to Lys63 (K63)-linked ubiquitin chains through its subunit TAB2. Here we analyze crystal structures of the TAB2 NZF domain bound to Lys63-linked di- and triubiquitin, revealing that TAB2 binds adjacent ubiquitin moieties via two distinct binding sites. The conformational constraints imposed by TAB2 on a Lys63 dimer cannot be adopted by linear chains, explaining why TAK1 cannot be activated by linear ubiquitination events.