UBR4 is a 574 kDa E3 ligase (E3) of the N-degron pathway with roles in neurodevelopment, age-associated muscular atrophy and cancer. The catalytic module that carries out ubiquitin (Ub) transfer remains unknown. Here we identify and characterize a distinct E3 module within human UBR4 consisting of a 'hemiRING' zinc finger, a helical-rich UBR zinc-finger interacting (UZI) subdomain, and an N-terminal region that can serve as an affinity factor for the E2 conjugating enzyme (E2). The structure of an E2-E3 complex provides atomic-level insight into the specificity determinants of the hemiRING toward the cognate E2s UBE2A/UBE2B. Via an allosteric mechanism, the UZI subdomain modestly activates the Ub-loaded E2 (E2∼Ub). We propose attenuated activation is complemented by the intrinsically high lysine reactivity of UBE2A, and their cooperation imparts a reactivity profile important for substrate specificity and optimal degradation kinetics. These findings reveal the mechanistic underpinnings of a neuronal N-degron E3, its specific recruitment of UBE2A, and highlight the underappreciated architectural diversity of cross-brace domains with Ub E3 activity.