We present here a structure-aided design of inhibitors targeting the active site as well as exosites of glutamate carboxypeptidase II (GCPII), a prostate cancer marker, preparing potent and selective inhibitors that are more than 1000-fold more active toward, GCPII than its closest human homologue, glutamate carboxypeptidase III (GCPIII). Additionally, we demonstrate that the prepared inhibitor conjugate can be used for sensitive and selective imaging of GCPII in mammalian cells.