Codon usage bias affects the genomes of organisms from all kingdoms of life and results from both background substitution biases and natural selection. Natural selection on codon usage to increase translation accuracy and efficiency has long been known to affect gene sequences. Such selection is stronger on highly, compared with lowly expressed genes, resulting in higher levels of codon bias within genes with higher expression levels. Additionally, selection on translation accuracy affects more strongly codons encoding conserved amino acids, since these will more often affect protein folding and/or function. By applying tests of selection on the gene sequences of the bacterium Escherichia coli, we demonstrate that both highly and lowly expressed genes display signals of selection on codon usage. Such signals are found for both conserved and less conserved amino acid positions, even within the 10% of E. coli genes expressed at the lowest levels. We further demonstrate experimentally that single synonymous codon replacements within a lowly expressed, essential gene can carry substantial effects on bacterial fitness. Combined, our results demonstrate that even within genes expressed at relatively low levels there is substantial selection on codon usage and that single synonymous codon replacements within such genes can have a marked effect on bacterial fitness.