Conjugated polymers achieve redox activity in electrochemical devices by combining redox-active, electronically conducting backbones with ion-transporting side chains that can be tuned for different electrolytes. In aqueous electrolytes, redox activity can be accomplished by attaching hydrophilic side chains to the polymer backbone, which enables ionic transport and allows volumetric charging of polymer electrodes. While this approach has been beneficial for achieving fast electrochemical charging in aqueous solutions, little is known about the relationship between water uptake by the polymers during electrochemical charging and the stability and redox potentials of the electrodes, particularly for electron-transporting conjugated polymers. We find that excessive water uptake during the electrochemical charging of polymer electrodes harms the reversibility of electrochemical processes and results in irreversible swelling of the polymer. We show that small changes of the side chain composition can significantly increase the reversibility of the redox behavior of the materials in aqueous electrolytes, improving the capacity of the polymer by more than one order of magnitude. Finally, we show that tuning the local environment of the redox-active polymer by attaching hydrophilic side chains can help to reach high fractions of the theoretical capacity for single-phase electrodes in aqueous electrolytes. Our work shows the importance of chemical design strategies for achieving high electrochemical stability for conjugated polymers in aqueous electrolytes.