You can reduce the amount of gear you have with you at the waterfront for short sessions. This will enable you to be lighter, more reactive and more discrete at the waterfront.
For your quick fishing trips, using two rods rather than the usual four will save you some weight. Regarding rod rests, opt for rod pod spikes, which are less bulky for this type of session. A bag such as the Caperlan Carryall would be perfect to carry your bait and assembly accessories, and stores all the necessary gear for your fishing trip. All you would have left to carry is the keepnet and landing mat.
For additional comfort, you can take a level chair with you, so you can wait patiently for the ideal conditions between two starts.