Yes. This is one of the things that makes collections more than just a list. Within any collection, you can add a text annotation to each item - an artist, album, or track - to give it context, a review, a memory, or whatever you want to say about it.
You can also add a custom thumbnail image and a hero image to the collection itself, giving it a distinct visual identity.
This makes collections useful not just for personal organization but as a curatorial format - a way to share music with context and intention, similar to an editorial playlist or a written recommendation.