So in the end, I actually choose two different fabrics! The first on I choose is fleece! I choose it because it's really soft, and fuzzy, also is light weight and warm. The most amazing of its properties is its ability to insulate the body even when it is completely wet.
The second fabric I choose is nylon! I picked it kinda last minute, but I'm thinking of putting a little patch of it on each side, not covering your fingers though! I choose it because it is water proof so it would go well; warm, comfy, and water proof!