I've used both linen and muslin with good results.
Here's a funny story about me picking out some cloth to make patches from. When I first got into trad muzzleloading, I had a 50 caliber rifle made and I was learning all about its care and feeding. Through much shooting, I determined that it liked patches that were .015" thick the best. So when I started running low on patches, I went to Walmart to look at their linen selection. After the nice lady pointed me to the bolts of cloth I needed to examine, imagine the look on her face when I whipped out a set of digital calipers so I could measure the thickness of her wares. She, of course, asked what I was doing and after I went into a rather lengthy explanation, she just rolled her eyes and walked away. That was well over ten years ago and I'm still cutting patches out of that square yard of cloth.
Darren