In times past when rocks were used in the bathroom, if you did not prepare them before Shabbos and they are now muktza, you can still take some as we are lenient for ‘kavod habriyos’ [respect for human beings].

Chapter 312 Section 2

If you had prepared rocks from before Shabbos and the rain made them stuck in the mud, you are permitted to take them out of the ground.

Chapter 312 Section 3

If you had prepared rocks from before Shabbos and grass grew on them, you are permitted to use them.