Mud that is on clothes, as long as you have no intent to polish it, you can take it off with a knife or rub it off from the inside. If it is dry and will make a pile of sand, it should not be done.

Chapter 302 Section 8

A sharp knife should not be used on leather to clean it off.

Chapter 302 Section 9

Putting a little water on any material, even if there is no dirt, is not allowed.