I got bored of picking colours around mipmap 10 (16x16), so I didn't bother going lower. :)
I guess unity doesn't like that.
I'll add a couple more.
Here's a new debug texture. It's added:
- olive = 8x8
- purple - 4x4
- dark grey = 2x2
- pale sky blue = 1x1
Hopefully that works better. :)
Here's an example (model from MakeHuman):

The tshirt texture is normally 1024x1024. Changing it to the debug texture shows that at this distance most of it is cyan, which is 512x512, so if the model is never closer than this to the camera, 512x512 would be enough.
(Screen res also affects this, if I ran the demo in higher res then the shirt would need higher res textures for the same distance)
Or course in VR it's harder to tell the minimum distance to the camera (lean in close enough and even 8192x8192 is insufficient).
A combination of detail textures (as mentioned by others above) and procedural generation can make up for texture resolution.