How do you tell if your code needs to be refactored?
If it looks like code, it needs to be refactored. You want code so clean and simple that anyone could understand it.
Successful refactoring and eloquent DSLs lead to the same place: your code becomes semantically valid. As your code achieves semantic validity, it becomes both easier to maintain and more powerful. People often emphasize the connection between maintenance and successful refactoring, but it's also very important to realize that precise expressiveness and power go hand in hand. Rails provides a very precise, expressive DSL for building web sites, and in doing so, gives you so much power that where people used to throw shell scripts at problems casually, because shell scripts are so easy to make, today people in the Rails community throw Web apps at problems casually.
Obviously at some level code will always be something only programmers can understand. However, doing high-level things in a low-level way is like going to Spain and speaking Chinese. The language you choose (or define) should match the context you're in. It doesn't even have to apply to totally green newbies - if you just write code that less-skilled programmers can understand, you end up with better code. You also find yourself using it. At that point, you'll find that some bits of your semantic mapping work better than others. When you make your code semantic, you find out how much you do - or don't - understand your problem domain. If you can just look at it and grok it instantly, you got it.
If not, fix it up!