Test Driven Development provides a very structural way for agile coding. One of the steps in TDD is refactor, and this is when we improve our design, but to what extent can it still be deemed as being agile? To remind myself, I would like to write down the *only* two factors that I think are agile to drive this design step:
1) no duplicated code - if the new code is some sort of duplication of an existing logic, or it is destined to be reused by other developers in the team, I would improve the design to avoid (potential) duplication.
2) no unreadable code - if the new code is hard to read, or it makes the existing file(s) difficult to read (for example, the new code might make it harder to navigate), I would improve the design to achieve readability.
That's it. If I find myself designing due to any factors other than these two, I would feel myself being less agile. It happens a lot, sometimes due to the limitations of the technical platform, sometimes due to the "process", but I would keep reminding myself (and maybe the team) that it's not what I preferred.