It is important to realize that the terms lumped or distributed are not properties of the system itself. These properties are related to the size of the system compared to the wavelength of the voltages and currents passing through it. So a resistor is or isn't a lumped element (even though it is usually meant to be one), depending on the frequency of the applied signals.
An element can be considered as lumped if its size is much smaller than the wavelength of the applied voltages and currents. In this case wave propagation effects may be neglected and a lumped element can be treated as a black box which is completely described by the behavior at its terminals.
EDIT: I forgot to mention the differential equations. Lumped systems are described by ordinary differential equations because due to the small size of the system (compared to the wavelength), the spatial derivatives can be neglected and we only need to consider time derivatives. On the other hand, for distributed systems we need to take wave propagation phenomena into account and we get non-zero spatial as well as time derivatives, which leads to partial differential equations.