The Laplace Transform is more representative of real systems that have a starting point, which is why the integral starts at 0, and also why the unit step function is generally talked about alongside the Laplace Transform. With the Laplace Transform, we can examine the transient and steady-state behavior of a system.
Using $e^{st}$ instead of $e^{iwt}$ allows us to examine different aspects of a physical system. The variable $s$ is complex, and if the real part was set to 0, it would reduce to a truncated Fourier Transform. The real part of $s$ is related to the amount of damping in the system. Also, with the Laplace Transform, a system's stability can be considered.
In short, Laplace is used to consider damping, stability, transient and steady-state behavior of a physical system (represented by a differential equation).