how to find the inverse response of a system

For the system described by the differential equation below find its inverse zero-state unit step response $$\dfrac{d^2y(t)}{dt^2}-2\dfrac{dy(t)}{dt}-8y(t)=\dfrac{d^2x(t)}{dt^2}-2\dfrac{dx(t)}{dt}-3x(t)$$

so that the cascade system of its inverse system and this system is the identity system.

I am not sure how to tackle this type of problem. My first thought is to switch $$x$$ and $$y$$ and find the response of that system using laplace transform, which I presume would be the inverse system. Is this reasoning correct? if not, how does one tackle this problem? I'm not asking for a solution (although one is welcome), but for a general approach to solve problems like this. Thanks!

A general approach would be to take the Laplace Transform of the equation and put it in form of a transfer function:

$$H(s) = \frac{Y(s)}{X(s)}$$

And then invert that and solve for the unit step response from $$1/H(s)$$

Note that the inverse Laplace transform of $$1/H(s)$$ would give the impulse response, since that implies an impulse is at the input to the system (the Laplace Transform of an impulse is 1 so the output is the product of 1 with $$1/H(s)$$ which is simply $$1/H(s)$$). To get the step response, the s-domain equation is first multiplied by the Laplace Transform of the unit step response prior to solving for the inverse Laplace Transform (I will leave it for the OP to determine that as this is likely a HW problem).

Dan's answer -- to compute $$H(s)$$ as normal, and then compute $$1/H(s)$$ -- is equivalent to your suggestion of swapping the $$x$$ and $$y$$ (or doing it in one step by solving for $$H_{yx}(s) = \frac{X(s)}{Y(s)}$$).

In the Laplace domain it's justified by noting that $$\frac{H(s)}{H(s)} = H(s)H_{yx}(s) = 1$$
In theory this means that a system followed by its inverse system has an output equal to its input. In practice you need to make sure that $$H_{xy}$$ is realizable, and you have to recognize that if you're actually chaining real systems, then your $$H(s)$$ is just a model that may not capture all of your real-world system's real-world dynamics. So the actual end-to-end equation is something like $$\hat X(s) = \left(H(s) X(s) + N(s)\right) \hat H_{yx}(s)$$ where $$\hat X$$ denotes that it's just an estimate, $$N(s)$$ is whatever noise is injected in your measurement of $$Y$$ and your implementation of $$\hat H_{yx}$$, and $$\hat H_{yx}$$ emphasizes that it's a guess at your system model, not the real thing.

It's also useful to note that doing this in the Laplace domain immediately lets you know whether what you're trying is practical. If $$H(s) = 1 / (\tau_0 s + 1)$$, for instance, you'd immediately see that $$1 / H(s)$$ is, strictly, unrealizable (because you can't have a true naked differentiator).

However, as long as noise is negligible you could approach it arbitrarily closely by setting $$\hat H_{xy}(s) = \frac{\tau_0 s + 1}{\tau_1 s + 1}$$, which basically says you're approximating a derivative as $$\frac{s}{\tau_1 s + 1}$$ -- then you could use my end-end equation above, along with your knowledge of the nature of $$X(s)$$ and an estimate of the noise you're going to inject at $$N(s)$$ to estimate your end-end system fidelity.