Dan's answer -- to compute $H(s)$ as normal, and then compute $1/H(s)$ -- is equivalent to your suggestion of swapping the $x$ and $y$ (or doing it in one step by solving for $H_{yx}(s) = \frac{X(s)}{Y(s)}$).
In the Laplace domain it's justified by noting that
$$\frac{H(s)}{H(s)} = H(s)H_{yx}(s) = 1$$
In theory this means that a system followed by its inverse system has an output equal to its input. In practice you need to make sure that $H_{xy}$ is realizable, and you have to recognize that if you're actually chaining real systems, then your $H(s)$ is just a model that may not capture all of your real-world system's real-world dynamics. So the actual end-to-end equation is something like
$$\hat X(s) = \left(H(s) X(s) + N(s)\right) \hat H_{yx}(s)$$
where $\hat X$ denotes that it's just an estimate, $N(s)$ is whatever noise is injected in your measurement of $Y$ and your implementation of $\hat H_{yx}$, and $\hat H_{yx}$ emphasizes that it's a guess at your system model, not the real thing.
It's also useful to note that doing this in the Laplace domain immediately lets you know whether what you're trying is practical. If $H(s) = 1 / (\tau_0 s + 1)$, for instance, you'd immediately see that $1 / H(s)$ is, strictly, unrealizable (because you can't have a true naked differentiator).
However, as long as noise is negligible you could approach it arbitrarily closely by setting $\hat H_{xy}(s) = \frac{\tau_0 s + 1}{\tau_1 s + 1}$, which basically says you're approximating a derivative as $\frac{s}{\tau_1 s + 1}$ -- then you could use my end-end equation above, along with your knowledge of the nature of $X(s)$ and an estimate of the noise you're going to inject at $N(s)$ to estimate your end-end system fidelity.