I only seek a relationship $\text{ifft}(\text{operation}(\text{fft}(x))) \equiv |x|$.
So, let's $\DeclareMathOperator{\DFT}{DFT}\DeclareMathOperator{\IDFT}{IDFT}$ be a bit more clean here:
The FFT (IFFT) is just a fast-to-compute implementation of the DFT, so let's use the mathematical concept instead of the method to calculate its result. You demand an $\text{operation}$ for which
$$\IDFT\left(\text{operation}\left(\DFT(x)\right)\right) = |x|.$$
Since $\DFT(x)$ is an $N$-long vector of complex numbers, $\text{operation}$ maps $\mathbb C^N \mapsto C^N$, so let's give it a letter: $g(y): \mathbb C^N \mapsto \mathbb C^N$.
Let's rewrite your equation in this notation (nothing changed from your original statement):
$$\IDFT\left(g\left(\DFT(x)\right)\right) = |x|.$$
Now, the $\IDFT$ is bijective, even invertible (by, surprise, the $\DFT$), so this is equivalent to
$$g\left(\DFT(x)\right) = \DFT\left(\lvert x\rvert\right).$$
Now, there's infinitely many $x$ with the same $|x|$, and even for real-only $x$, it's still $N^2$ different $x$. (You get $N^2$ permutations of signs for all the elements of $x$).
Let's substitute $y=\DFT(x)$:
$$g\left(y\right) = \DFT\left(\lvert \IDFT(y)\rvert\right)$$
What is written literally spells out: The operation you're looking for is first IDFT'ing your input, then calculating the absolute, then DFT'ing the result (that suggests there's no shortcut).
Let's rule out any candidates for simpler representations:
- $|x|$ is not linear ($|-2|+|1|\ne |-2+1|$), so $g$ can't be linear either;
- it hence can't be an integral, or a polynomial
- $g$ is not differentiable ($|x|$ isn't, and the DFT can't do anything about the points where it isn't)
- therefore, $g$ can't also be a differentiable function or linear combination of such
Not quite sure for what other kinds of relationships you're looking for, but if a relationship isn't linear, not even differentiable, and the "native" way is IDFT->abs->DFT, then in my terms, there's no "easier" way to express the relationship than through actual computation of the abs.
This comes very natural to signal processing folks – we all know that nonlinear operations like abs (but also, squaring, taking the signal as exponent and such) lead to distortions that mean that suddenly all components in the signal mix with each other, and you can't find an easy expression for the result of a Fourier transform anymore. It's really that simple: While the FT of a sum of many tones is simply the combination of the FT of all the tones, as soon as you introduce any nonlinearity, the whole "summability" breaks down, and you get intermodulation problems.
This is especially easy to see for the abs of a sine: To calculate the abs, you just need to multiply the sine wave with a -1,+1 square wave that has the same frequency and is shifted just right so that the negative half-periods are the same (in other words, it's the signum function of the sine!).
That means your spectrum is the convolution of the sine's spectrum, which is one (complex) or two (real sine) diracs, convolved with an infinite series of diracs (from the Fourier series of the square wave); the result is an infinite series of diracs....
Now, when you have more than one sine, you get one of two cases:
- both tones' frequencies are rationally related: the "absolutifying" "not-really-square"-wave for the sum of the two sines (i.e. the signum of the sum) is now a repeating, but within one periode irregular wave. Its spectrum is hence a line spectrum with infinitely many, interestingly-spaced diracs.
- both tones' frequencies are not rationally related: oh well; your signum function isn't periodic. So, you convolve the sum of the sine spectra with the spectrum of an aperiodic function - which is continuous. So, the spectrum becomes continuous! This is really really awkward.
So, the absolute value can make formerly easy spectra arbitrarily complex. Sorry, there's no easier representation for it!