Taking the inverse DFT is like evaluating the Fourier Series produced by the DFT coefficients at integer domain points. This is very efficiently when using the iFFT.
Each bin is a complex value which can be represented by a real part and imaginary part, or as a magnitude and phase value.
$$ X[k] = a_k + i b_k = r_k e^{i\theta} $$
The latter form comes from Eulers formulas (See https://www.dsprelated.com/showarticle/754.php)
The right half, or upper half, counterpart is the complex conjugate for real valued signals.
$$ X[-k] = X[N-k] = a_k - i b_k = r_k e^{-i\theta} = X^*[k] $$
When you are "reading the bin values" in code, you are actually doing this in math:
$$ a_k = (X[k] + X[N-k])/2 $$
$$ b_k = (X[k] - X[N-k])/2i $$
For completeness:
$$ \|X[k]\| = \sqrt{ a_k^2 + b_k^2} = |r| $$
$$ \angle( X[k] ) = \arg( X[k] ) = \theta $$
For bin centered pure tones, these values directly reflect the signal values.
When you zero out the top half and take the inverse DFT you get a complex signal back where each Fourier term(s) has been changed, and compensate by double the coefficients.
$$ x[n] = \dots + 2 a_k \cos \left( \frac{2\pi}{N}kn \right)+ i 2 b_k \cos \left( \frac{2\pi}{N}kn \right) \dots $$
Here the Sine term is being added in the imaginary axis, so if you just strip the results, keeping the real part, all you've done is throw away the phase shift.
Here is a comparison, stripped:
$$
\begin{aligned}
\Re(x[n]) &= \dots + 2 a_k \cos \left( \frac{2\pi}{N}kn \right) + \dots \\
\end{aligned}
$$
Versus using the upper half:
$$
\begin{aligned}
x[n] &= \dots + 2 a_k \cos \left( \frac{2\pi}{N}kn \right) + 2 b_k \cos \left( \frac{2\pi}{N}kn \right) + \dots \\
&= \dots + 2 r_k \cos \left( \frac{2\pi}{N}kn + \theta_k \right) + \dots
\end{aligned}
$$