Are the frequencies a histogram in the frequency domain?
No.
A histogram is the visualisation of a sequence of discrete counts. For example, a population pyramid can tell you the distribution of ages in a population.
The frequency spectrum represents strength. It doesn't have to be integer and it does not really mean that it represents counts of sinusoids at a specific frequency. The most trivial counter example of this is what is known as the Direct Current (DC) value which is frequency 0 (Zero) Hertz.
Although, you could possibly formulate a view of the frequency spectrum as some form of histogram (counts of elementary parcels of sinusoids), it would be very difficult to fit this view to the average value across the sample (which is what 0 Hz represents).
So, the frequency spectrum is not a histogram, it has nothing to do with a histogram. But what is it?
It is difficult to tell you without going through equations and properties because they represent our understanding of how the world works (for example, linearity and how things can be decomposed to parts and recomposed from parts). But here is an attempt:
This is a string trio:
Notice that the instruments have more or less the same shape but their size changes. Now, without going into equations, take it as a rule of thumb: The bigger the instrument the "lower" (in frequency) is its sound. (And, they can get...big).
When they all play together, they make up one continuous stream of sound that hits one's ears or a microphone.
The accurate analogy here is that the frequency spectrum of the sound recording would tell you how loud was each one of these 'violins' playing.
Joseph Fourier worked out a finer analogy that is very useful in DSP and goes by the name Discrete Fourier Transform (DFT).
He postulated that given any periodic waveform, there is a way to decompose it down to an infinite sum of sinusoids of varying strength and phase.
This is what the DFT does and from the DFT we derive various frequency spectra (namely, amplitude, power, phase).
...since if there are two signals of the same frequency but different amplitude, wouldn't they have a representation on the frequency graph where it's not possible to distinguish between them?
You see how this histogram view of the frequency spectrum doesn't really work now I hope. Two signals of the same frequency contribute to the strength of the same frequency in "the mix" and therefore do not appear as two individual measurements.
I'm even more confused because this example shows 20 millisecond samples being taken from a Fourier spectrum...
But what you show is not a frequency spectrum. What you show is a spectrogram.
Without going into equations: The DFT decomposes a periodic waveform (any waveform) into an infinite sum of sinusoids. Because of the properties of those sinusoids, the DFT can tell you that a sinusoid is present in the mix but it cannot tell you WHEN it was present.
So, if you had a 1 hour recording of just two violins, a big one and a small one but with the small one just performing for 2 seconds, the DFT will still show high frequencies but it will not be able to inform you WHICH 2 SECOND SEGMENT of the 1 hour recording was it that the small violin was louder.
For this reason, the Short Time Fourier Transform (STFT) was developed. What this does is decompose the signal down to small (possibly) overlapping windows and obtain the DFT's of those windows.
This leads to an intermediate representation of the signal where we now can tell (within limits) which frequencies were present at which times.
This is what the last diagram in your question is depicting and this is why it still retains an axis of "Time". In that diagram, Time runs left to right and frequency runs bottom to top.
Hope this helps.
PS: I think that I understand why you don't feel like going through the equations but if you make a start, first with understanding the mathematical notation and then towards reading and understanding what it is that the equations are trying to describe, then I think that you will be greatly benefited in the long term.