All implementation aspects aside, the constellation you propose performs worse than QPSK in an additve white gaussian noise (AWGN) channel.
I claim this based on simulations that I have run with Matlab calculating the symbol error rate (SER) as a function of signal-to-noise ratio (SNR). Here is the result:
As you can see, for a given SNR, the proposed constellation has an higher SER than a QPSK constellation. Additionally, I expect the proposed scheme to perform even worse in terms of bit error rate (BER) as -- in contrast to QPSK -- Gray labelling is not possible. Some details concerning the simulation:
- 1 million bits per SNR value
- maximum likelihood decision
- additive, Gaussian distributed noise
- memoryless channel (no inter-symbol interference)
It might well be that I made a mistake writing the simulation script, so I uploaded it in case you would like to check my implementation.
This might not be the mathematically rigorous explanation that you have been hoping for. To answer your question theoretically, one would have to calculate the SER analytically. For that, one should first find the decision thresholds, which is the main problem (for me, at least). It's far from obvious, what the optimal decision thresholds would be. They are not straight lines as with QAM constellations but probably a kind of zero-centered disc with three "rays" emerging from its center. Calculation of the SER should probably involve a transformation of the problem to polar coordinates.
With that kind of irregular constellation it also doesn't help much to compare the minimum distance, i.e. the length of a straight line between two given constellation points. It's true, that for a given mean signal energy, the outer points of the proposed contellation have greater distance to the inner point than any two points of the QPSK constellation. However, the inner point of the proposed contellation is surrounded by three other points which decreases its decision area and consequently increases the symbol error probability for that point. It's not obvious what is better. According to the simulation, QPSK is better.
The areas of the maximum likelihood decision can also be simulated. In the following image, each color corresponds to a distinct, decided symbol. The input to the decision device is a very noisy (0 dB SNR) signal with the proposed constellation.
We can clearly see how the inner point is "squeezed" between the outer points. In my opinion, that is the reason why the proposed constellation performs worse than QPSK. Now that I see this plot, I find the SER calculation not so difficult anymore. With a little time it could be done.