This depends on your modulation choice and approach to carrier recovery. But consider this for a specific case of a raised cosine pulse shaped signal:
Observe (using the eye diagram would be simple) the significant symbol to symbol jitter everywhere except in close proximity to the symbol decisions — for this case I would opt to use 1 sample per symbol (the decisions) to estimate symbol to symbol phase since those samples have the least jitter (this would be the samples after all equalization, timing recovery and matched filtering is complete, just prior to decision where the complex phase can still be measured between two successive symbols).
This approach works well if you are using the Gardner TED for symbol clock recovery since it can function well with relatively large carrier offsets.
You can evaluate your specific waveform with the same considerations to determine what approach would have the minimum estimation error (and not the error on any given sample but the rms error within your carrier tracking loop bandwidth).
Here is an example shown below for a raised-cosine filtered waveform. Typically the filtering is split between the transmitter and receiver with Root-Raised Cosine (RRC) filters such that sufficient pulse shaping for reducing out of band emissions is provided while allowing for a matched filter for optimum SNR (against white noise conditions) in the receiver. In the plots below you see the signal as received (if noise-free) and then after it is passed through the RRC matched filter. Notice the wider variation in the zero crossings after the RRC filter in the plot on the right compared to the plot on the left. It is the zero crossing locations that are used, on average, to determine the data clock. This is why, without additional processing to minimize the zero crossing variation (which can be done), it is better to use the the waveform on the left for timing recover. However notice the sharp points at the correct sampling locations for symbol decision on the plot on the right compared to the one on the right-- these are the best samples (at one sample per symbol using the best derived sample from timing recovery) to measure phase rotation for carrier recovery given the Gardner TED can operate with significant carrier offset.