I'm not sure whether this is a purely academic exercise or supposed to do something useful in the real world. Assuming it's the later, there are a few additional points to consider

 1. Typically when doing a cross over design, the phase contribution from the cross over itself is a minor factor. There are the transfer functions of the drivers themselves, the contributions by the enclosure (specifically the edge diffractions) and than of course, the room itself (for a variety of placements). 
 2. You will generally try to optimize the magnitude spectrum over the listening area with a reasonable room loading assumption. Trying to equalize the group delay itself is problematic. The room does a LOT of damage to the monaural phase response and the the group delay of loudspeaker in a room is poorly defined and hard to measure. Put it differently: the group delay at any one frequency will change rapidly with only very small movements of either speaker or microphone. 
 3. Instead of using Linkwitz Riley you can also use odd order Butterworth filters. A 5th order Butterworth will give you high frequency selectivity and less peak group delay as compared to a 4th order LR 
 4. Both odd order Butterworth and even order Linkwitz Riley crossovers add up to an causal all pass filter. In order to equalize the group delay you would have to apply the inverse of that, which is an anti-causal allpass filter. The only practical way to do this, is to add a good junk of overall delay, which may result in latency issues.
 5. In a real world cross over design all these different factors need to be taken into account. It's actually very common to NOT end up with a standard crossover but with different high/low pass frequencies and orders for both drivers.
 6. Correcting non-incident drivers is difficult. The interference pattern varies a lot with listener/microphone position. Your best shot is good overall system design: choose dimension, driver placement and cross over frequencies so that the worst interference frequencies are outside if your cross over range.