1
$\begingroup$

A dataset of mri and ct scan images of patients has been prepared. There is a feature /damaged area/ in the mri image that is easily visible. But the injury of this area is not visible in the CT scan image. We want to train the model using CT scan data of injured brains. Is there a way to specify the damaged areas in the ct scan images using the mri images so that the model focuses more on that area? (should be noted that these injuries appear in different areas of the brain.)

CT enter image description here

MRI enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Hi and welcome to DSP.SE. This question doesnt seem to be in the scope of signal processing. I would assume that it better fits in to the machine learning area with the scope on physiological data. $\endgroup$ Commented Jan 17, 2023 at 8:39
  • $\begingroup$ please show us a picture. -- are you asking how to annotate a 3D/volumetric dataset for semantic segmentation? $\endgroup$ Commented Jan 17, 2023 at 16:51
  • $\begingroup$ Christoph Rackwitz Hi, Look at the images (dataset included 3D dicom images) First, the images must be registered and aligned. Then, the white area of the MRI image should be extracted as a numpy array. $\endgroup$
    – Erfan Pot
    Commented Jan 17, 2023 at 21:05

1 Answer 1

1
$\begingroup$

Is there a way to specify the damaged areas in the ct scan images using the mri images so that the model focuses more on that area?

Yes. There is a way to do that. But it is doubtful that the model's performance would be improved by this.

You hinted at image registration and alignment at the comments.In fact, you would have to cross register both datasets as volumes in order to bring both datasets on the same stereotactic space. This will allow you to "address" both volumes using the same coordinates, so that coordinate $(x,y,z)$, within a particular brain area on one image corresponds to the same brain area within the other image.

But, because of the way MRI images are acquired, you would have to perform an alignment step of the MRI volume itself to avoid slight distortions caused by involuntary head movements as well as the pulsing of the heart. This is usually done by selecting a frame from the 3D volume and then aligning the rest of the frames with respect to that reference frame. Whether you absolutely have to take this step depends on the data acquisition protocol (e.g. shorter slices with fast acquisition times might not be as affected).

As far as alignment of the MRI image is concerned, you can use something like SPM which has this particular functionality out of the box (see Chapter 2 - Realign).

SPM can also definitely handle co-registration of images but mostly for MRI (See Chapter 4 - Coregister). You might get a good enough result if you exclude the cranial area and the skin area from the CT and MRI respectively and focus on co-registering the brain tissue but because of the contrast differences, this might still be too coarse. The next "easier" thing to do would be to cross register manually specified points across the two images, specified by a specialist. And finally, you could write your own cost function to drive a typical cross registration process.

After the two volumes have been cross registered, you then need to perform segmentation. SPM will perform some typical segmentation on the MRI image for you. For example segment various types of tissues (gray matter, white matter, cerebrospinal fluid, other). See Chapter 5 - Segment.

But, in any case, once you have your two volumes co-registered, you can still process the MRI images separately with any segmentation process and transfer the detected pixels across to the other image.

Now, I don't know if these two images are from the same person or what you are exactly trying to do, but because there are fundamental differences between the way the "signal" is formed in CT and MRI, I feel I should also mention here the following:

Just because you might be able to "see something" on one image (e.g. the MRI), does not mean that it will be visible on the other (or at least at the same time). A trivial example is the skull. The skull is very easy to see on the CT image, it is that stark white thick oval fence around the brain that denotes high X-Ray absorption. But, the same "fence" is practically invisible in the MRI image. Why? Because the skull contains very very very few hydrogen atoms that drive the "signal" that MRI measures.

Similarly, the fact that a feature is clearly visible in the MRI image, does not mean that it will be showing up on the CT too. Or, to put it in a different way, aspects of that feature MIGHT become visible on the CT but at a much later stage in the process than the MRI, especially on the brain.

Let alone that there is no additional gain from recommending someone to get a CT versus an MRI because CT uses ionising radiation.

Therefore, the whole operation of getting a model to "focus on the CT image" should be approached with care.

Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.