0
$\begingroup$

I hope this is the right location to post this, as my question isn't really stack-overflow material but involves images. I have hundreds of SEM-Images like this:

SEM-image

and need to track changes in certain features over the position at which the image was captured. This is why I would like to automatically retrieve metadata using something like bash or python for all images and save ti in a csv file or something.

The manufacturer of the SEM implements metadata on the images in the file itself, but I haven't had much success retrieving it. It doesn't show in the EXIF header if I inspect it with gimp or with image-magick's identify image.tiff. I only saw it if I save the image as an .xml file and view it in a texteditor.

The manufacturer ships an .exe program to inspect images, but as I am using Linux and have hundreds of images from which to retrieve the sample position, this isn't a very useful method for me.

enter image description here

Before I start writing a python / bash program that will convert the files to xml and use complex regex stuff to retrieve what I am looking for, I wanted to ask if anyone could imagine a simpler approach. My knowledge on image metadata is very limited.

Thanks in advance :)

$\endgroup$
  • $\begingroup$ When you save the image as "xml", does this information show up formatted as XML or did you just make out some of these strings knowing what they are supposed to look like (?) $\endgroup$ – A_A Apr 9 at 10:45
  • $\begingroup$ @A_A in fact, it still displays everything as random characters. the only program showing me the info is Windows notepad, where it is dispayed as a random string enclosed in something that looks like html. It is the same info displayed aside the hex-bytes in the hex-editor $\endgroup$ – cripcate Apr 9 at 11:25
  • $\begingroup$ Yeah, it makes sense. Try something like the exiftool or exiv2, or if you want full control, try the exiftags module from PIL. However, some of these tags might be custom and come up under a generic EXIF tag. Can you post a sample image? $\endgroup$ – A_A Apr 9 at 11:46
0
$\begingroup$

Open a few files in a hex editor and check, if the header containing the information is always the same size, i.e. if the image data always begins at the same adress. If this is so, you can just retrieve the header with python ("normal" open, not binary). Then, you have the complete header as a string and can create any data structure you want, using any xml package you want.

$\endgroup$
  • $\begingroup$ I did what you suggested, but I have almost no understanding of HEX at all. I can see the information I want on the side of the HEXcode at the bottom, it is always starting at 0044:02F0, so I guess it comes after the image data? How do I get this into python now, if I openthe file in normal readmode i get UnicodeDecodeError $\endgroup$ – cripcate Apr 9 at 11:14
  • $\begingroup$ Then try to open it in binary mode and decode it to a string like this: data.decode(encoding). Try different encodings, starting with "utf-8". $\endgroup$ – Max Apr 9 at 11:37
  • $\begingroup$ This works, thanks. It gives me the random characters I see in the hexeditor, with the metadata enclosed in html stuff at the end. I think I can use regex from here to get what I want. I'll upload the script in the end if I get it to work. Maybe someone's interested.... $\endgroup$ – cripcate Apr 9 at 12:24
0
$\begingroup$

Based on @Max answer I was able to write a little python script to extract my metadata. It's a bit hacky and some stuff is hardcoded where it shouldn't be but it does the job:

import argparse
import re

from bs4 import BeautifulSoup
import pandas as pd

# %% Argparse setup
parser = argparse.ArgumentParser(description='Extract metadata from PhenomPRO SEM-images')

parser.add_argument('files', metavar='F', nargs='+',
                    help='Files to extract the metadata from.')
parser.add_argument('output', metavar='O',
                    help='Filename to save the output data to.')

# %%


def get_info(filename):
    # Read File and get relevant info
    with open(filename, 'rb') as f:
        lines = f.readlines()

    lines_dec = []

    for line in lines:
        try:
            lines_dec.append(line.decode('utf_8'))
        except UnicodeDecodeError:
            continue

    file_utf = '\n'.join(lines_dec)
    info = re.findall(r'4\.4\.4\..*$', file_utf)[0]

    return info[36:-1]


def extract_metadata(info_string):
    # Extract metadata
    i = BeautifulSoup(info_string, features='xml')
    img = i.FeiImage

    data = {'time': img.time.contents,
            'scale_unit': img.pixelWidth.attrs['unit'],
            'scale_x': img.pixelWidth.contents[0],
            'scale_y': img.pixelHeight.contents[0],
            'sample_position_x': img.samplePosition.x.contents[0],
            'sample_position_y': img.samplePosition.y.contents[0],
            'contrast': img.appliedContrast.contents[0],
            'brightness': img.appliedBrightness.contents[0],
            'dwell_time': img.acquisition.dwellTime.contents[0],
            'dwell_time_unit': img.acquisition.dwellTime.attrs['unit'],
            'spot_size': img.acquisition.spotSize.contents[0],
            'rotation_deg': img.acquisition.rotation.contents[0],
            'source_tilt_x': img.acquisition.sourceTilt.x.contents[0],
            'source_tilt_y': img.acquisition.sourceTilt.y.contents[0],
            'stigmator_x': img.acquisition.stigmator.x.contents[0],
            'stigmator_y': img.acquisition.stigmator.y.contents[0],
            'voltage_kV': img.acquisition.highVoltage.contents[0],
            'emission_uA': img.acquisition.emissionCurrent.contents[0],
            'filament_power_W': img.acquisition.filamentPower.contents[0],
            'workig_distance': img.workingDistance.contents[0],
            'working_distance_unit': img.workingDistance.attrs['unit']
            }

    return data


if __name__ == "__main__":
    args = parser.parse_args()

    files = args.files
    out = args.output

    for i, file in enumerate(files):
        file_info = get_info(file)
        file_metadata = extract_metadata(file_info)

        if i == 0:
            meta = pd.DataFrame(file_metadata)
        else:
            meta = meta.append(file_metadata, ignore_index=True)

    meta.to_csv(out)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.