Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update loading of .topostats now just data is returned #1086

Open
Tracked by #517
ns-rse opened this issue Feb 5, 2025 · 0 comments
Open
Tracked by #517

Update loading of .topostats now just data is returned #1086

ns-rse opened this issue Feb 5, 2025 · 0 comments
Assignees
Labels
refactor Refactoring of code

Comments

@ns-rse
Copy link
Collaborator

ns-rse commented Feb 5, 2025

In light of changing what [AFMReader returns](AFM-SPM/AFMReader#112** so that _just** the data is
returned we will need to update TopoStats to handle this.

IMPORTANT This will require co-ordination with AFMReader releases with clear advice to update to newer versions of
AFMReader and pip install --no-cache-dir --upgrade topostats.

Points that need changing....

io.py

LoadScans.load_topostats()

Line 679

            image, px_to_nm_scaling, data = topostats.load_topostats(self.img_path)

...will be...

            data = topostats.load_topostats(self.img_path)

We then have a section that was introduced to handle whether we wanted different components starting on Line
683

        try:
            # We want everything if performing any step beyond filtering (or explicitly ask for None/"all")
            if extract in [None, "all", "grains", "grainstats"]:
                return (image, px_to_nm_scaling, data)
            # Otherwise we want the raw/image_original
            if extract == "filter":
                return (image, px_to_nm_scaling, None)
            return (data[map_stage_to_image[extract]], px_to_nm_scaling, None)
        except KeyError as ke:
            raise KeyError(f"Can not extract array of type '{extract}' from .topostats objects.") from ke

...as we will only ever be returning data this can be replaced with

        return data

LoadScans.get_data()

There are a logical checks here that may need changing to account for only receiving data. This section extracts
self.image selt.pixel_to_nm_scaling and optionally data depending on the type of processing being undertaken. This
is stored in the self.extract and represents all or the stage (e.g. filter, grain, grainstats).

A key point which I missed when working on this was that the else: after the try: ... except: ... as I forgot that
the else: will run if no exception is raised (i.e. try: is successful), and this step builds a dictionary of the
images/data.

Line 782

                try:
                    if suffix == ".topostats" and self.extract in (None, "all", "grains", "grainstats"):
                        self.image, self.pixel_to_nm_scaling, data = self.load_topostats()
                    elif suffix == ".topostats" and self.extract not in (None, "all"):
                        self.image, self.pixel_to_nm_scaling, _ = self.load_topostats(self.extract)
                    else:
                        self.image, self.pixel_to_nm_scaling = suffix_to_loader[suffix]()
                except Exception as e:
                    if "Channel" in str(e) and "not found" in str(e):
                        LOGGER.warning(e)  # log the specific error message
                        LOGGER.warning(f"[{self.filename}] Channel {self.channel} not found, skipping image.")
                    else:
                        raise
                else:
                    if suffix == ".asd":
                        for index, frame in enumerate(self.image):
                            self._check_image_size_and_add_to_dict(image=frame, filename=f"{self.filename}_{index}")
                    # If we have extracted the image dictionary (only possible with .topostats files) we add that to the
                    # dictionary
                    elif data is not None:
                        data["img_path"] = img_path.with_suffix("")
                        self.img_dict[self.filename] = self.clean_dict(img_dict=data)
                    # Otherwise check the size and add image to dictionary
                    else:
                        self._check_image_size_and_add_to_dict(image=self.image, filename=self.filename)

Could probably be

                try:
                    if suffix == ".topostats":
                        data = self.load_topostats()
                        self.image = data["image"]
                        self.pixel_to_nm_scaling = data["pixel_to_nm_scaling"]
                    else:
                        self.image, self.pixel_to_nm_scaling = suffix_to_loader[suffix]()
                except Exception as e:
                    if "Channel" in str(e) and "not found" in str(e):
                        LOGGER.warning(e)  # log the specific error message
                        LOGGER.warning(f"[{self.filename}] Channel {self.channel} not found, skipping image.")
                    else:
                        raise
                else:
                    if suffix == ".asd":
                        for index, frame in enumerate(self.image):
                            self._check_image_size_and_add_to_dict(image=frame, filename=f"{self.filename}_{index}")
                    # If we have extracted the image dictionary (only possible with .topostats files) we add that to the
                    # dictionary
                    elif data is not None:
                        data["img_path"] = img_path.with_suffix("")
                        self.img_dict[self.filename] = self.clean_dict(img_dict=data)
                    # Otherwise check the size and add image to dictionary
                    else:
                        self._check_image_size_and_add_to_dict(image=self.image, filename=self.filename)

run_modules.py

The entry points/functions for running topostats {filters|grains} shouldn't need changing I don't think but worth
noting this is where the returned data is used. Something in the back of my mind says they will need changing but I
can't see it right now, perhaps it will be the process_filters() as this will have to extract the original_image from
the returned data as it will no longer be receiving a dictionary with this as the first item. Probably similar for the
process() function too, but I think the process_grains() function will be ok as its expecting to get a dictionary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactor Refactoring of code
Projects
None yet
Development

No branches or pull requests

1 participant