How the GemProcessor looks with a new Annotation Processing Api

[LukasDetermann]

Hey, to make annotation processors easier to read and write, i built an api around javax.lang.model. One of the processors I migrated to test how the api feels was your GemProcessor. If you like the api i can create a pull request for tools-gem.

/*
 * Copyright MapStruct Authors.
 *
 * Licensed under the Apache License version 2.0, available at http://www.apache.org/licenses/LICENSE-2.0
 */
package org.mapstruct.tools.gem.processor;

import freemarker.template.Configuration;
import freemarker.template.Template;
import freemarker.template.TemplateException;
import freemarker.template.Version;
import io.determann.shadow.api.ShadowApi;
import io.determann.shadow.api.ShadowProcessor;
import io.determann.shadow.api.TypeKind;
import io.determann.shadow.api.shadow.Package;
import io.determann.shadow.api.shadow.*;
import io.determann.shadow.api.wrapper.AnnotationValueTypeChooser;

import javax.lang.model.type.TypeMirror;
import java.io.IOException;
import java.io.Writer;
import java.lang.Class;
import java.util.*;
import java.util.stream.Collectors;

import static io.determann.shadow.api.ShadowApi.convert;
import static io.determann.shadow.api.TypeKind.*;

/**
 * @author sjaakd
 */
public class GemProcessor extends ShadowProcessor {

    private final EnumMap<TypeKind, Class<?>> primitiveTypeToClass = new EnumMap<>( TypeKind.class );

    {
        primitiveTypeToClass.put( BOOLEAN, Boolean.class );
        primitiveTypeToClass.put( BYTE, Byte.class );
        primitiveTypeToClass.put( CHAR, Character.class );
        primitiveTypeToClass.put( SHORT, Short.class );
        primitiveTypeToClass.put( INT, Integer.class );
        primitiveTypeToClass.put( LONG, Long.class );
        primitiveTypeToClass.put( FLOAT, Float.class );
        primitiveTypeToClass.put( DOUBLE, Double.class );
    }

    private List<GemInfo> gemInfos = new ArrayList<>( 10 );

    @Override
    public void process(ShadowApi shadowApi) {
        
        shadowApi.setDiagnosticHandler( (shadowApi1, diagnosticContext) -> {} );

        Annotation gemDefinitionsAnnotation = shadowApi.getAnnotationOrThrow( "org.mapstruct.tools.gem.GemDefinitions" );

        for ( Declared definingType : shadowApi.getAnnotatedWith( gemDefinitionsAnnotation ).declaredTypes() ) {

            AnnotationUsage gemDefinitionsUsage = definingType.getUsageOfOrThrow( gemDefinitionsAnnotation );

            gemDefinitionsUsage.getValueOrThrow( "value" )
                .asListOfValues()
                .stream()
                .map( AnnotationValueTypeChooser::asAnnotationUsage )
                .forEach( gemDefinitionUsage -> addGemInfo( gemDefinitionUsage, definingType ) );
        }
        postProcessGemInfo();
        write();
    }

    private void addGemInfo(AnnotationUsage gemDefinitionUsage, Declared definingType) {

        // create gem info
        Declared gemDeclaredType = convert( gemDefinitionUsage.getValueOrThrow( "value" )
            .asType() ).toDeclaredOrThrow();
        Package pkg = definingType.getPackage();
        String gemName = gemDeclaredType.getSimpleName();
        String gemFqn = gemDeclaredType.getQualifiedName();
        String gemPackage = pkg.getQualifiedName();

        // collect value info
        List<GemValueInfo> gemValueInfos = gemDeclaredType.getMethods()
            .stream()
            .map( method -> new GemValueInfo( method.getSimpleName(), method.getReturnType() ) )
            .collect( Collectors.toList() );

        GemInfo gemInfo = new GemInfo(
            gemPackage,
            gemName,
            gemFqn,
            gemValueInfos,
            definingType.getElement(),
            gemDeclaredType.getElement()
        );
        gemInfos.add( gemInfo );
    }

    private void postProcessGemInfo() {
        for ( GemInfo gemInfo : gemInfos ) {
            for ( GemValueInfo gemValueInfo : gemInfo.getGemValueInfos() ) {
                if ( gemValueInfo.getShadow().isTypeKind( ARRAY ) ) {

                    Array arrayType = convert( gemValueInfo.getShadow() ).toArrayOrThrow();

                    gemValueInfo.setValueType( getGemValueType( arrayType.getComponentType(), true ) );
                }
                else {
                    gemValueInfo.setValueType( getGemValueType( gemValueInfo.getShadow(), false ) );
                }
            }
        }
    }

    private GemValueType getGemValueType(Shadow<TypeMirror> type, boolean isArray) {

        if ( type.getTypeKind().isPrimitive() ) {

            return new GemValueType( primitiveTypeToClass.get( type.getTypeKind() ), false, isArray );
        }
        if ( type.isTypeKind( ENUM ) ) {

            return new GemValueType( String.class, true, isArray );
        }
        if ( type.getTypeKind().isDeclared() ) {

            // class, other annotation
            String fqn = convert( type ).toDeclaredOrThrow().getQualifiedName();

            if ( Class.class.getName().equals( fqn ) ) {
                return new GemValueType( TypeMirror.class, false, isArray );
            }
            if ( String.class.getName().equals( fqn ) ) {
                return new GemValueType( String.class, false, isArray );
            }
            GemInfo usedGem = gemInfos.stream()
                .filter( g -> fqn.equals( g.getAnnotationFqn() ) )
                .findFirst()
                .orElse( null );
            if ( usedGem != null ) {
                return new GemValueType( usedGem, isArray );
            }

            return new GemValueType( TypeMirror.class, false, isArray );
        }
        throw new IllegalArgumentException( "unrecognized annotation type" );
    }

    private void write( ) {
        for ( GemInfo gemInfo : gemInfos ) {
            try (Writer writer = processingEnv.getFiler()
                .createSourceFile(
                    gemInfo.getGemPackageName() + "." + gemInfo.getGemName(),
                    gemInfo.getOriginatingElements()
                )
                .openWriter()) {
                Configuration cfg = new Configuration( new Version( "2.3.21" ) );
                cfg.setClassForTemplateLoading( GemProcessor.class, "/" );
                cfg.setDefaultEncoding( "UTF-8" );

                Map<String, Object> templateData = new HashMap<>();

                templateData.put( "gemInfo", gemInfo );
                Template template = cfg.getTemplate( "org/mapstruct/tools/gem/processor/Gem.ftl" );
                template.process( templateData, writer );
            }
            catch ( TemplateException | IOException ex ) {
                throw new IllegalStateException( ex );
            }
        }
        // handled all info, clear
        gemInfos.clear();
    }
}

[filiphr]

I've opened Discussions for the tools-gem project and moved this to there.

I can see how this can be something interesting for writing annotation processors. The idea of the tools-gem is also tailored towards annotation processor writers that want to reduce their dependency and have type safe access to the value in their annotations. It is a great idea for making it easier for annotation processor

From an API perspective I do not like the fact that I need to extend from something. I prefer if I am in control of my structure and can use a composition. I need to play around a bit, but from first sight some things look a bit too verbose to me:

• convert( gemDefinitionUsage.getValueOrThrow( "value" ).asType() ).toDeclaredOrThrow()
• toArrayOrThrow
• switch statement has been replaced with if

Apart from that, we are trying to keep the dependencies of this project to a minimum. It is a tool to help other library developers, so having as little dependencies is the best.

[LukasDetermann]

Thanks for the Feedback!

i can completely understand if you dont want to use the new api for this project.

Im fully onboard when it comes to composition over inheritance. The problem with that in this case is, that the jdk api does use inheritance. Keeping this new api close to the jdk one makes it easier to learn for people who already have written annotation processors and it simplifies migrations to the new api. But staying to close the the jdk api makes the new one useless.

that is the exact line where the concept of tools-gem itself could help. When getting the value of a annotation field it is not typesave. Convinces methods could remove the need for the convert() call.
Do you dislike the length of the method names?
Or would you like the ability to cast to Declared?

i should have left the switch a switch. Would have made the comparison clearer.

[filiphr]

that is the exact line where the concept of tools-gem itself could help. When getting the value of a annotation field it is not typesave. Convinces methods could remove the need for the convert() call.

Indeed it felt to me that a concept like the tools-gem would help here to extract the values that are available in the annotations. It of course is not entirely possible to use tools-gem for itself 😄

Do you dislike the length of the method names?

Yes, I think that it is a bit too long. I think that it would be perfectly fine to have methods like asDeclared (that would return an optional or something like that) and toDeclared (when you know that it is a declared type) that will return Declared and throw an exception if it isn't.

Regarding the convert, I think that you can do it as part of the Shadow. I didn't clone your project to try it out. I need to do that, it looks quite interesting.

Im fully onboard when it comes to composition over inheritance. The problem with that in this case is, that the jdk api does use inheritance

The JDK indeed offers AbstractProcessor which makes use of the different annotations like SupportedAnnotationTypes, SupportedOptions, SupportedSourceVersion. Actually what is defined in SupportedAnnotationTypes and / or returned by Processor#getSupportedAnnotationTypes is going to be passed to the process method. Which I noticed is something that you do not have access to in the overridden method.

[LukasDetermann]

Regarding the convert, I think that you can do it as part of the Shadow. I didn't clone your project to try it out. I need to do that, it looks quite interesting.

a major pain point for me with the jdk api are lines like this

TypeElement typeElement = (TypeElement) element;
DeclaredType declaredType = (DeclaredType) typeElement.asType();

you need to know both data structures and how they overlap and there naming even makes it harder at times.
i want the convert() method to answer the question: in what type can i convert this object? Its revers inheritance if that makes sense. The closer i get to the leaf the less options should be there.

The JDK indeed offers AbstractProcessor which makes use of the different annotations like SupportedAnnotationTypes, SupportedOptions, SupportedSourceVersion. Actually what is defined in SupportedAnnotationTypes and / or returned by Processor#getSupportedAnnotationTypes is going to be passed to the process method. Which I noticed is something that you do not have access to in the overridden method.

yeah the jdk has 2 options to get annotated elements. i just chose to support only the direct one.
i think claiming annotations (the result of javax.annotation.processing.Processor#process) should never be done. if that is true every processor can support every annotation as none are ever claimed. if someone needs to claim annotations they would need to overwrite getSupportedAnnotationTypes as well with my processor or all annotations would be claimed.