Quarkus: Support sending emails in native mode

[himanshukapoor04]

I am creating a service using quarkus framework to send emails. This service use Apache commons email library to send emails over SMTP. If I create a native image then the email is not sent with the following root cause

Caused by: javax.mail.NoSuchProviderException: smtp
        at javax.mail.Session.getService(Session.java:874)
        at javax.mail.Session.getTransport(Session.java:804)
        at javax.mail.Session.getTransport(Session.java:745)
        at javax.mail.Session.getTransport(Session.java:725)
        at javax.mail.Session.getTransport(Session.java:782)
        at javax.mail.Transport.send0(Transport.java:249)
        at javax.mail.Transport.send(Transport.java:124)

It works fine otherwise if the service is executed through jar file.
Complete stack trace is

Exception: Sending the email to the following server failed : smtp.abc.net:587
        at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1469)
        at org.apache.commons.mail.Email.send(Email.java:1496)
        at com.abc.cde.service.notification.service.EmailServiceImpl.sendEmailAfterSettingValues(EmailServiceImpl.java:77)
        at com.abc.cde.service.notification.service.EmailServiceImpl.sendEmail(EmailServiceImpl.java:43)
        at com.abc.cde.service.notification.service.EmailServiceImpl_ClientProxy.sendEmail(Unknown Source)
        at com.abc.cde.service.notification.endpoint.v1.Endpoints.sendEmail(Endpoints.java:40)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:152)
        at org.jboss.resteasy.core.MethodInjectorImpl.lambda$invoke$3(MethodInjectorImpl.java:123)
        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
        at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614)
        at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983)
        at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:110)
        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:123)
        at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:543)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:418)
        at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:372)
        at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:374)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:343)
        at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invoke$1(ResourceMethodInvoker.java:317)
        at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981)
        at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124)
        at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:110)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:317)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:476)
        at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:252)
        at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:153)
        at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
        at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:156)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:238)
        at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:229)
        at io.quarkus.resteasy.runtime.ResteasyFilter.doFilter(ResteasyFilter.java:45)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
        at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
        at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
        at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
        at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132)
        at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
        at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
        at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
        at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
        at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
        at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
        at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at io.quarkus.undertow.runtime.UndertowDeploymentTemplate$7$1$1.call(UndertowDeploymentTemplate.java:415)
        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
        at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1998)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1525)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1382)
        at java.lang.Thread.run(Thread.java:748)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:481)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
Caused by: javax.mail.NoSuchProviderException: smtp
        at javax.mail.Session.getService(Session.java:874)
        at javax.mail.Session.getTransport(Session.java:804)
        at javax.mail.Session.getTransport(Session.java:745)
        at javax.mail.Session.getTransport(Session.java:725)
        at javax.mail.Session.getTransport(Session.java:782)
        at javax.mail.Transport.send0(Transport.java:249) 

Sample code to reproduce this

@ApplicationScoped
public class EmailServiceImpl implements EmailService {

@Inject
ApplicationProperties applicationProperties;

public void sendEmail(Email importEmail) {
   try {
            Email email = new HtmlEmail();
            email.setHtmlMsg(htmlBody );
            email.setCharset("UTF-8");
            email.setHostName(applicationProperties.getEmailHost());
            email.setSmtpPort(applicationProperties.getEmailPort());


            email.setAuthenticator(authenticator);
            email.setStartTLSEnabled(true);
            setEmailFrom(email, importEmail);
            email.setSubject(importEmail.getSubject());
            email.addTo(importEmail.getTo());
            email.send();
        } catch (Exception e) {
            logger.error("Error occurred when sending email", e);            
        }
    }
}

[himanshukapoor04]

Tried adding the java files which native image couldn't find using below code

import com.sun.mail.smtp.SMTPTransport;
import lombok.extern.slf4j.Slf4j;
import org.graalvm.nativeimage.Feature;
import org.graalvm.nativeimage.RuntimeReflection;

@AutomaticFeature
@Slf4j
public class RuntimeReflectionRegistrationFeature implements Feature {

    @Override
    public void beforeAnalysis(BeforeAnalysisAccess access) {
        try {
            RuntimeReflection.register(SMTPTransport.class);
            RuntimeReflection.register(SMTPTransport.class.getFields());
            RuntimeReflection.register(SMTPTransport.class.getMethods());
            RuntimeReflection.register(SMTPTransport.class.getDeclaredConstructors());
        } catch (Exception e) {
            log.error("Error occurred while doing automatic feature ", e);
        }
    }
}

After this it fails with following error

Caused by: java.net.SocketException: java.security.NoSuchAlgorithmException: class configured for SSLContext (provider: SunJSSE) cannot be found.
        at javax.net.ssl.DefaultSSLSocketFactory.throwException(SSLSocketFactory.java:248)
        at javax.net.ssl.DefaultSSLSocketFactory.createSocket(SSLSocketFactory.java:270)
        at com.sun.mail.util.SocketFetcher.startTLS(SocketFetcher.java:552)
        at com.sun.mail.smtp.SMTPTransport.startTLS(SMTPTransport.java:2150)
Caused by: java.lang.ClassNotFoundException: sun.security.ssl.SSLContextImpl$DefaultSSLContext
        at com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:51)
        at java.lang.Class.forName(DynamicHub.java:1051)
        at java.security.Provider$Service.getImplClass(Provider.java:1634)
        ... 82 more

Then I updated the RuntimeReflectionRegistrationFeature class by adding following snippet

    RuntimeReflection.register(SSLContextImpl.DefaultSSLContext.class);
    RuntimeReflection.register(SSLContextImpl.DefaultSSLContext.class.getDeclaredConstructors());
    RuntimeReflection.register(SSLContextImpl.DefaultSSLContext.class.getFields());
    RuntimeReflection.register(SSLContextImpl.DefaultSSLContext.class.getMethods());

I also added following in application.properties

quarkus.ssl.native=true

and in pom.xml updated the plugins with enableJni flag

<plugins>
                    <plugin>
                        <groupId>io.quarkus</groupId>
                        <artifactId>quarkus-maven-plugin</artifactId>
                        <version>${quarkus.version}</version>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>native-image</goal>
                                </goals>
                                <configuration>
                                    <enableHttpUrlHandler>true</enableHttpUrlHandler>
                                    <enableJni>true</enableJni>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>

After that got another error while running native image

Caused by: javax.mail.MessagingException: IOException while sending message;
  nested exception is:
        javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed; 
        boundary="----=_Part_0_1526231940.1554298313558"
        at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1365)
        at javax.mail.Transport.send0(Transport.java:255)
        at javax.mail.Transport.send(Transport.java:124)
        at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1459)
        ... 67 more
Caused by: javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed; 
        boundary="----=_Part_0_1526231940.1554298313558"
        at javax.activation.ObjectDataContentHandler.writeTo(DataHandler.java:896)
        at javax.activation.DataHandler.writeTo(DataHandler.java:317)
        at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1694)
        at javax.mail.internet.MimeMessage.writeTo(MimeMessage.java:1913)
        at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1315)
        ... 70 more

Did some googling and found out over here(https://stackoverflow.com/a/25650033) that could be because reflection issue of MimeMultipart.java class
(Although not the accepted answer but made sense as reflection is being used)

            RuntimeReflection.register(MimeMultipart.class);
            RuntimeReflection.register(MimeMultipart.class.getDeclaredConstructors());
            RuntimeReflection.register(MimeMultipart.class.getDeclaredFields());
            RuntimeReflection.register(MimeMultipart.class.getMethods());

But still getting the same error as earlier i.e. no object DCH for MIME type multipart/mixed

[himanshukapoor04]

Registered few more Java classes as given below
`
RuntimeReflection.register(multipart_mixed.class);
RuntimeReflection.register(multipart_mixed.class.getMethods());
RuntimeReflection.register(multipart_mixed.class.getDeclaredClasses());
RuntimeReflection.register(multipart_mixed.class.getDeclaredConstructors());

        RuntimeReflection.register(text_html.class);
        RuntimeReflection.register(text_html.class.getMethods());
        RuntimeReflection.register(text_html.class.getDeclaredClasses());
        RuntimeReflection.register(text_html.class.getDeclaredConstructors());

        RuntimeReflection.register(handler_base.class);
        RuntimeReflection.register(handler_base.class.getMethods());
        RuntimeReflection.register(handler_base.class.getDeclaredClasses());
        RuntimeReflection.register(handler_base.class.getDeclaredConstructors());

        RuntimeReflection.register(image_gif.class);
        RuntimeReflection.register(image_gif.class.getMethods());
        RuntimeReflection.register(image_gif.class.getDeclaredClasses());
        RuntimeReflection.register(image_gif.class.getDeclaredConstructors());

        RuntimeReflection.register(image_jpeg.class);
        RuntimeReflection.register(image_jpeg.class.getMethods());
        RuntimeReflection.register(image_jpeg.class.getDeclaredClasses());
        RuntimeReflection.register(image_jpeg.class.getDeclaredConstructors());

        RuntimeReflection.register(message_rfc822.class);
        RuntimeReflection.register(message_rfc822.class.getMethods());
        RuntimeReflection.register(message_rfc822.class.getDeclaredClasses());
        RuntimeReflection.register(message_rfc822.class.getDeclaredConstructors());

        RuntimeReflection.register(text_plain.class);
        RuntimeReflection.register(text_plain.class.getMethods());
        RuntimeReflection.register(text_plain.class.getDeclaredClasses());
        RuntimeReflection.register(text_plain.class.getDeclaredConstructors());

        RuntimeReflection.register(text_xml.class);
        RuntimeReflection.register(text_xml.class.getMethods());
        RuntimeReflection.register(text_xml.class.getDeclaredClasses());
        RuntimeReflection.register(text_xml.class.getDeclaredConstructors());

`
It didn't removed the error. Then added following code at the startup of the application

MailcapCommandMap mc = (MailcapCommandMap) CommandMap.getDefaultCommandMap(); mc.addMailcap("text/html;; x-java-content-handler=com.sun.mail.handlers.text_html"); mc.addMailcap("text/xml;; x-java-content-handler=com.sun.mail.handlers.text_xml"); mc.addMailcap("text/plain;; x-java-content-handler=com.sun.mail.handlers.text_plain"); mc.addMailcap("multipart/*;; x-java-content-handler=com.sun.mail.handlers.multipart_mixed"); mc.addMailcap("message/rfc822;; x-java-content-handler=com.sun.mail.handlers.message_rfc822"); CommandMap.setDefaultCommandMap(mc);

After these changes the mails can be sent.

[gsmet]

There are 2 approaches here:

• either we make it work by adding things in quarkus-core as we consider mail should work out of the box
• or create a specific extension for emailing support

I lean towards the second option as a lot of applications won't need the emailing layer and low level Java stuff often requires a ton of classes.

@machi1990 I wonder if you would be interesting trying to put together a mail extension? That would be a nice challenge.

I think we already have quite a lot of information here thanks to @himanshukapoor04 . The last part with MailcapCommandMap needs a bit more work. We need to understand what's missing for them to be registered as they usually are.

[himanshukapoor04]

@gsmet I assume this code in MailCommanndMap is not getting executed properly. It could be because of the reason that files from META-INF of activation.jar are not getting copied over in graalvm image.

public MailcapCommandMap() {
    super();
    List dbv = new ArrayList(5);    // usually 5 or less databases
    MailcapFile mf = null;
    dbv.add(null);      // place holder for PROG entry

    LogSupport.log("MailcapCommandMap: load HOME");
    try {
        String user_home = System.getProperty("user.home");

        if (user_home != null) {
        String path = user_home + File.separator + ".mailcap";
        mf = loadFile(path);
        if (mf != null)
            dbv.add(mf);
        }
    } catch (SecurityException ex) {}

    LogSupport.log("MailcapCommandMap: load SYS");
    try {
        // check system's home
        if (confDir != null) {
        mf = loadFile(confDir + "mailcap");
        if (mf != null)
            dbv.add(mf);
        }
    } catch (SecurityException ex) {}

    LogSupport.log("MailcapCommandMap: load JAR");
    // load from the app's jar file
    loadAllResources(dbv, "META-INF/mailcap");

    LogSupport.log("MailcapCommandMap: load DEF");
    mf = loadResource("/META-INF/mailcap.default");

    if (mf != null)
        dbv.add(mf);

    DB = new MailcapFile[dbv.size()];
    DB = (MailcapFile[])dbv.toArray(DB);
    }

[gsmet]

@himanshukapoor04 good detective work.

Looks like we now have everything to write a quarkus-mail extension :). Who wants to do it? I can mentor and help, of course.

[himanshukapoor04]

@gsmet I can do it.

[machi1990]

@gsmet Indeed a nice challenge and I was interested to go for it. But I think we have our taker here @himanshukapoor04: nice investigation of the issue by the way, bravo. I can offer my help if needed.

[himanshukapoor04]

@gsmet is there any documentation like best practices on creating extensions?

[machi1990]

@himanshukapoor04 there is this writing your extension guide.

[cescoffier]

FYI, we are going to provide a mail client soon. It won't be based on javax.mail (due to its blocking aspects) but would allow you to send emails (and attachments).

[himanshukapoor04]

Would it also have ability to sign messages like using DKIM?

[cescoffier]

@himanshukapoor04 we can add this feature. @pmlopes WDYT?

[himanshukapoor04]

Should I close this bug then?

[gsmet]

@himanshukapoor04 I changed the title of the issue to be more general. We definitely need an issue to track this.

[pmlopes]

I've added the following issue: https://github.com/vert-x3/vertx-mail-client/issues/102 to track DKIM support

[gsmet]

@cescoffier any news on this one? It's really something we need. I'm sure we could find people to help if we had a design document.

[pmlopes]

@gsmet the vertx mail client is now "reflection" free but it's still on master no releases yet, you can try going from there. The DKIM feature is still not implemented yet.

[gsmet]

@pmlopes OK, so I suppose we would need a release then.

But what I was asking was more about a plan on how to integrate it. Do we have Quarkus config to setup the mail config, if so what is necessary.

If we have some solid plan, I think it would be perfectly suited for a contributor.

/cc @cescoffier

[Dufgui]

hello. Could i help on this ?

[Dufgui]

The idea is to use vertx mail client instead of apache Common mail ? And provide an extension for this. Maybe 2 extensions: one for sync approach (Apache common) and one for async (vertx) ?

[Dufgui]

Hello, Could I help on this?

/cc @gsmet @cescoffier

[gsmet]

We now have a Mailer extension thanks to @cescoffier .