Osticket: Mail fetcher - new email fails to create ticket

[TechCare-Aus]

osTicket Version | v1.11 (7b1eee8) —  Up to date
Web Server Software | Apache 2.4
MySQL Version | 10.2.21
PHP Version | 7.2.14

Updated today via Installatron.

Issue:
Prior to running PHP 7.2 I was running PHP 7.1 and if I sent an email to [email protected] it would show up in the inbox (unread). Cron would poll and then mail fetcher would do it's thing and place it in the folder RETRIEVED as read. But nothing would show up in osTicket.

If I created a ticket inside osTicket, then replied to it (so that it would send an email out to the end user) they could reply and osTicket would fetch that and business as usual.

But never did it create a NEW ticket off an email sent through to the primary email mailbox.

This was working previously.

Now I just on the off hand decided to turn on PHP 7.2 and then bam... all is well again. The only error showing in php is:

\var\log\apache2\error_log = PHP Warning: array_merge(): Argument #2 is not an array in /home/USERNAME/public_html/support/include/class.plugin.php on line 290

\home\USERNAME\log\cron_error.log = PHP Fatal error: APCuIterator::__construct(): APC must be enabled to use APCuIterator in /home/USERNAME/public_html/support/include/class.search.php on line 966

This is from when I ran PHP 7.1

\home\USERNAME\log\cron_error.log = [12-Feb-2019 00:53:01 UTC] PHP Warning: Declaration of TransferForm::render($options) should be compatible with Form::render($staff = true, $title = false, $options = Array) in /home/USERNAME/public_html/support/include/class.forms.php on line 4487

\var\log\apache2\error_log = [core:error] [pid 15415:tid 46933651949312] (104)Connection reset by peer: [client XXX.XXX.XXX.XXX:50942] AH00574: ap_content_length_filter: apr_bucket_read() failed

I couldn't find any other errors. But I'm happy to post anything you like. I'm running WHM and have full access to anything on the server.

[Siriuo]

Its good to know I am not the only one having this issue with osTicket v1.11. Only difference here is that I am running apache 2.2, php 5.6, mysql 5.5.62 (not mariadb), also tried php 7.2 with the same issue.

For some reason I am not seeing any errors in the logs. However it appears only some emails are not being created. I am trying to find similarities in the email itself.

[Siriuo]

So after some testing it appears that it is the contents of the email which causes the ticket to not be created. It seems that if the email contains a "," followed by a space on the first line, for example sending "Test, ", will result in the ticket not being created.

[JediKev]

@Siriuo

I cannot replicate this using PHP 5.6 and the same steps you provided. Maybe there is a special character cutting it off?

Cheers.

[Siriuo]

@JediKev

I have just tested with a single line saying "Good morning, " and this is failing. I have exported the source from the email which may help reproduce this. https://pastebin.com/2LQEuCm0

Also the testing has all been done on php 7.2 however the symptoms appeared with php 5.6

[JediKev]

@Siriuo

Oh....you're using outlook 😷

I will try to test but I'll need the entire raw email including headers, not just the body.

Cheers.

[Siriuo]

@JediKev

Yep Outlook 2016 and Office 365 😊

Headers are below.

https://pastebin.com/rNnMd0CY

[JediKev]

@Siriuo

I need the entire raw email including headers and body, not just the headers and not just the body lol

Cheers.

[Siriuo]

@JediKev

I knew that lol. Here 😂

https://pastebin.com/zXEHKsGK

[JediKev]

@Siriuo

Okay, so the only way I was able to replicate your issue is by piping the mail instead of fetching (fetching works for me all the time not matter what - v1.11, PHP v7.2, MariaDB). The ticket is created it just has no status_id so therefore it will not appear in the queues.

Looking into why piping doesn't set a status_id.

Cheers.

[adlerweb]

So, I guess I chime in here. I had a similar problem of "ignored" mails with v1.11 (d4e240b), Apache 2.4.34, PHP 7.1.26/7.2.14 and MySQL 5.7.24. New incoming mails where processed (aka deleted), however no new ticket created. MUA or text didn't seem to make any difference.

I tried running mailfetch directly on console and traced it - for me it was killed due to APC:
PHP Fatal error: APCuIterator::__construct(): APC must be enabled to use APCuIterator in [...]/include/class.search.php on line 966

APCu is installed, however not available/active when running in CLI. Checks like Apc->isAvailable(), function_exists or ini_get( 'apc.enable_cli' ) don't seem to work in this case since it is somewhat there but not usable. Sadly the error doesn't seem to be logged anywhere when running via cron.

I fixed it for me by omitting APCu-calls for CLI-SAPI:

--- a/include/class.search.php
+++ b/include/class.search.php
@@ -960,7 +960,7 @@ class SavedQueue extends CustomQueue {
     }

     static function clearCounts() {
-        if (function_exists('apcu_store')) {
+        if (PHP_SAPI !== 'cli' && function_exists('apcu_store')) {
             if (class_exists('APCUIterator')) {
                 $regex = '/^counts.queues.\d+.' . preg_quote(SECRET_SALT, '/') . '$/';
                 foreach (new APCUIterator($regex, APC_ITER_KEY) as $key) {

[JediKev]

@adlerweb

Yes, this is the exact error I found when debugging. Will look into why APCU is broken on PHP 7.2 cli.

Cheers.

[adlerweb]

I guess it was "broken" all the time - APCu was afaik only added starting with 1.11. APCu is usually enabled only for web, not CLI (apc.enabled=1, apc.enable_cli=0). The latter usually doesn't make a lot of sense since the cache is cleared after the PHP-thread exits anyway.

[pablopol24]

Lo arreglé para mí al omitir APCu-calls para CLI-SAPI:

--- a/include/class.search.php
+++ b/include/class.search.php
@@ -960,7 +960,7 @@ class SavedQueue extends CustomQueue {
     }

     static function clearCounts() {
-        if (function_exists('apcu_store')) {
+        if (PHP_SAPI !== 'cli' && function_exists('apcu_store')) {
             if (class_exists('APCUIterator')) {
                 $regex = '/^counts.queues.\d+.' . preg_quote(SECRET_SALT, '/') . '$/';
                 foreach (new APCUIterator($regex, APC_ITER_KEY) as $key) {

it worked for me

Versión osTicket | v1.11 (0f229ae) —  Hasta la fecha
Servidor del Software Web | Apache/2.4.29 (Ubuntu)
Versión MySQL | 5.7.25
Versión PHP | 7.2.15-0ubuntu0.18.04.1

[JediKev]

@pablopol24 @adlerweb @Siriuo @TechCare-Aus

Below is the official fix for the issue. Please test and ensure this addresses the issue.

4731

I tested this locally with my PHP 7.2 install (that had the same errors) and now it works correctly and creates the tickets.

Cheers.

[kosli]

@JediKev
i had the same issue (tickets created with status_id=0 and not showing up in the UI. But mine where fetched via IMAP, not piped in, do you thinkt that fix should resolve this issue too?
fyi, i am using the docker image of https://github.com/CampbellSoftwareSolutions/docker-osticket

[JediKev]

@kosli

I cannot replicate issues with fetching at all so it's unsure. Just try and see!

I'm not familiar with osTicket docker images so not sure if that's a cause as well.

Cheers.

[adlerweb]

But mine where fetched via IMAP, not piped in, do you thinkt that fix should resolve this issue too?

I was also using IMAP, so good chances it's the same problem.

[Siriuo]

@JediKev

I have just tested this fix and I can confirm it has fixed the issue. I will keep an eye on this just to be 100% sure.

[lubo]

@TechCare-Aus #4731 hasn't been merged yet. Why did you close this then?

[TechCare-Aus]

@lubo to be honest the issue was gone for me and @JediKev has created a working fix reported by others.

Is that against Github protocol?

I'm happy to re-open it again. I greatly dislike seeing so many issues that are still open and yet are clearly dealt with.

[lubo]

@TechCare-Aus Thank you for reopening. Technically, the issue is still not fixed in osTicket itself. Therefore, it's a good practice to leave the issue open until it is fixed. It helps the people interested in the issue to track progress.

[TechCare-Aus]

@JediKev as of a couple of days ago (I think 3) my osTicket installation stopped receiving emails again. I've managed to get tickets through by using the "autocron" feature.
To my knowledge I have changed nothing. It was working on PHP v7.2 but then stopped. I run my own WHM/cPanel hosting which is automatically updated but that is all I can think it might be.

Here is any logs that I've found:

Cron output:

Cannot load Zend OPcache - it was already loaded
PHP Warning:  Module 'apcu' already loaded in Unknown on line 0 <br />
<b>Warning</b>:  ini_set(): Headers already sent. You cannot change the session module's ini settings at this time in <b>/home/USERNAME/public_html/support/bootstrap.php</b> on line <b>19</b><br />
<b>Warning</b>:  session_cache_limiter(): Cannot change cache limiter when headers already sent in <b>/home/USERNAME/public_html/support/bootstrap.php</b> on line <b>21</b><br />

Cron in cPanel

/usr/bin/php -q /home/USERNAME/public_html/support/api/cron.php > /home/USERNAME/logs/cron_osticket.log 2>&1

Alternative Cron I've used in cPanel which shows up no error in the log, but doesn't bring the email through as a ticket

/opt/cpanel/ea-php72/root/usr/bin/php /home/techcare/public_html/support/api/cron.php >> /home/techcare/logs/cron_osticket.log 2>&1

PHP Error

[01-Mar-2019 11:07:23 UTC] PHP Warning:  get_class() expects parameter 1 to be object, null given in /home/USERNAME/public_html/support/include/class.thread.php on line 231

[adlerweb]

Sounds more like a local config issue. Try to run /opt/cpanel/ea-php72/root/usr/bin/php on the console without any arguments and check if the error persists. If yes there is something wrong with the config. Sounds like extension=apcu.o is somewhere duplicated inside php.ini or included files.

[TechCare-Aus]

@adlerweb thanks for the reply.
You were partly right it was a legacy reference in a php.ini file. But it still doesn't fetch email via cron.
Output from cron is:

Status: 301 Moved Permanently
X-Powered-By: PHP/7.2.15
Location: https://
Content-type: text/html; charset=UTF-8

Also when I try run /opt/cpanel/ea-php72/root/usr/bin/php it comes up with an error saying that I no location exists. Which actually makes sense because in shell for the user it is jailed.
But when I run /usr/bin/php it shows no error just just waits for more information or Ctrl + C to quit.

[TechCare-Aus]

If I run /usr/bin/php -f /home/USERNAME/public_html/support/api/cron.php >> /home/USERNAME/logs/cron_osticket.log 2>&1 I get zero errors in the cron log, but still no email fetch. Should the cron.php be treated as a file? Or something else?

[TechCare-Aus]

My extreme apologies guys. It was my own fault. The Status: 301 Moved Permanently should have immediately made me remember that about 4 days ago I noticed (at a clients house) that if I inputted into the URL bar mydomain.com/support/scp it would load up the page but not in https. So I google and found this https://sebtombs.com/computing/web-design-publishing/osticket/44-force-ssl-in-osticket-without-htaccess-changes.html but that is what broke my osTicket.