Psalm: storage issues

Created on 31 Jan 2020 · 37Comments · Source: vimeo/psalm

I had weird errors at work. I tried to run Psalm against almost all the project for the first time and I had this error:

Exception

Uncaught UnexpectedValueException: Expecting c:\users\Username\documents\phpstormprojects\project_name\req\v0.0.0\_global\objets\internes\customer_name.class.php:4544:237529:-:closure to have storage in C:\Users\Username\Documents\PhpstormProjects\project_name\req\v0.0.0\_GLOBAL\objets\internes\customer_name.class.php in C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Codebase.php:642
Stack trace:
#0 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClosureAnalyzer.php(24): Psalm\Codebase->getClosureStorage('C:\\Users\\Username...', 'c:\\users\\Username...')
#1 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer.php(462): Psalm\Internal\Analyzer\ClosureAnalyzer->__construct(Object(PhpParser\Node\Expr\Closure), Object(Psalm\Internal\Analyzer\StatementsAnalyzer))
#2 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Expression\CallAnalyzer.php(542): Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Expr\Closure), Object(Psalm\Context))
#3 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Expression\Call\FunctionCallAnalyzer.php(421): Psalm\Internal\Analyzer\Statements\Expression\CallAnalyzer::checkFunctionArguments(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Array, Array, 'uasort', Object(Psalm\Context))
#4 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer.php(442): Psalm\Internal\Analyzer\Statements\Expression\Call\FunctionCallAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Expr\FuncCall), Object(Psalm\Context))
#5 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\StatementsAnalyzer.php(683): Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Expr\FuncCall), Object(Psalm\Context), false, NULL, true)
#6 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer.php(738): Psalm\Internal\Analyzer\StatementsAnalyzer->analyze(Array, Object(Psalm\Context))
#7 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer.php(345): Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer::analyzeIfBlock(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Stmt\If_), Object(Psalm\Internal\Scope\IfScope), Object(Psalm\Context), Object(Psalm\Context), Object(Psalm\Context), Array)
#8 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\StatementsAnalyzer.php(359): Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Stmt\If_), Object(Psalm\Context))
#9 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\FunctionLikeAnalyzer.php(526): Psalm\Internal\Analyzer\StatementsAnalyzer->analyze(Array, Object(Psalm\Context), Object(Psalm\Context), true)
#10 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClassAnalyzer.php(1650): Psalm\Internal\Analyzer\FunctionLikeAnalyzer->analyze(Object(Psalm\Context), Object(Psalm\Internal\Provider\NodeDataProvider), Object(Psalm\Context))
#11 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClassAnalyzer.php(813): Psalm\Internal\Analyzer\ClassAnalyzer->analyzeClassMethod(Object(PhpParser\Node\Stmt\ClassMethod), Object(Psalm\Storage\ClassLikeStorage), Object(Psalm\Internal\Analyzer\ClassAnalyzer), Object(Psalm\Context), Object(Psalm\Context))
#12 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\FileAnalyzer.php(201): Psalm\Internal\Analyzer\ClassAnalyzer->analyze(Object(Psalm\Context), Object(Psalm\Context))
#13 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(335): Psalm\Internal\Analyzer\FileAnalyzer->analyze(NULL)
#14 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(567): Psalm\Internal\Codebase\Analyzer->Psalm\Internal\Codebase\{closure}(7004, 'C:\\Users\\Username...')
#15 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(258): Psalm\Internal\Codebase\Analyzer->doAnalysis(Object(Psalm\Internal\Analyzer\ProjectAnalyzer), 1)
#16 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ProjectAnalyzer.php(528): Psalm\Internal\Codebase\Analyzer->analyzeFiles(Object(Psalm\Internal\Analyzer\ProjectAnalyzer),1, false)
#17 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\psalm.php(553): Psalm\Internal\Analyzer\ProjectAnalyzer->check('C:\\Users\\Username...', false)
#18 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\psalm(2): require_once('C:\\Users\\Username...')
#19 {main}
(Psalm 3.8.3@389af1bfc739bfdff3f9e3dc7bd6499aee51a831 crashed due to an uncaught Throwable)

I looked at the code, unfortunately, this particular file is thousands of line long and I couldn't reproduce on a smaller sample. I then tried to ignore this whole file but I had an other error which seems related:

Exception

Uncaught InvalidArgumentException: Could not get class storage for customer_name in C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Provider\ClassLikeStorageProvider.php:47
Stack trace:
#0 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\ClassLikes.php(1447): Psalm\Internal\Provider\ClassLikeStorageProvider->get('customer_name')
#1 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Type\Reconciler.php(493): Psalm\Internal\Codebase\ClassLikes->getConstantForClass('customer_name', '$t_params_code_...', 4, NULL)
#2 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Type\Reconciler.php(229): Psalm\Type\Reconciler::getValueForKey(Object(Psalm\Codebase), 'customer_name::$t_para...', Array, Object(Psalm\CodeLocation), false, false)
#3 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer.php(256): Psalm\Type\Reconciler::reconcileKeyedTypes(Array, Array, Array, Array, Array, Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Array, true, Object(Psalm\CodeLocation))
#4 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\StatementsAnalyzer.php(359): Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Stmt\If_), Object(Psalm\Context))
#5 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\LoopAnalyzer.php(195): Psalm\Internal\Analyzer\StatementsAnalyzer->analyze(Array, Object(Psalm\Context))
#6 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\ForeachAnalyzer.php(308): Psalm\Internal\Analyzer\Statements\Block\LoopAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Array, Array, Array, Object(Psalm\Internal\Scope\LoopScope), Object(Psalm\Context))
#7 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\StatementsAnalyzer.php(371): Psalm\Internal\Analyzer\Statements\Block\ForeachAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Stmt\Foreach_), Object(Psalm\Context))
#8 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer.php(1553): Psalm\Internal\Analyzer\StatementsAnalyzer->analyze(Array, Object(Psalm\Context))
#9 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer.php(382): Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer::analyzeElseBlock(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Stmt\Else_), Object(Psalm\Internal\Scope\IfScope), Object(Psalm\Context), Object(Psalm\Context))
#10 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\StatementsAnalyzer.php(359): Psalm\Internal\Analyzer\Statements\Block\IfAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Stmt\If_), Object(Psalm\Context))
#11 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\FunctionLikeAnalyzer.php(526): Psalm\Internal\Analyzer\StatementsAnalyzer->analyze(Array, Object(Psalm\Context), Object(Psalm\Context), true)
#12 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClassAnalyzer.php(1650): Psalm\Internal\Analyzer\FunctionLikeAnalyzer->analyze(Object(Psalm\Context), Object(Psalm\Internal\Provider\NodeDataProvider), Object(Psalm\Context))
#13 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClassAnalyzer.php(813): Psalm\Internal\Analyzer\ClassAnalyzer->analyzeClassMethod(Object(PhpParser\Node\Stmt\ClassMethod), Object(Psalm\Storage\ClassLikeStorage), Object(Psalm\Internal\Analyzer\ClassAnalyzer), Object(Psalm\Context), Object(Psalm\Context))
#14 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\FileAnalyzer.php(201): Psalm\Internal\Analyzer\ClassAnalyzer->analyze(Object(Psalm\Context), Object(Psalm\Context))
#15 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(335): Psalm\Internal\Analyzer\FileAnalyzer->analyze(NULL)
#16 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(567): Psalm\Internal\Codebase\Analyzer->Psalm\Internal\Codebase\{closure}(7269, 'C:\\Users\\Username...')
#17 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(258): Psalm\Internal\Codebase\Analyzer->doAnalysis(Object(Psalm\Internal\Analyzer\ProjectAnalyzer), 1)
#18 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ProjectAnalyzer.php(528): Psalm\Internal\Codebase\Analyzer->analyzeFiles(Object(Psalm\Internal\Analyzer\ProjectAnalyzer),1, false)
#19 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\src\psalm.php(553): Psalm\Internal\Analyzer\ProjectAnalyzer->check('C:\\Users\\Username...', false)
#20 C:\Users\Username\Documents\PhpstormProjects\project_name\vendor\vimeo\psalm\psalm(2): require_once('C:\\Users\\Username...')
#21 {main}
(Psalm 3.8.3@389af1bfc739bfdff3f9e3dc7bd6499aee51a831 crashed due to an uncaught Throwable)

The "customer_name" string comes back on the second error so I guess those errors are related.

I don't have a code example to produce (as I said, I couldn't reproduce and the highlighted code was just a standard closure).

Do you have any hints what could cause that kind of errors or things I could try to fix it?

(Note: I couldn't make a lot of tests, our project is quite massive and it takes almost half an hour before failing...)

more info needed

Source

orklah

👍1

All 37 comments

Hey @orklah, can you reproduce the issue on https://psalm.dev ?

psalm-github-bot[bot] on 31 Jan 2020

Please run --debug-by-line to see where the error's being thrown, then copy-paste as much of that source code as possible.

muglug on 31 Jan 2020

👍1

Thanks, will try that next week.

orklah on 31 Jan 2020

Here's the trace

Exception

C:\{PATH}\{FOLDER}\failing_object_parent.class.php:1456
C:\{PATH}\{FOLDER}\failing_object_parent.class.php:1458
C:\{PATH}\{FOLDER}\failing_object_parent.class.php:5
C:\{PATH}\failing_object.class.php:4
C:\{PATH}\failing_object.class.php:58
C:\{PATH}\failing_object.class.php:59
…
C:\{PATH}\failing_object.class.php:9592
C:\{PATH}\failing_object.class.php:9602
C:\{PATH}\failing_object.class.php:9605
C:\{PATH}\failing_object.class.php:-1
C:\{PATH}\failing_object.class.php:9610
C:\{PATH}\failing_object.class.php:9611
C:\{PATH}\failing_object.class.php:9614
…
C:\{PATH}\failing_object.class.php:9900
C:\{PATH}\failing_object.class.php:9908
C:\{PATH}\failing_object.class.php:9909
C:\{PATH}\failing_object.class.php:9910
C:\{PATH}\failing_object.class.php:9911
C:\{PATH}\failing_object.class.php:9913
C:\{PATH}\failing_object.class.php:9916
C:\{PATH}\failing_object.class.php:9925
Uncaught UnexpectedValueException: Expecting c:\{PATH}\failing_object.class.php:9925:388710:-:closure to have storage in C:\{PATH}\failing_object.class.php in C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Codebase.php:642
Stack trace:
#0 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClosureAnalyzer.php(24): Psalm\Codebase->getClosureStorage('C:\\{PATH}...', 'c:\\{PATH}...')
#1 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer.php(462): Psalm\Internal\Analyzer\ClosureAnalyzer->__construct(Object(PhpParser\Node\Expr\Closure), Object(Psalm\Internal\Analyzer\StatementsAnalyzer))
#2 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\Expression\AssignmentAnalyzer.php(234): Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Expr\Closure), Object(Psalm\Context))
#3 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer.php(120): Psalm\Internal\Analyzer\Statements\Expression\AssignmentAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Expr\Variable), Object(PhpParser\Node\Expr\Closure), NULL, Object(Psalm\Context), NULL)
#4 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\StatementsAnalyzer.php(683): Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze(Object(Psalm\Internal\Analyzer\StatementsAnalyzer), Object(PhpParser\Node\Expr\Assign), Object(Psalm\Context), false, Object(Psalm\Context), true)
#5 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\FunctionLikeAnalyzer.php(526): Psalm\Internal\Analyzer\StatementsAnalyzer->analyze(Array, Object(Psalm\Context), Object(Psalm\Context), true)
#6 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClassAnalyzer.php(1650): Psalm\Internal\Analyzer\FunctionLikeAnalyzer->analyze(Object(Psalm\Context), Object(Psalm\Internal\Provider\NodeDataProvider), Object(Psalm\Context))
#7 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClassAnalyzer.php(813): Psalm\Internal\Analyzer\ClassAnalyzer->analyzeClassMethod(Object(PhpParser\Node\Stmt\ClassMethod),Object(Psalm\Storage\ClassLikeStorage), Object(Psalm\Internal\Analyzer\ClassAnalyzer), Object(Psalm\Context), Object(Psalm\Context))
#8 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\FileAnalyzer.php(201): Psalm\Internal\Analyzer\ClassAnalyzer->analyze(Object(Psalm\Context), Object(Psalm\Context))
#9 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(335): Psalm\Internal\Analyzer\FileAnalyzer->analyze(NULL)
#10 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(567): Psalm\Internal\Codebase\Analyzer->Psalm\Internal\Codebase\{closure}(3307, 'C:\\{PATH}...')
#11 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Codebase\Analyzer.php(258): Psalm\Internal\Codebase\Analyzer->doAnalysis(Object(Psalm\Internal\Analyzer\ProjectAnalyzer), 1)
#12 C:\{PATH}\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ProjectAnalyzer.php(528): Psalm\Internal\Codebase\Analyzer->analyzeFiles(Object(Psalm\Internal\Analyzer\ProjectAnalyzer),1, false)
#13 C:\{PATH}\vendor\vimeo\psalm\src\psalm.php(553): Psalm\Internal\Analyzer\ProjectAnalyzer->check('C:\\{PATH}...', false)
#14 C:\{PATH}\vendor\vimeo\psalm\psalm(2): require_once('C:\\{PATH}...')
#15 {main}
(Psalm 3.8.3@389af1bfc739bfdff3f9e3dc7bd6499aee51a831 crashed due to an uncaught Throwable)

I put ... where it was just a lot of repetition of consecutive lines.

The failing code look like this:

public function function_name(ident $ident) {
    $func = function(array $a) {
        return $a["id"];
    };
    $tmp  = array_map($func, $ident->get_activity_list());

    return in_array($this->get_id(), $tmp);
}

I tried to replicate on the same file as the first time but I couldn't. However, I reproduced on different files on differents runs. I was trying to create a baseline but I'm not sure it's related to the issue.
I run Psalm on my windows work computer with 16Go RAM and psalm is using between 8 and 10Go of that. I'm wondering if PHP may be triggering garbage collection and somehow deleting cache? I think the issue happened only on massive files.

EDIT: some runs actually completed (about half of them I'd say) so this bug is due to context and not because of a particular faulty file.

PS: sorry for the obfuscation, I'm not too keen of copy/pasting work code as-is.

orklah on 3 Feb 2020

I'm experiencing the same issue. It seems to occur mainly when I save files in PhpStorm (2019.3.3 on Windows 10) while psalm is running.

@orklah Can you tell whether the file mentioned in your exception message was modified during a psalm run?

caugner on 17 Mar 2020

I was using phpstorm too, but I don't think those files were modified. I'll keep an eye next time.

Thanks

orklah on 17 Mar 2020

Otherwise we both have in common that we're on Windows.

caugner on 17 Mar 2020

If the crash happens while updating a file, try to remember the changes you were making, and re-do the changes exactly. But otherwise there's not much I can do here.

muglug on 17 Mar 2020

Could Psalm conflict with itself, if it's being run both as LSP server and normally, at the same time?

weirdan on 17 Mar 2020

nope, there are separate caches – --clear-global-cache clears both, --clear-cache just clears the CLI one

muglug on 17 Mar 2020

Note: I'm not using the LSP server at all because of this issue.

However: I just observed the issue again.

Relevant diff

-                /** @var  $rows */
                 $rows = array_merge(
                     array_values($x),
                     array_values($y)
                 );

grafik

Exception

   UnexpectedValueException  : Expecting d:\path\to\project\src\parser\abstractparser.php:168:4231:-:closure to have storage in D:\path\to\project\src\Parser\AbstractParser.php

  at D:\path\to\project\vendor\vimeo\psalm\src\Psalm\Codebase.php:649
    645|         if (isset($file_storage->functions[$closure_id])) {
    646|             return $file_storage->functions[$closure_id];
    647|         }
    648|
  > 649|         throw new \UnexpectedValueException(
    650|             'Expecting ' . $closure_id . ' to have storage in ' . $file_path
    651|         );
    652|     }
    653|

  Exception trace:

  1   Psalm\Codebase::getClosureStorage("D:\path\to\project\src\Parser\AbstractParser.php", "d:\path\to\project\src\parser\abstractparser.php:168:4231:-:closure")
      D:\path\to\project\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\ClosureAnalyzer.php:30

  2   Psalm\Internal\Analyzer\ClosureAnalyzer::__construct(Object(PhpParser\Node\Expr\Closure), Object(Psalm\Internal\Analyzer\StatementsAnalyzer))
      D:\path\to\project\vendor\vimeo\psalm\src\Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer.php:462

  Please use the argument -v to see more details.

Closure context

    /**
     * @psalm-param list<Text|Table> $elements
     * @param Text[]|Table[] $elements
     * @psalm-return list<Text|Table>
     * @return Text[]|Table[]
     */
    public function splitSingleTexts(array $elements): array
    {
        return array_map(
            /**
             * @param Text|Table $element
             */
            function (TextOrTable $element): TextOrTable {
                if ($element instanceof Text && $element->getLineCount() === 1) {
                    $lines = $element->getLines();
                    $lines = preg_split("/([ \r\t]*\n){2,}/", $lines[0]);
                    $lines = array_map('trim', $lines);
                    $element = new Text($lines);
                }
                return $element;
            },
            $elements
        );
    }

grafik

caugner on 18 Mar 2020

Another change that caused an Exception:

UnexpectedValueException : Expecting d:\path\to\project\src\parser\scriptinitializer.php:54:1374:-:closure to have storage in D:\path\to\project\src\Parser\ScriptInitializer.php

         $paragraphs = array_merge(
             ...array_map(
                 /**
                  * @param string
                  * @psalm-return list<string>
                  * @return string[]
-                 */
-                function (string $str): array {
+                */function (string $str): array {
                     return preg_split("@\r?\n\s*\r?\n@ums", $str) ?: [];
                 },
                 $paragraphs
             )
         );

grafik

Hypothesis: The issue occurs when the line number of a closure changes during analysis (in this case from line 54 to 53).

caugner on 18 Mar 2020

I ran Psalm multiple times these days on a separate console and while making sure no code was modified and I didn't reproduce. This must have been the same thing as @caugner is decribing.

For me it's a missuse of the tool, so I consider this can be closed if there's nothing more we can do...

orklah on 18 Mar 2020

For me it's a missuse of the tool

@orklah Are you suggesting I must not change any code while psalm is running? In one of my projects, a full run takes 60 sec. I have had the same issue when I tried using psalm as a LSP.

caugner on 18 Mar 2020

As I said when i created the issue, our project takes about 30 minutes to get analyzed.
When I tried to replicate these days, I only took the core of the application and it still takes 10 minutes.

I can't speak for you, but its a non-issue for us. If we want to run a full analysis, this should be done on CI or something like that, not on the developper computer.

For LSP, I can't speak much, I never tried. I see how it could be an issue.

However, I don't know how Psalm could handle changing code nicely. It's a cache invalidation nightmare, without speaking of detecting such changes. Best case scenario, it will output a nice error output instead of crashing. Any other output it could generate should be treated as highly suspicious...

orklah on 18 Mar 2020

30 minutes indicates that something is quite wrong - could you profile running a subset with Blackfire.io and tell me what the trace looks like?

muglug on 18 Mar 2020

I'll try. Probably next week though.

orklah on 18 Mar 2020

Also, note that it's an old project with no namespace, custom autoloading, analyzed on a local machine and without cache due to https://github.com/vimeo/psalm/issues/2992

This may explain things...

orklah on 18 Mar 2020

@muglug Is there any explanation for the high column number (237529) here:

Uncaught UnexpectedValueException: Expecting c:\path\to\file.php:4544:237529:-:closure to have storage

I have also noticed this unrealistically high number in my Exceptions.

caugner on 19 Mar 2020

Is there any explanation for the high column number

Yeah, it's not a column (which takes a while to compute) but an offset into the file

muglug on 19 Mar 2020

👍1

Just had the issue once again after having changed a file while psalm was running.

https://github.com/vimeo/psalm/blob/89a3af54a62ccc7baa128e2ffe01ddc1140825df/src/Psalm/Codebase.php#L641-L653

If the location of the closure changes between filling Codebase::$file_storage and accessing it, Codebase::getClosureStorage() cannot find the closure and therefore throws the Exception.

Since this case cannot really be avoided (apart from not changing files while psalm is running), would one of these options be viable?

Emit a warning rather than terminating the run.
Trigger the run to restart.
Rescan just the changed file.

caugner on 19 Mar 2020

Rescanning the changed file, but as long as you’re using cache Psalm could use its cached copy of files once a run starts

muglug on 19 Mar 2020

👍1

30 minutes indicates that something is quite wrong - could you profile running a subset with Blackfire.io and tell me what the trace looks like?

I couldn't run on the full project because my computer doesn't have enough memory, but I ran psalm against the main files of the application.

This represents 318 files and psalm took a little more than 10 minutes:

Checks took 647.15 seconds and used 2,144.011MB of memory
Psalm was able to infer types for 88.7283% of the codebase

Here's the generated profile:
https://blackfire.io/profiles/f528a139-d5ce-4c74-aa35-728a664a819a/graph?settings%5Bdimension%5D=wt&settings%5Bdisplay%5D=landscape&settings%5BtabPane%5D=nodes&selected=&callname=main()

I don't know how long it will stay online as I'm on the demo version of Blackfire

orklah on 29 Mar 2020

@orklah it looks like there's one _very big_ hot path there – the AlgebraChecker::simplifyCNF function, which takes up almost half the runtime.

I imagine you could make everything much faster by putting return array_values($clauses) at the top of that function.

It looks like you have some very complex switch statements – Psalm is taking over 100ms (on average) to analyse them, which is an aberration.

Doing some profiling on Psalm's own source code, only one file has a switch statement that takes over 100ms to analyse, and it has some _very_ convoluted logic.

I've added some profiling in debug mode here: ce35324

Could you look at some of the slowest-performing methods and tell me what you see?

muglug on 29 Mar 2020

Is it possible that somehow, Psalm ends up analyzing a file in a directory that has been excluded? For example, if there's an include?

When I looked at psalm's cache, the biggest files were related to a big dependancy that's located in an excluded folder. That's the only thing that comes to my mind that could cause this issue.

Also, if you're interested, I tried to run a biggest sample and I had an Out of Memory error(8Go):
https://blackfire.io/profiles/dca62709-9307-4e0d-bbdc-a14672807c91/graph

I'll try with your debug mode but I have to modify the code each time to add some set_time_limit here and there. I don't understand, Psalm seem to reset it to 60 seconds each time.

orklah on 29 Mar 2020

https://blackfire.io/profiles/c3e8ee48-4f64-4fea-a716-448599fe1a37/graph

This is the profile when using return array_values($clauses) in simplifyCNF.

Checks took 395.02 seconds and used 2,144.008MB of memory
Psalm was able to infer types for 88.7295% of the codebase

It's about twice faster.

I'll try the debug commit now

orklah on 29 Mar 2020

ooookay!

We indeed have a file that has BIG switch split into 2 methods. There's 780 (very simple) "case" in that file and it's making the analyze very long. Those two methods takes around 3-4 minutes each to analyze.
(FYI, one of the two method is just a big switch with an endless case "string": return "other string";)

I guess we'll refactor this or ignore this file from analysis!

Just for curiosity, I ran psalm just against this file:
https://blackfire.io/profiles/8d22295d-61f0-4575-aa1e-d802e13d81d4/graph

I'll use the analyze time to see if we have other places where it could be an issue to psalm. Are you interested to know if I happen to find other abnormalities?

Thanks a lot for the support!

orklah on 29 Mar 2020

Yeah, any other abnormalities are very interesting.

How are those case statements arranged? Does each case always break, or is there a flow from one case statement to the one below?

muglug on 29 Mar 2020

Two big blocks like that:

switch($string){
  case 'str1':
     return 'str2';
  case 'str3':
     return 'str4';
}

EDIT(one of the two returns array instead of string but it doesn't change anything)

About the OOM error from earlier, this is caused by big docbloc annotation like "object1|object2|object3...'
The one crashing psalm has 130 objects in it (I went a little overboard with this one :p )

orklah on 29 Mar 2020

The one crashing psalm has 130 objects in it

Ah ok, that's a little crazy, but it should be possible to fix the crashing

muglug on 30 Mar 2020

The one crashing psalm has 130 objects in it

Ah ok, that's a little crazy, but it should be possible to fix the crashing

FYI, the method is a factory that returns objects with different signatures. We know which object will be returned from context only(no template possible). That's why I documented every possible type here, to have phpstorm autocompletion. I fixed the error by putting an @psalm-return with the common interface (that do not cover 99% of the methods)

orklah on 30 Mar 2020

@muglug Would you mind looking at this profiling?
https://blackfire.io/profiles/d6859c36-fdbd-4c3f-a580-3af0293f85a4/graph?settings%5Bdimension%5D=io&settings%5Bdisplay%5D=landscape&settings%5BtabPane%5D=nodes&selected=&callname=main()
(EDIT: I tried after clearing cache, it may provide a better view: https://blackfire.io/profiles/f1ff27fb-5881-4fd1-9ae6-60f86e415614/graph?settings%5Bdimension%5D=io&settings%5Bdisplay%5D=landscape&settings%5BtabPane%5D=nodes&selected=&callname=main() )

It was launched using the command blackfire run vendor\bin\psalm --config=psalm5.xml --alter --issues=MissingReturnType --safe-types on something around 300 files.

Psalter crashes after a minute with a time limit of 60 reached. It spend the entire time reading and writing files in cache. Unfortunately, I cannot disable the cache on psalter because there is no such option. I think this is pretty much the same issue I had in #2992 but at least, I could disable the cache as a workaround.

Do you have any solution for any of these issues:

Timeout limited to 60 (ignore my php.ini configuration and set_time_limit in my autoloader)
Underlying issue that could explain the time consumption
a way to disable cache on psalter

Any of those could solve my issue :)

Thanks a lot!

orklah on 2 Apr 2020

Just wondering: Is that crash still related to the same storage issue?

caugner on 2 Apr 2020