Psalm: Documentation improvement for plugin system

[robehickman]

As @muglug demonstrated to me in another issue, one possible use for afterFunctionCallAnalysis is to add more type information to a return value. I think that the documentation needs to demonstrate how to construct different types. From looking around the source and available plug-ins I've seen a few ways of doing so:

// I assume this constructs from any valid doc string type
\Psalm\Type::parseString('WP_Comment|null');

// Construct programatically
new Union([new TNamedObject('some class name')]);
new Union([new TArrayKey]);

I think this should be documented. In the second case, I think it would be useful to have code examples showing several different types, especially more complex things such as 'object like' associative arrays. All type descriptions appear to use 'Union' as a base type?

Is an 'object-like array' constructed something like this?

new Union([
    new TArray([
        new TArrayKey(['key_1' => new TClassString()]),
        new TArrayKey(['key_2' => new TInt()]),
        new TArrayKey(['key_3' => new TBool()])])]);

I'm unsure as 'Type.php' never passes anything into 'TArrayKey' that I can see.

Is 'TClassString' a literal string or something else? If it is a literal string I find this name confusing as php strings aren't classes as far as I'm aware.

What is the difference between TLiteralInt and TInt?

There are several other things that I think could be clarified in the documentation. For instance, the one about checking non php files mentions 'FileChecker' which isn't mentioned in 'plugins.md'. The documentation could also offer some example uses of each of the different plug-in types. Such as:

AfterFunctionCallAnalysisInterface - called after Psalm analyzes a function call. You can use this to supplement return type information on function calls, and...

I would be willing to contribute to this, although my time is very limited until next month.

[edited to add additional detail]

[robehickman]

I just wrote the following, which could be a basis of a page documenting the internals of the type system for plug-in authors. It is based on my assumptions made in the previous post. The code example of constructing an associative array is probably incorrect. Text in [] is notes.

Psalm's type system represents the types of variables within a program using different classes. Primitive types like floats, integers and strings, plus arrays, and classes. You can find all of these in 'src/Psalm/Types'. Plugins both receive, and can update this type information.

Note that some types have multiple classes, Integers, for example, are represented by TLiteralInt and TInt. [Describe how these two differ.]

There are several ways of constructing new types. One is to new up objects directly. This constructs a type representing a class 'some_class'. Note that all types are contained within a union, even if there is only one [is this correct?].

new Union([new TNamedObject('some_class')]);

[It may be worth starting with usage of the Union class, if this is the container for everything else]

More complex types can be constructed as follows. The following represents an assosiative array with 3 keys. [I assume this code is incorrect.]

new Union([
    new TArray([
        new TArrayKey(['key_1' => new TClassString()]),
        new TArrayKey(['key_2' => new TInt()]),
        new TArrayKey(['key_3' => new TBool()])])]);

[Give example of a type describing an object as well.]

A possibly simpler way is to use the class \PsalmType includes a static method 'parseString', which will produce instances of the type classes from any doc string.

\Psalm\Type::parseString('int|null');

[weirdan]

Note that some types have multiple classes, Integers, for example, are represented by TLiteralInt and TInt. [Describe how these two differ.]

TLiteral*s represent cases when Psalm knows not only the type, but also specific value. So, for example, here

function f(int $p): void {}

type of $p parameter is represented internally as TInt (or was that TInteger?), but here

f(1);

Psalm knows not only the type, but actual value of the parameter, so it's int(1) (TLiteralInt(1) internally), not just int. Naturally literal types are subtypes of their non-literal counterparts, so TLiteralInt is accepted where TInt is expected.

Another example would be this:
https://github.com/vimeo/psalm/blob/8408effe5794d575a966dc0bae388d77abc3e50f/src/Psalm/Internal/Provider/ReturnTypeProvider/VersionCompareReturnTypeProvider.php#L69-L73

version_compare may return 1, 0, or -1, but never, say, 256.

[robehickman]

@weirdan Cool, that makes sense. I've updated the text as follows. I think that the representation of the associative array is now correct as well.

Psalm's type system represents the types of variables within a program using different classes. Primitive types like floats, integers and strings, plus arrays, and classes. You can find all of these in 'src/Psalm/Types/Atomic'. Plugins both receive, and can update this type information.

Receiving type object instances

Psalm attaches type information to the AST nodes passed into plugins. [expand]

Creating type object instances

There are two ways of creating the object instances which describe a given type. They can be created directly using new, or created declaratively from a doc string. Normally, you'd want to use the second option. Howeaver, understanding the structure of this data will help you understand types passed into a plugin.

Creating type object instances directly

The following example constructs a types representing a string, a floating point number, and a class called 'some_class'.

new TLiteralString('A text string')
new TLiteralFloat(3.142)
new TNamedObject('some_class')

Psalms internal representation of types can be considered a tree structure, and this tree is always rooted with type Union. A union specifies a set of types. A union of type int or string could hold either an integer or a string, but never both at once.

As noted, all types in Psalm are rooted to a union. If only a single item is to be represented, this union contains only one item, as in the first example. The second example shows a union of type string or int.

new Union([new TNamedObject('some_class')]);
new Union([new TString(),
                  new TInt()]);

More complex types can be constructed as follows. The following represents an assosiative array with 3 keys. Psalm calls these 'object-like arrays', and represents them with the 'ObjectLike' class.

        new Union([
            new ObjectLike([
                'key_1' => new Union([new TString()]),
                'key_2' => new Union([new TInt()]),
                'key_3' => new Union([new TBool()])])]);

Creating type object instances from doc string types

Another way of creating these instances is to use the class \PsalmType which includes a static method 'parseString'. You may pass any doc string type description to this, and it will return the corresponding object representation.

\Psalm\Type::parseString('int|null');

You can find how psalm would represent a given type as objects, by specifying the type as an input to this function, and calling var_dump on the result.

Idiosyncrasies

Some types are represented by two specialisations, for example TInt and TLiteralInt. The first variant refers to the type integer, where the exact value is unknown. Such as an integer parameter to a function.

function foo(int $bar): void {}

TLiteralInt is used to represent an integer variable where the exact value is known. This is the case when a function is called, or an integer is assigned to a variable.

foo(42);
$baz = 4567;

[weirdan]

Psalms internal representation of types can be considered a tree structure, and this tree is always rooted with type Union.

I wouldn't concentrate much on that. Pervasive usage of unions in Psalm is not an inherent feature of the type system itself. As far as I understand the reason Psalm uses them in most of the places is because (almost) anywhere you may expect a type, you can get a union as well (property types, return types, argument types, etc). So wrapping a single atomic type (like TInt) in a union container allows to uniformly handle that type elsewhere, without repetitive checks like this:

if ($type instanceof Union) 
   foreach ($types->getTypes() as $atomic) 
      handleAtomic($atomic); 
else handleAtomic($type);

// with union container it becomes
foreach ($types->getTypes() as $atomic)
   handleAtomic($atomic);

Also union trees are always shallow, because Psalm will flatten union of unions into a single-level union ((A|B)|(C|D) => A|B|C|D).

[muglug]

@robehickman pls feel free to open a PR (you can use GitHub's new WIP functionality if you want).

The first option I'd mention is to just use Type::parseString(...) using the same syntax you'd use in a docblock. If you're aiming for exactness you can also construct types by hand, but Type::parseString is always going to be the more robust option.

The syntax for object-like arrays is using the regrettably-named ObjectLike class (I plan to change to TObjectLikeArray in Psalm 4).

So you'd do

new Union([
    new Type\Atomic\ObjectLike([
        'first' => Type::getInt(),
        'second' => Type::getString(),
    ])
])

Notice the Type::getInt() shorthand - you can also use Type::getInt(5) to generate a union type corresponding to the literal int value 5.

[robehickman]

Thanks @weirdan @muglug I'll make changes based on the suggestions, and make a PR tomorrow.

What is the difference between TString and TClassString? Does this matter to a user of the plugin API? Also, why are 'TTrue' and 'TFalse' dedicated types in addition to TBool? What is TArrayKey if it dosn't represent a key in an associative array?

I assume that of the classes in 'src/Psalm/Types/Atomic', only the ones prefixed 'T' are actually used to represent types at runtime?

[weirdan]

What is the difference between TString and TClassString?

TClassString represents valid (but not necessarily known) class name, TLiteralClassString - a known class name and TString is your ordinary string containing some arbitrary bytes.

Also, why are 'TTrue' and 'TFalse' dedicated types in addition to TBool?

They are useful for signatures like this:

/** @return false|string false when string is empty, first char of the parameter otherwise */
function firstChar(string $s) { return empty($s) ? false : $s[0]; }

Here, the function may never return true, but if you had to replace false with bool, Psalm would have to consider true as a possible return value. With narrower type it's able to report meaningless code like this (https://psalm.dev/r/037291351d):

$first = firstChar("sdf");
if (true === $first) {
  echo "This is actually dead code";
}

I assume that of the classes in 'src/Psalm/Types/Atomic', only the ones prefixed 'T' are actually used to represent types at runtime?

No, all classes (but not traits) there are valid types.

[muglug]

No, all classes (but not traits) there are valid types.

yeah, and that’s what will change in v4 - all types will start with T

[robehickman]

@weirdan I guess I'm not clear on why TClassString and TLiteralClassString are needed in addition to TNamedObject. Does TNamedObject represent a class, or only an instance of that class?

Is 'ObjectLike' used for all associative arrays? They aren't always used as a struct/value object. I frequently use them to create an index of a sub-value of another structure, or as a mapping between a set of inputs and outputs.

[weirdan]

or only an instance of that class

These are for objects (maybe something else as well, given the generic name) as far as I'm aware. ClassStrings, on the other hand, are strings.

Is 'ObjectLike' used for all associative arrays?

Only where keys are known. So

$a = ["a" => 1, "b" => 2]; // is ObjectLike, array{a:int(1),b:int(2)}
// but
$a = [];
foreach (range(1,1) as $_) $a[(string)rand(0,1)] = rand(0,1); // is just array<string,int>

[robehickman]

Misc

TVoid.php
TNull.php

TMixed.php
TNonEmptyMixed.php
TIterable.php
TNever.php
TResource.php

Scalar supertype

Scalar.php
TScalar.php

Numeric supertype

TNumeric.php

Float

TFloat.php
TLiteralFloat.php

Int

TInt.php
TLiteralInt.php

Strings

TString.php
TLiteralString.php
TNumericString.php
TSingleLetter.php

Array

TArray.php
TNonEmptyArray.php
ObjectLike.php object like array
TArrayKey.php

Bool

TBool.php
TFalse.php
TTrue.php

Callables

TCallable.php
TCallableArray.php
TCallableObject.php
TCallableObjectLikeArray.php
TCallableString.php

Empty

TEmpty.php
TEmptyMixed.php
TEmptyScalar.php

Class

TClassString.php
TLiteralClassString.php
TScalarClassConstant.php

Function (including closures?)

Fn.php

Object

TObject.php
TGenericObject.php
TNamedObject.php
TObjectWithProperties.php #add support for object{foo:int, bar:string} annotation 3 months ago

Template

TTemplateParam.php
TTemplateParamClass.php

Special

THtmlEscapedString.php
TSqlSelectString.php Add special type for SQL select strings for plugins to consume

[robehickman]

@weirdan After looking through all of the types, the only one I'm still unsure of the function of is TNever.

[weirdan]

It's used as a return type for functions that never return (much like noreturn in Hack).

[robehickman]

@weirdan Thanks, I've added a note on that.

https://github.com/vimeo/psalm/pull/1549/files

[robehickman]

@muglug Thanks for merging, and the additional details added. I actually called the file 'plugins-type_system.md' so that it would sort to be close to the 'plugins.md' file. As you also renamed this file it no longer does. Would putting the two in a subdir make sense?

There are other things that I want to contribute to in the documentation regarding plugins, although doing so would involve digging and experimenting more deeply. I won't be able to do so before May.

[robehickman]

Also, I was planning to add a section to that page about reading the types that psalm attaches to the AST nodes.

[muglug]

Please feel free to add more info. Naming-wise they could probably benefit from being in a plugins subdirectory, yes.

[robehickman]

I was going to leave this open to discuss other things when I'm able to get to it. Although, if you get notifications from this when closed, that would work too. I didn't want to clutter your issue tracker with tons of threads relating to the docs.