Silverstripe-framework: Simple class rename helper

[sminnee]

In SilverStripe 4, a lot of classes are going to be renamed, as we introduce namespacing. Although a script that automatically fixes 100% of these is close enough to NP-hard to be discarded, a script that does the following will cover about 95% of the cases and make the refactoring work a lot less tedious.

I'd expect the script would do something like this:

• Maintain a list of old class => new class
• For parse each PHP files
• Looks for simple references to altered classnames: e.g., <literal>:: or new <literal>.
• For each occurrence:
  
  
  
  • Replaces the class name with the new one, e.g. StringField => DBString
  
  
  • Adds the relevant use statement to the top of the file, e.g. use SilverStripe\Model\FieldType\DBString;.
  
  

My hope is that we can rely on an PHP parser packages to do this. Presumably this will be a separate packagist package, as we wouldn't want to download all that code for every SilverStripe user.

• [x] Build tool - https://github.com/silverstripe/silverstripe-upgrader/
• [ ] Upgrading notes - https://github.com/silverstripe/silverstripe-framework/pull/5695

[sminnee]

Note: other common cases I've come across:

• instanceof checks
• assertInstanceOf calls with the class name given as a string literal
• docblock references to classnames

[camspiers]

I would have a look https://github.com/endel/galapagos which uses https://github.com/nikic/PHP-Parser

I have used these before and they are really great.

[camspiers]

I don't have time to make the whole transform, but here is a start. It isn't at all finished, but it allows you to pass in a map from old class to new class, and it handles class extension replacement, interface implements replacement and static call replacement. With a couple of hours of dev work, it could be made to introduce use statements (for cleaner code), and made to support other cases like ClassName::class etc.

<?php

use function galapagos\transform_with_visitors;

use PhpParser\Node;
use PhpParser\Node\Stmt;
use PhpParser\Node\Expr;

class RenameClass extends \PHPParser_NodeVisitorAbstract {
  protected $map;
  public function __construct($map) {
    $this->map = $map;
  }
  public function leaveNode(Node $node) {
    if ($node instanceof Stmt\Class_) {
      if ($node->extends !== null) {
        if (array_key_exists($node->extends->parts[0], $this->map)) {
          $node->extends->parts[0] = $this->map[$node->extends->parts[0]];
        }
      }

      if ($node->implements !== null) {
        foreach ($node->implements as $part) {
          if (array_key_exists($part->parts[0], $this->map)) {
            $part->parts[0] = $this->map[$part->parts[0]];
          }
        }
      }
    }

    if ($node instanceof Expr\StaticCall) {
      if (array_key_exists($node->class->parts[0], $this->map)) {
        $node->class->parts[0] = $this->map[$node->class->parts[0]];
      }
    }

    if ($node instanceof Expr\New_) {
      if (array_key_exists($node->class->parts[0], $this->map)) {
        $node->class->parts[0] = $this->map[$node->class->parts[0]];
      }
    }

    return $node;
  }
}


function run($files) {
  $visitors = [
    new \PhpParser\NodeVisitor\NameResolver(),
    new RenameClass([
      'ExampleSSField' => 'SilverStripe\Fields\ExampleSSField',
      'ExampleSSInterface' => 'SilverStripe\Interfaces\ExampleSSInterface',
    ]),
  ];

  foreach ($files as $file) {
    echo transform_with_visitors(
      file_get_contents($file),
      $visitors
    );
  }
}

which would take:

<?php

class ExampleField extends ExampleSSField implements ExampleSSInterface 
{
}

ExampleSSField::doSomething();
new ExampleSSField();

and output:

<?php

class ExampleField extends \SilverStripe\Fields\ExampleSSField implements \SilverStripe\Interfaces\ExampleSSInterface
{
}
\SilverStripe\Fields\ExampleSSField::doSomething();
new \SilverStripe\Fields\ExampleSSField();

[sminnee]

Thanks @camspiers! Do you know if PHP-Parser supports AST modification that have minimal changes to the underlying code when outputted? If, for example, we want to add a "use" statement at the top, I don't necessary want it to go and fiddle with all my indentation.

[sminnee]

PS: My preference would be to insert "use" statements, I think but I'm sure that can be left as an exercise for the reader. ;-)

[camspiers]

Unfortunately not. But we can make custom pretty printers etc, and we can make it so it doesn't edit files that it doesn't change.

[camspiers]

I agree on the use of use statements, hence:

"With a couple of hours of dev work, it could be made to introduce use statements (for cleaner code), and made to support other cases like ClassName::class etc."

I don't think it will take that long for an experienced dev to add that support, but it isn't as trivial as just rewriting the names.

I think a tool like this will be such a boon for module developers and larger SilverStripe using companies who have a lot of investment in SilverStripe. It is so awesome that you are keen on this work.

If someone really wants to dive deep on this work, it could be made as an extensible library of transforms that handle other more complex upgrade cases, both for SilverStripe 4 and the future. A tool like this, I suspect, will also make the community more amenable to small API breakages in minor versions.

[sminnee]

I would probably choose "more frequent major releases" over "small API
breakages in minor versions", but the principle remains the same: we can
change things without leaving people in the cold.

On Mon, 31 Aug 2015 at 12:54 Cam Spiers [email protected] wrote:

I agree on the use of use statements, hence:

"With a couple of hours of dev work, it could be made to introduce use
statements (for cleaner code), and made to support other cases like
ClassName::class etc."

I don't think it will take that long for an experienced dev to add that
support, but it isn't as trivial as just rewriting the names.

I think a tool like this will be such a boon for module developers and
larger SilverStripe using companies who have a lot of investment in
SilverStripe. It is so awesome that you are keen on this work.

If someone really wants to dive deep on this work, it could be made as an
extensible library of transforms that handle other more complex upgrade
cases, both for SilverStripe 4 and the future. A tool like this, I suspect,
will also make the community more amenable to small API breakages in minor
versions.

—
Reply to this email directly or view it on GitHub
https://github.com/silverstripe/silverstripe-framework/issues/4559#issuecomment-136225130
.

[sminnee]

I've started work on this in https://github.com/sminnee/silverstripe-upgrader/, and you can see the class rename function, based on @camspiers' sketch, here: https://github.com/sminnee/silverstripe-upgrader/commit/1fa0422ca669aac993c5e78394106c56d18b7d74

[sminnee]

OK, this is coming along nicely, and I've managed to refactor it to be minimally invasive - it doesn't edit any code that it doesn't understand.

A review of the initial code/commits of https://github.com/sminnee/silverstripe-upgrader would be appreciated if anyone gets a change.

I've hard-coded an upgrade rule into the command here:
https://github.com/sminnee/silverstripe-upgrader/blob/master/src/Console/UpgradeCommand.php#L57

You can see a sample of the changes it generates here:
https://github.com/sminnee/silverstripe-framework/commit/c67c4010a48bfbf8837a516ef6ee6d012832334c

[camspiers]

If you are looking for more inspiration for how the tool's interaction could work, have a look at:
https://github.com/facebook/codemod and https://github.com/facebook/jscodeshift

codemod is language agnostic and works by simple regexes. A nice feature is that it gives you a interactive chance to approve or ignore the changes that were made. Effectively allowing you to review them.

jscodeshift is for JavaScript codemods, but also has some really nice features we could take inspiration from.

This tool is really nice: https://astexplorer.net/

Switch the transform menu to 'jscodeshift' and you will get an extra input where you can write jscodeshift transforms which will transform the AST. Something similar for PHP would be so awesome.

I can have a look through some of the code you have written @sminnee. From what I have seen so far it looks like you've made great progress.

[sminnee]

Yeah the JS ecosystem seems to be a lot richer w.r.t. parsing & manipulation tools, but nikic's project is a great base! The only thing it would benefit from is being whitespace-aware.

An AST sandbox for PHP would be cool, but Sandboxing the arbitrary PHP would be a bit of a hassle. Do you know of any projects that would be useful as a base PHP code-pen?

[camspiers]

The only thing that I can think of is to make an offline local tool for the job, and wrap it up in an app. Something like this recently released project could be cool: https://github.com/gabrielrcouto/php-gui

[camspiers]

Although, thinking about it a little more, there isn't any particular reason why it can't be a webapp, and instead of having the AST transforms being written in PHP you write them in JavaScript. If you have a server that code can be sent to, not in order to run it, but instead simply to use nikic's project to turn the file into an PHP AST which you subsequently transform using JavaScript. I'm not sure this is worth working on, and is certainly outside of the scope of this task. Could be useful though.

[sminnee]

And instead of having the AST transforms being written in PHP you write them in JavaScript.

This sounds like a recipe for soup. :-/ I'll keep working PHP tools to transform PHP code via PHP ASTs. ;-)

[camspiers]

Good for people who know both languages I guess. But yeah, it was only a suggestion as not being able to run/test the transforms in a browser makes it harder to market the tool. Gone are the days where "go download this app" sounds inviting to people.

[sminnee]

Yeah, I think having a way of running a PHP sandbox (docker, maybe?) would probably be a better bet. If the goal is to showcase a tool that people would use "for real" after it's downloaded, then showcasing it with as a sub-optimal use-case (i.e. a mix of JS and PHP) seems counter productive.

[camspiers]

Yeah I know what you mean. People at Facebook regularly use the astexplorer tool to develop specific codemods. It's really great for development because you get instant feedback. The online tool isn't how you eventually do the codemod, but it is how you develop complex codemods.

https://www.youtube.com/watch?v=d0pOgY8__JM

[tractorcow]

I've built on the renamer at https://github.com/silverstripe/silverstripe-upgrader/pull/4 with support for applying namespaces to files (or whole directory trees).

[tractorcow]

I'll write up some docs for 4.0.0.md for how to use this tool.

[tractorcow]

Some basic docs at https://github.com/silverstripe/silverstripe-framework/pull/5695

[sminnee]

I think we can call this issue closed; the goal now is to use it!

[sminnee]

I've created a 0.0.1 release to close this off

 composer global require silverstripe/upgrader

[camspiers]

whoop whoop!