Generator-jhipster: Issues with jhipster manual upgrade: .yo-rc.json not updated

[vishal423]

Overview of the issue

• [ ] Jhipster version is not getting updated in .yo-rc.json file after manual upgrade to master
• [x] Compilation errors after upgrade due to presence of ProfileInfoResource and associated test
• [x] Liquibase checksum issues after upgrade -- Not implemented
• [x] Client side build fails after upgrade due to old files

Motivation for or Use Case

Jhipster version should be updated to ensure correct upgrade path for future upgrades

Reproduce the error

1. Generate monolith application with Jhipster version 4.14.5.
2. Link "generator-jhipster" master codebase to new generated application.
3. Enter jhipster command to upgrade to newer version
4. Check jhipster version in .yo-rc.json. It still points to 4.14.5

Related issues

Suggest a Fix

JHipster Version(s)

v5.1.0

JHipster configuration

{
  "generator-jhipster": {
    "promptValues": {
      "packageName": "com.mycompany.myapp",
      "nativeLanguage": "en"
    },
    "jhipsterVersion": "4.14.5",
    "baseName": "rmono4",
    "packageName": "com.mycompany.myapp",
    "packageFolder": "com/mycompany/myapp",
    "serverPort": "8080",
    "authenticationType": "jwt",
    "cacheProvider": "ehcache",
    "enableHibernateCache": true,
    "websocket": false,
    "databaseType": "sql",
    "devDatabaseType": "h2Disk",
    "prodDatabaseType": "postgresql",
    "searchEngine": false,
    "messageBroker": false,
    "serviceDiscoveryType": false,
    "buildTool": "maven",
    "enableSocialSignIn": false,
    "enableSwaggerCodegen": false,
    "jwtSecretKey": "<secret>",
    "clientFramework": "angularX",
    "useSass": false,
    "clientPackageManager": "yarn",
    "applicationType": "monolith",
    "testFrameworks": [],
    "jhiPrefix": "jhi",
    "enableTranslation": true,
    "nativeLanguage": "en",
    "languages": [
      "en"
    ]
  }
}

Entity configuration(s) entityName.json files generated in the .jhipster directory

Browsers and Operating System

• [x] Checking this box is mandatory (this is just to show you read everything)

[vishal423]

I am also seeing compilation errors after manual upgrade:

/home/vishal/amono/src/main/java/com/mycompany/myapp/web/rest/ProfileInfoResource.java:[37,62] cannot find symbol
[ERROR] symbol: method getRibbon()
[ERROR] location: variable jHipsterProperties of type io.github.jhipster.config.JHipsterProperties

I don't see ribbon property (in application.yml) after upgrade, but, java code is still referring to it.

Edit: While looking into generator code, I couldn't find ProfileInfoResource class. If not required, this should be removed as part of upgrade

[vishal423]

I am also seeing Liquibase checksum issues after upgrade:

2018-08-10 20:36:19.681 ERROR 9373 --- [no-4-Executor-1] i.g.j.c.liquibase.AsyncSpringLiquibase : Liquibase could not start correctly, your
database is NOT ready: Validation Failed:
1 change sets check sum
config/liquibase/changelog/00000000000000_initial_schema.xml::00000000000001::jhipster was: 7:b24857ae96390484c3b88439ddbb0906 but i
s now: 7:8966e014f629e8066e3cb3de05a568e1

liquibase.exception.ValidationFailedException: Validation Failed:

I do see changes in generated change-set like removal of indexes (idx_user_login, idx_user_email etc) and addition of addNotNullConstraint.

To avoid such issues during upgrade, this change-set should contain attribute runOnChange="true" and provide rollback of used refactorings. Also, it would be clean if initial schema change-set is split into smaller change-sets.

[gmarziou]

I would not use runOnChange="true" because it could break subsequent changesets hand written by user.
Personally I prefer Liquibase to fail on checksum changes because that's the least surprising option.

[DanielFran]

@vishal423 ProfileInfoResource and Ribbon properties have been removed in v5.

See #7050.

[vishal423]

On further looking into existing change-set, runOnChange="true" wouldn't solve issue in current scenario unless accompanied by corresponding drop refactorings which is not what I want as that would result into data loss.

At moment, I see following as one way to resolve this:

• Move all changes done as part of 5.x into separate refactorings contained in separate change-sets to ensure checksum of last Major release is respected and current release changes can be applied on top of that. This is standard way to upgrade database using Liquibase. However flip side would be that the fresh installations would also contain it as separate change-sets which may not be acceptable.

I don't have much understanding about upgrade sub-generator, however, if it's possible to conditionally add/update refactorings based on if it's upgrade or fresh, then, it might resolve it.

Also, I believe upgrade sub-generator should handle auto removal of removed files (like ProfileInfoResource) during upgrade.

[gmarziou]

The upgrade sub generator relies only on git merging capabilities and the app generator itself. So if the generator removes obsolete files, they should be removed by git too unless it uses wrong merging strategy or the user has inserted custom code into these files. I wonder whether the upgrade generator could benefit from using git merge -s recursive -Xours which is what I use to keep custom code when re-generating entities.

[vishal423]

@gmarziou, I noted issue with manual upgrade and not sure if git merge would come into play. At moment, it seems we missed removing ProfileInfoResource and it's integration test unless someone else think otherwise and for clean upgrade experience, this should be fixed.

[gmarziou]

Ideally, we could have upgrade tests in Travis.

[ruddell]

@vishal423 You can add ProfileInfoResource to cleanup.js to remove in newer versions

The v4->v5 upgrade had several breaking changes in the Angular layout so I don't think a manual upgrade is possible without more manual changes. The upgrade subgenerator should handle both of these issues (example).

You're right that the .yo-rc.json isn't updated, it also isn't changed with the upgrade subgenerator

[PierreBesson]

@vishal423 If this is fixed for you, can you close the issue ? Otherwise can you explicit what remains to be done ?

[vishal423]

@PierreBesson, we still have 2 issues to be fixed. I have updated ticket with what's fixed at moment.

[vishal423]

FYI: I was able to fix Liquibase checksum issues with custom change sets like below on top of 4.14.5 that I used before.

    <changeSet id="00000000000002" author="jhipster">
        <modifyDataType columnName="email"
                        newDataType="varchar(254)"
                        tableName="jhi_user"/>
    </changeSet>
    <changeSet id="00000000000003" author="jhipster" >
        <dropIndex  indexName="idx_user_login"
                    tableName="jhi_user"/>
    </changeSet>
    <changeSet id="00000000000004" author="jhipster" >
        <dropIndex  indexName="idx_user_email"
                    tableName="jhi_user"/>
    </changeSet>
    <changeSet id="00000000000005" author="jhipster" >
        <addNotNullConstraint   columnName="password_hash"
                                columnDataType="varchar(60)"
                                tableName="jhi_user"/>
    </changeSet>

Re-writing change set for all changes done post 4.14.x would be much time consuming. I am closing this point, but, would love to have clean migration support.

[jdubois]

@vishal423 yes your changelog looks good, and I should have done it for helping people migrate, but I just didn't have the time.... it's like many other things on JHipster, I'm just lacking the time to do everything. I'm doing a lot of this on my free time, and I have a lot of other personal do to, so typically annoying stuff like this isn't done...

[jdubois]

I'm closing this as you seem to have everything fixed.
We need to better document the Liquibase upgrade part, and I'm not sure this can be automated (this depends on what the user has done in his current Liquibase changelogs).
So I've added your changelog to our 5.0 release notes at https://www.jhipster.tech/2018/06/20/jhipster-release-5.0.0.html

[jdubois]

Oh and thanks a lot @vishal423 for the changelog!!

[vishal423]

@jdubois , can you re-open this as one issue is still pending and I don't have clue on why that's broken.

Jhipster version is not getting updated in .yo-rc.json file after manual upgrade to master

[pascalgrimaud]

Looking at this PR https://github.com/jhipster/generator-jhipster/pull/7903 from @ruddell
I give a fast try, and change the phase of the function saveConfig() to be in writing phase (instead of prompting or configuring) -> it seems to work

If someone (better with Yeoman/NodeJS than me) can give a try...