Generator-jhipster: Mapping on an existing database - feature request

[AdrienHorgnies]

EDIT : This issue was concluded by the creation of a JHipster module.

Overview of the issue

Currently, it is not possible using an existing database without prior modifications because the naming strategy makes it impossible matching database and ORM's names. This is especially a problem for databases without naming convention which cannot easily be refactored, neither the current services using them.
A feature I'm currently investing is modifying the entity generator so it keeps table and column's names as they're specified by the user.

Motivation for or Use Case

Such a modification would enable JHipster to use existing databases with naming conventions different from the ones used by Spring. Afterwards we could investigate generating entities from an existing database schema.

Reproduce the error

yo jhipster:entity foobar --table-name FooBar_SUFFIX
will produce the ORM @column( name="foo_bar_suffix") but we need @column( name="FooBar_SUFFIX").

Related issues

None

Suggest a Fix

1. The table-name option should use exactly the user input without processing it. The behaviour of this option should change.
2. I removed the processing and changed the naming strategy for :

implicit-strategy: org.hibernate.boot.model.naming.ImplicitNamingStrategyJpaCompliantImpl
physical-strategy: org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl

However we don't know the application well enough to avoid side effects and would need your advice on how to bring our modifications.

The feature we suggest is to enable the user to choose the wanted behaviour. If he does, we change the naming strategy, we set a global option inside a configuration json file and the entity generator would take account of it. It makes mapping on an existing database easier.

• [X] Checking this box is mandatory (this is just to show you read everything)

[jdubois]

There have been many people who have tried to do it, without success. So I understand the business value, but beware, this is really hard to do.

[jdubois]

-> you should try to do this as a module, and if you have something that partially work, we will try to help you.

In the meantime, we currently have too many opened tickets, and the goal of JHipster isn't to work on existing databases, so I'm closing this as this isn't our current priority.

[bastienmichaux]

Hello @jdubois, @AdrienHorgnies and me will work on this module then. We're interns under the supervision of Grégory Schiano at Altissia International. Thanks for your help !

[jdubois]

Oh sorry!!!!!! Then you're with the right people, this is the best solution
I've seen for this.
Don't hesitate to discuss with us!

Le 23 mars 2017 7:28 PM, "Bastien Michaux" [e-mail] a
écrit :

Hello @jdubois https://github.com/jdubois, @AdrienHorgnies
https://github.com/AdrienHorgnies and me will work on this module then.
We're interns under the supervision of Grégory Schiano at Altissia
International. Thanks for your help !

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jhipster/generator-jhipster/issues/5479#issuecomment-288817863,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AATVo4R-oB3IG_UG-jrJdtdQw_XSplZ4ks5rornZgaJpZM4Mm9a1
.

[pascalgrimaud]

@bastienmichaux : for your information, a new version of generator-jhipster-module should be available soon, with Yeoman 1.0 (see
https://github.com/jhipster/generator-jhipster-module/pull/15)

[pascalgrimaud]

@amrnablus @bastienmichaux : for your information, the new generator-jhipster-module has been released! https://www.npmjs.com/package/generator-jhipster-module

[jdubois]

Thanks Pascal ! Yes that's what you should use, I'll update the "fortune" module with it

[AdrienHorgnies]

I have a few questions about how our module should do things.
The current obstacles are that :

1. the entity generator processes user input.
2. the entity generator prevents the user from giving certain input. You can't enter a capital letter to begin a fieldName !
3. the entity generator uses a template which uses already processed data (<%-fieldNameUnderscored %>at the moment but you changed that post release 4.1.1).
4. The naming strategy used by hibernate/spring is applied after all that.

We can :

• modify the behavior of the entity generator (through search and replace)
• modify the generated .json for the newly created entity and regenerate the entity; use the entityConfig object in a post hook, retrieve wanted values, inject it into the .jhipster/entityName.json and finally regenerating the entity. Template will still mess with the field names thou.
• modify the template through search and replace so it uses another variables.
• clone the entity generator and adapt its behavior (thus our module would be an alternative entity generator).

Each action has its pro's and con's but except for the clone, we can't build a complete solution based on a single one. Any opinion on this ?

[jdubois]

I'm trying to answer your questions, but first of all you need to know we just migrated to Yeoman 1.0, and that broke quite a number of APIs. So simple APIs should work, but if you have trouble (like a variable missing), don't hesitate to ping us, that wouldn't be very surprising. The new API isn't very complex, it's mostly a refactoring, so this shouldn't be too annoying.

Yes there are rules in what the generator accepts. For you example: field names begin with a lower case letter, because that's how it works in Java... If you want something different, you're going to have to add a specific Hibernate mapping (with "@Column").

I think that for a first version, you should not worry about all those options, there are things that will be very difficult to "reverse engineer" into JHipster, but when you know JHipster better you will know more tricks to solve those issues.

• Cloning the generator isn't a good idea: you will then need to merge our changes, and you'll see we do a lot of them. So in the end you'll have a very hard time merging everything.
• Trying to do the best .jhipster/entityName.json is a very good thing: that's the format we use, so as long as you generate those, no problem at all. That's typically what the JDL does, you could have a look at what they do.
• Once an entity is generated, you can then tune it using a module: for example you could add the "@Column" annotations. It might be a bit complex, but your code would be decoupled from the generator, so wouldn't have any merges to do - basically that's a "search and replace", which should be good enough for most cases, but there are more advanced options (you could even use a Java parser, but I haven't found a good one yet). For example, that's what we do with the internationalization - we just go through all the files and remove the internationalization code.

[AdrienHorgnies]

Thank you for the helpful answer.

We're already using the new version of JHipster module, and thus Yeoman 1.0.

I made some tests to see how to tweak the field validation and it is indeed a bad idea. After digging in a bit more, we understood that we don't need to change all that and we think we can build something neat. We'll use both post app hook and post entity hook :

1. We'll replace naming scheme in post app hook.
2. In the post entity hook
   
   
   
   1. We'll introduce a new question and a new .jhipster/Entity.json field to get a column name without so many restrictions
   
   
   2. We'll search and replace relevant values, with the configuration we're using atm, the @Column annotation is already generated and we'll see for other values afterwards.
   
   

As we're looking for literal values, in precise files and with strong context, we should be good.
I didn't find a 'search and replace' tool in jhipster dependencies, did I miss it ? I use the npm package 'replace' at the moment.

Also I forgot to ask you : Is the entity generator option 'table-name' doing what it's meant to do ?
The documentation say :
--table-name <table_name> - By default JHipster will generate a table name based on your entity name, if you would like to have a different table name you can do so by passing this option.
But in the code we have :

this.entityTableName = this.getTableName(this.options['table-name'] || this.name);

So the option parameter is processed and to my sense, the option is rendered pointless. If we change this line to :

this.entityTableName = this.options['table-name'] || this.getTableName(this.name);

That would make more sense. Of course you must change the naming strategy to use the option in this state. But the option would be useful.

[bastienmichaux]

Hello, and thank you @jdubois for your help.

I covered the table-name option in this document, because it does strange things.

[deepu105]

You could also check the entity-audit module and react module for some tricks. Using our entity hooks you can invoke your module automatically each time an entity is created.

[deepu105]

@bastienmichaux the names are generated as such due to the naming strategy we follow which is form spring/hibernate

[bastienmichaux]

@deepu105 I understand. I thought that this option was made specifically to bypass the naming strategy.

[AdrienHorgnies]

jdubois modified the milestone: 4.2.0 7 hours ago

What does it mean ?

So we begin to have something working. It's messy and not tested but we wanted to produce results asap. We'll tidy things after having a review from our seniors and we're already working on unit tests.

We do have a problem we'd like to get help onto. It seems that the entity generator runs install() simultaneously with afterRunHook(), causing ouput from the two methods to collide. It is because install() uses spawnCommand which is asynchronous.

/**
 * Normalize a command across OS and spawn it (asynchronously).
 *
 * @param {String} command
 * @param {Array} args
 * @param {object} [opt]
 */
spawnCommand.spawnCommand = function (command, args, opt) {
  opt = opt || {};
  return spawn(command, args, _.defaults(opt, {stdio: 'inherit'}));
};

Is there a way to wait it's done so our outputs are not intertwined ?

Also, the entity generator gives its hooks a nice variable containing lots of useful information. However this variable doesn't contain the user input (for table name, field names, etc...). Would you consider accepting a PR adding these information ? I think it would make sense to give these information to the post hooks. For our part, we'd like to use these information as default input, at the moment we use capitalized values which is the closest we found.

Thanks

[PierreBesson]

@AdrienHorgnies Don't worry about the "milestone" tag, this is because @jdubois is preparing a release so he uses the "select all closed issues" to tag them for the release.

[jdubois]

Yes it's just me preparing the release, don't worry! That just shows when it was closed, but for you it's different - btw we need a way to discuss with you, it's not something we have already done. In the meantime, don't hesitate to comment here, or even create another ticket.

[deepu105]

@AdrienHorgnies it would be difficult make it synchronous as the install task is handled by yeoman. But can I ask you what is exact thing you are trying to do with the hook? I have used the hook in the entity-audit module can you check if you are doing it in a similar way?

[deepu105]

@AdrienHorgnies about the PR yes ofcourse we will accept it.

[AdrienHorgnies]

We're retrieving information about the entity just created in the post entity hook that the user can accept as defaults. We're offering the user to input other data as table, columns and joins's name, inputs that won't be processed. So we ask the user which value we should use but the questions happen while other outputs print from entity generator install().

I've already planned to look (again) at your module :-D.

[AdrienHorgnies]

@deepu105 Your module suffers from the same problem :

I'm updating the entity for audit PUCK
 conflict .jhipster/PUCK.json
? Overwrite .jhipster/PUCK.json? (Ynaxdh) [09:37:55] Using gulpfile ~/JHipster/testAudit/gulpfile.js
[09:37:55] Starting 'inject:app'...
[09:37:55] gulp-inject 126 files into index.html.
[09:37:55] Finished 'inject:app' after 215 ms
? Overwrite .jhipster/PUCK.json? (Ynaxdh) 

[deepu105]

May be the behaviour changed with yeoman 1.x I never noticed this before. Can you open a new issue for this. I'll take a look when I have time. In the mean time if you find a fix let me know

[deepu105]

other than the UI glitch does it cause any other issue?

[AdrienHorgnies]

Well, I don't exactly know what install() does but I guess that if you try to use some files this task modify, you'll have unpredictable results.

I'll open the issue then.

[AdrienHorgnies]

We have something working :-D.

We'll finish pending tasks and then we'll begin to advertise our module to get some attention. We should be able to make a beta release next week !

I didn't look into the UI glitch... yet #5548

[jdubois]

Congrats :-)
Sorry I'm not very active at the moment (on holidays!!), if you do this next week don't forget to mention @java_hipster on Twitter - I'll retweet you, of course.