Teammates: Data Migration: FeedbackSession timeZone, all time fields, is*EmailEnabled and feedbackSessionType

[xpdavid]

Data migration initiated by issue #8448

• [x] Merged code that work for both the old and new versions of the data schema.
• [x] Released a version containing the above in V6.5.0.
• [x] Created a client script to migrate all data following the old schema to the new schema in PR #9081 and #8759.
• [x] Ran the script against the live site.
• [x] Removed the code that is specifically tailored for the old data schema in PR #9118.

[xpdavid]

@damithc Kindly find a suitable time to run DataMigrationForSessions.java in the master branch.

The risk level should be considered as low since we are just loading data and saving it back.

[damithc]

I have started running the script. Will take a few days to complete. So far no affected entities reported.

[damithc]

Script run completed in preview mode. No affected entities reported. Is that possible?

[xpdavid]

Not possible, probably some bugs in the script. XD

[damithc]

Also, the number of entities reported in the two runs (about 22K) is lower than the total entities reported by GAE (33K)

[xpdavid]

Also, the number of entities reported in the two runs (about 22K) is lower than the total entities reported by GAE (33K)

So the difference is about 10K? The cursor implementation in GAE doens't guarantee to scan all entities if entities are added during the scanning process. This should be fine as the newly added data is already in new format and thus not migration is need. But 10K is kind of too big.

[damithc]

Yes, it's unlikely 10k entities were added in one day :-) I can try to run the script again to confirm there is indeed a shortfall.

[xpdavid]

I can try to run the script again to confirm there is indeed a shortfall.

Can we try this after I fix the bug in the script?

[damithc]

Sure.
On a related note, may be there should be a way to feed the previous entity count to the script at the point of resumption so that we get an accurate entity count at the end?

[xpdavid]

@damithc Can try DataMigrationForSessions now in master.

In addition, you can fill

        numberOfScannedKey.set(0L);
        numberOfAffectedEntities.set(0L);
        numberOfUpdatedEntities.set(0L);

with the entity count in the last stopped position.

[damithc]

Started running it. Already the results are different from the previous run:

Number Of Entity Key Scanned: 300
Number Of Entity affected: 226
Number Of Entity updated: 0

[damithc]

Preview Completed!
Total number of FeedbackSessions: 33171
Number of affected FeedbackSessions: 29333
Number of updated FeedbackSessions: 0

[xpdavid]

Seems like the number matches 🎉

We can go ahead with the preview mode off 💯

[damithc]

Done (resumed twice due to timeouts):

Total number of FeedbackSessions: 33244
Number of affected FeedbackSessions: 29295
Number of updated FeedbackSessions: 29295