Tidb: support restoring SQL text from a AST tree

[leoppro]

See: Proposal 中文教程

Our main work is in pingcap/parser/ast.
Most of ast.Nodes are in pingcap/parser/ast(Others are in pingcap/tidb/types).

We regard each ast.Node as a subtask, and we can complete one or several subtasks as a PR.
Considering that some ast.Node depends on another ast.Node, we divide sub-tasks into five stages.

Test

We believe that a Restore() function is correct when the SQL text it outputs can be parsed successfully
and the generated AST is equal to the original AST.

See: RunTest and RunRestoreTest

Some ast.Node can't be restored to a complete SQL text, such as an ast.ExprNode.
To test them, we can construct a complete SQL text.

See: expressions_test.go

There are examples which include complete implementation and test:
parser#71

Sub tasks list:

Stage One:

• [x] ast/expressions.go: ColumnName @leoppro pingcap/parser#63
• [x] ast/expressions.go: ColumnNameExpr @leoppro pingcap/parser#63
• [x] ast/expressions.go: UnaryOperationExpr @leoppro pingcap/parser#56
• [x] ast/expressions.go: IsNullExpr @lovewin99 pingcap/parser#72
• [x] ast/expressions.go: BetweenExpr @haplone pingcap/parser#71
• [x] ast/expressions.go: BinaryOperationExpr @haplone pingcap/parser#81
• [x] ast/expressions.go: WhenClause @haplone pingcap/parser#81
• [x] ast/expressions.go: CaseExpr @haplone pingcap/parser#81
• [x] ast/expressions.go: ParenthesesExpr @haplone pingcap/parser#81
• [x] ast/expressions.go: DefaultExpr @exialin pingcap/parser#84
• [x] ast/expressions.go: PatternLikeExpr @hanchuanchuan pingcap/parser#89
• [x] ast/expressions.go: IsTruthExpr @knarfeh pingcap/parser#90
• [x] ast/expressions.go: VariableExpr @exialin pingcap/parser#87
• [x] ast/expressions.go: ValuesExpr @exialin pingcap/parser#99
• [x] ast/expressions.go: PatternRegexpExpr @kangkaisen pingcap/parser#96
• [x] ast/expressions.go: MaxValueExpr @tony612 pingcap/parser#108
• [x] ast/expressions.go: RowExpr @o-oops pingcap/parser#113
• [x] ast/expressions.go: PatternInExpr @tony612 pingcap/parser#92
• [x] ast/expressions.go: PositionExpr @exialin pingcap/parser#125
• [x] ast/functions.go: FuncCallExpr @buzzers pingcap/parser#102
• [x] ast/functions.go: FuncCastExpr @buzzers pingcap/parser#102
• [x] ast/functions.go: AggregateFuncExpr @buzzers pingcap/parser#102
• [x] ast/functions.go: WindowFuncExpr @exialin pingcap/parser#143
• [x] ast/expressions.go: ExistsSubqueryExpr @SwanSpouse pingcap/parser#126
• [x] ast/expressions.go: SubqueryExpr @AndrewDi pingcap/parser#153
• [x] ast/expressions.go: CompareSubqueryExpr @AndrewDi pingcap/parser#153
• [x] tidb/types/parser_driver/value_expr.go: ValueExpr @leoppro pingcap/tidb#9302

Stage Two:

• [x] ast/ddl.go: DatabaseOption @leoppro pingcap/parser#56
• [x] ast/dml.go: TableName @feloxx pingcap/parser#73
• [x] ast/ddl.go: IndexOption @feloxx pingcap/parser#88
• [x] ast/dml.go: Limit @huachaohuang pingcap/parser#100
• [x] ast/ddl.go: TableToTable @tony612 pingcap/parser#109
• [x] ast/dml.go: WildCardField @exialin pingcap/parser#118
• [x] ast/dml.go: SelectField @exialin pingcap/parser#118
• [x] ast/dml.go: FieldList @exialin pingcap/parser#118
• [x] ast/ddl.go: IndexColName @feloxx pingcap/parser#91
• [x] ast/ddl.go: OnDeleteOpt @feloxx pingcap/parser#98
• [x] ast/ddl.go: OnUpdateOpt @feloxx pingcap/parser#98
• [x] ast/ddl.go: ReferenceDef @feloxx pingcap/parser#123
• [x] ast/ddl.go: Constraint @feloxx pingcap/parser#127
• [x] ast/ddl.go: ColumnOption @feloxx pingcap/parser#160
• [x] ast/ddl.go: ColumnDef @feloxx pingcap/parser#160
• [x] ast/ddl.go: ColumnPosition @tony612 pingcap/parser#110
• [x] ast/dml.go: Join @exialin pingcap/parser#111
• [x] ast/dml.go: DeleteTableList @wangbo pingcap/parser#117
• [x] ast/dml.go: TableSource @exialin pingcap/parser#111
• [x] ast/dml.go: ByItem @exialin pingcap/parser#125
• [x] ast/dml.go: GroupByClause @exialin pingcap/parser#125
• [x] ast/dml.go: HavingClause @SwanSpouse pingcap/parser#129
• [x] ast/dml.go: OrderByClause @SwanSpouse pingcap/parser#128
• [x] ast/dml.go: OnCondition @exialin pingcap/parser#132
• [x] ast/dml.go: TableRefsClause @exialin pingcap/parser#132
• [x] ast/dml.go: FrameBound @exialin pingcap/parser#135
• [x] ast/dml.go: Assignment @924060929 pingcap/parser#146
• [x] ast/dml.go: WindowSpec @exialin pingcap/parser#143
• [x] ast/dml.go: PartitionByClause @exialin pingcap/parser#143
• [x] ast/dml.go: FrameClause @exialin pingcap/parser#143
• [x] ast/dml.go: UnionSelectList @AndrewDi pingcap/parser#153

Stage Three:

• [x] ast/ddl.go: CreateDatabaseStmt @leoppro pingcap/parser#56
• [x] ast/ddl.go: DropDatabaseStmt @leoppro pingcap/parser#56
• [x] ast/ddl.go: DropIndexStmt @honestold3 pingcap/parser#83
• [x] ast/ddl.go: CreateTableStmt @feloxx pingcap/parser#170
• [x] ast/ddl.go: DropTableStmt @caohe pingcap/parser#107
• [x] ast/ddl.go: TruncateTableStmt @o-oops pingcap/parser#122
• [x] ast/ddl.go: RenameTableStmt @liutang123 pingcap/parser#114
• [x] ast/ddl.go: CreateViewStmt @AndrewDi pingcap/parser#139
• [x] ast/ddl.go: CreateIndexStmt @firekillice pingcap/parser#150
• [x] ast/ddl.go: AlterTableSpec @buzzers pingcap/parser#154
• [x] ast/ddl.go: AlterTableStmt @buzzers pingcap/parser#154

Stage Four:

• [x] ast/dml.go: InsertStmt @exialin pingcap/parser#168
• [x] ast/dml.go: SelectStmt @AndrewDi pingcap/parser#153
• [x] ast/dml.go: UnionStmt @AndrewDi pingcap/parser#153
• [x] ast/dml.go: ShowStmt @GunaYX pingcap/parser#178
• [x] ast/dml.go: LoadDataStmt @GunaYX pingcap/parser#184
• [x] ast/dml.go: DeleteStmt @arnkore pingcap/parser#189
• [x] ast/dml.go: UpdateStmt @arnkore pingcap/parser#190

Stage Five:

• [x] ast/misc.go: UseStmt @leoppro pingcap/parser#56
• [x] ast/stats.go: DropStatsStmt @TennyZhuang pingcap/parser#97
• [x] ast/misc.go: BeginStmt @shinytang6 pingcap/parser#144
• [x] ast/misc.go: CommitStmt @shinytang6 pingcap/parser#144
• [x] ast/misc.go: RollbackStmt @shinytang6 pingcap/parser#144
• [x] ast/stats.go: AnalyzeTableStmt @shinytang6 pingcap/parser#144
• [x] ast/stats.go: LoadStatsStmt @shinytang6 pingcap/parser#144
• [x] ast/misc.go: TraceStmt @GunaYX pingcap/parser#167
• [x] ast/misc.go: TableOptimizerHint @AndrewDi pingcap/parser#153
• [x] ast/misc.go: ExplainStmt @eiantee pingcap/parser#193
• [x] ast/misc.go: PrepareStmt @eiantee pingcap/parser#193
• [x] ast/misc.go: DeallocateStmt @eiantee pingcap/parser#193
• [x] ast/misc.go: ExecuteStmt @eiantee pingcap/parser#193
• [x] ast/misc.go: BinlogStmt @exialin pingcap/parser#196
• [x] ast/misc.go: VariableAssignment @exialin pingcap/parser#199
• [x] ast/misc.go: FlushStmt @exialin pingcap/parser#196
• [x] ast/misc.go: KillStmt @exialin pingcap/parser#196
• [x] ast/misc.go: SetStmt @exialin pingcap/parser#199
• [x] ast/misc.go: SetPwdStmt @exialin pingcap/parser#199
• [x] ast/misc.go: CreateUserStmt @exialin pingcap/parser#197
• [x] ast/misc.go: AlterUserStmt @exialin pingcap/parser#197
• [x] ast/misc.go: DropUserStmt @exialin pingcap/parser#197
• [x] ast/misc.go: DoStmt @exialin pingcap/parser#196
• [x] ast/misc.go: AdminStmt @exialin pingcap/parser#196
• [x] ast/misc.go: PrivElem @exialin pingcap/parser#198
• [x] ast/misc.go: RevokeStmt @exialin pingcap/parser#198
• [x] ast/misc.go: GrantStmt @exialin pingcap/parser#198

[AndrewDi]

Waiting this feature to rewriter view's sql...I think we could use create view func to test this feature later.

[AndrewDi]

@leoppro Do we need to considered sql_mode=ansi_quotes when process TableName\TableAliasName\ColumnName\ColumnAliasName ?

// escape the identifier for pretty-printing.
// For instance, the identifier "foo `bar`" will become "`foo ``bar```".
// The sqlMode controls whether to escape with backquotes (`) or double quotes
// (`"`) depending on whether mysql.ModeANSIQuotes is enabled.
func escape(cis model.CIStr, sqlMode mysql.SQLMode) string {
    var quote string
    if sqlMode&mysql.ModeANSIQuotes != 0 {
        quote = `"`
    } else {
        quote = "`"
    }
    return quote + strings.Replace(cis.O, quote, quote+quote, -1) + quote
}

[leoppro]

@leoppro Do we need to considered sql_mode=ansi_quotes when process TableName\TableAliasName\ColumnName\ColumnAliasName ?

// escape the identifier for pretty-printing.
// For instance, the identifier "foo `bar`" will become "`foo ``bar```".
// The sqlMode controls whether to escape with backquotes (`) or double quotes
// (`"`) depending on whether mysql.ModeANSIQuotes is enabled.
func escape(cis model.CIStr, sqlMode mysql.SQLMode) string {
  var quote string
  if sqlMode&mysql.ModeANSIQuotes != 0 {
      quote = `"`
  } else {
      quote = "`"
  }
  return quote + strings.Replace(cis.O, quote, quote+quote, -1) + quote
}

see: https://github.com/pingcap/parser/blob/master/ast/util.go#L189 @AndrewDi

[leoppro]

Many thanks for your contributions.
All the tasks have been completed.