Dolphinscheduler: [QUESTION] The same task start twice(任务重复启动)

[wangchangliang1008]

Installation machine distribution(安装机器分布)
master: master1、master2
worker: worker1、worker2
alertServer: dolphin1
apiServer: dolphin1

Describe the question
I have created 200+ tasks to start at 1:00 a.m., and about 60 tasks will run repeatedly every day
After observation, master1 and master2 will start a workflow instance respectively, and the interval between the two instances is 1-5 seconds
(我创建了200+个任务在凌晨1点定时启动，每天大约会有60个任务重复运行
经过观察，master1和master2会分别会启动一个工作流实例，两个实例启动的间隔时间在1-5秒内)

Which version of DolphinScheduler:
-[1.2.0]

Additional context
Related log description(相关日志描述)
in master1/master2 ：dophinscheduler-master-2020-03-XX_XX.0.log
no error message
find one command id:10021,type:SCHEDULER
...
...
submit task : mytask state SUBMITED_SUCCESS complete, instance id 7867 state: RUNNING_EXEUTION

**Requirement or improvement
My problem is same as the Issue :#1601
How can I ensure that only one instance is generated for each task？THX！

[dailidong]

very valuable issue
do you think how to reproduce this problem？

[dailidong]

I review the relate code ， but i can't find the problem

I will try to find the cause, if you have something new to report, please leave a message

[wangchangliang1008]

Set two or more masters, open at least 400 instances at the same time, may be repeated.

[KEN-LJQ]

We also meet this problem, but can’t find any clue of this, we still digging into it. I think it may be a critical issue, because it make our data duplicate or lost since two process instance are running same time

[dailidong]

I noticed this problem, I will discssion it with developers in the community, and if you find any clue, please share, thx

[KEN-LJQ]

In our situation, the version is [1.1.0], and cluster has 2 master and 3 worker
We found that the duplicate process instance

1. Has two row in table t_tds_process_instance with different id
2. Those process are shedule by two different master, and in master log, we found this log each org.apache.dolphinscheduler.server.master.runner.MasterSchedulerThread:[114] - find one command: id: xxxx, type: SCHEDULER, the command id is different, we belive two command had generated
3. we also check the Quartz with table QRTZ_JOB_DETAILS, QRTZ_CRON_TRIGGER, QRTZ_TRIGGER, and only one record can found

So, we think the the problem may happen when the schedule command is generate, but master log can not provide any more information about the job quartz schedule.

If you need any more info, we are happy to share

[gabrywu]

The PR may help you https://github.com/apache/incubator-dolphinscheduler/pull/1866

[dailidong]

the issue: #1863 describe this problem, but nobody met this problem, so I forgot it

thanks for @gabrywu 's reminder
update ： PR #1866 maybe can solve this problem， you can try set :
org.quartz.jobStore.acquireTriggersWithinLock=true
in quartz.properties, looking forward your deep message, if you solve the issue, please reply the comment

[KEN-LJQ]

Thanks for your quick reply, we will try this configuration, and keep watching if it works

[KEN-LJQ]

Wonderful, It works, and process instance did not duplicate any more

[xingchun-chen]

if your problem was solved, please close the issue