Alpaka: Prepare release 0.4.0

[BenjaminW3]

Tasks:

• [x] create release draft text
• [x] delete master branch. It is practically impossible to merge the current develop to master due to lots of incompatible changes to master (which did not incorporate the renamings). The release tags will keep alive the code that is linked in the existing releases (they do not link branches) and we can always create real branches from those tags if required later.
• [x] wait for #780 (CUDA 10.1 support)
• [x] wait for #755
• [x] The queue is named: QueueOmpCollective

• [x] solve #842
• [x] merge #844

[BenjaminW3]

@ComputationalRadiationPhysics/alpaka-maintainers Does anyone want to add something to the release that is not yet merged? Else I would not want to wait any further but create a release. Is there anything else we should wait for until we can create the release? What is the timeline for a test in PIConGPU, if this is necessary at all?

[psychocoderHPC]

I was for longer out of the office. Please give me at least two weeks for updating. Never the less you can open the release branch so I can use this branch as base for testing.

Am 15. April 2019 17:21:08 MESZ schrieb Benjamin Worpitz [e-mail]:

@ComputationalRadiationPhysics/alpaka-maintainers Does anyone want to
add something to the release that is not yet merged? Else I would not
want to wait any further but create a release. Is there anything else
we should wait for until we can create the release? What is the
timeline for a test in PIConGPU, if this is necessary at all?

--
You are receiving this because you are on a team that was mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/ComputationalRadiationPhysics/alpaka/issues/778#issuecomment-483296881

[BenjaminW3]

I am not sure if we need a release branch right now because I would simply create the release from the develop branch at the time we fixed all issues.
Going forward, I do not want to create such big release branches deviating from the main development branch as it was the case for the 0.3 release series. This created a lot of additional work. I would only create a bugfix release for severe bugs but all new features should be added to develop only and we should simply have releases more often.
Does anyone have an idea for naming a possible release branche? They should not conflict with the tag names (0.1.0, ..., 0.4.0).
Ideas:

• release/0.4.0 (some git UIs allow to show branches in a folder view where / is used as path separator)
• release-0.4.0
• alpaka-0.4.0 (I do not really see a advantage in using the project name here and additionally it does not make clear that it is a release)
• ...

My preferences are from top to bottom.
Other ideas?

[psychocoderHPC]

One PR we definitive need for the next release is: #755

@tdd11235813 will finish this PR today so that we can later cherry pick it to the release branch

[BenjaminW3]

All the necessary tickets for release-0.4.0 have been resolved. Is there anything left we have to wait for?

[psychocoderHPC]

Yes we will i introduce a new queue to fit the requirements of dlr
I was short in time and was not able to do it last week.

• [ ] The queue is named: QueueOmpCollective

I also need to rename the queues next week to blocking and nonblockming.

[psychocoderHPC]

I also still sitting on the verification that PIConGPU is working with the next alpaka release.

• [x] https://github.com/ComputationalRadiationPhysics/picongpu/pull/2958

[psychocoderHPC]

@BenjaminW3 I reopened this issue to not forget to add the special queue QueueOmpCollective. I extended the description with a checkpoint

[BenjaminW3]

@psychocoderHPC How is the QueueOmpCollective advancing?

[psychocoderHPC]

I am currently analyse a critical bug in PIConGPU which is triggered by pinning memory with cuda/alpaka. I will link the pic issue tomorrow here.
This has currently my absolute priority since it crashes applications.

Am 15. Juli 2019 19:39:26 MESZ schrieb Benjamin Worpitz [e-mail]:

@psychocoderHPC How is the QueueOmpCollective advancing?

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/ComputationalRadiationPhysics/alpaka/issues/778#issuecomment-511498985

[psychocoderHPC]

@BenjaminW3 https://github.com/ComputationalRadiationPhysics/picongpu/issues/2996 is the current issue. I am currently working on a mini example to reproduce it with plain CUDA that I can report it.

[ax3l]

btw, reading the edited PR description:

delete master branch. It is practically impossible ... -> oh no, just do: git merge --strategy-option=theirs dev to accept the PR on the command line ;)
https://stackoverflow.com/questions/173919/is-there-a-theirs-version-of-git-merge-s-ours

[BenjaminW3]

Now that we are nearly done with the release I would like to propose something for the next version (probably 0.5.0):

• raise the minimum requirement to c++14 (std::integer_sequence, auto return type deduction, ...)
  
  
  
  • drop gcc 4.9 (misses some c++14 features)
  
  
  • drop CUDA 8.0 (no official c++14 support)
  
  
  • finally drop support for ubuntu 14.04 (already end of life)
  
  
  • set boost-1.65.1 as minimum dependency (older versions do not work consistently on all supported platforms)
  
  

Do you see any problems with this or do you want to drop some more versions?

[ax3l]

I fully support this and am not aware of HPC systems in our daily use that do not have C++14 support by now (aka: have a new-enough NVCC...). PIConGPU can be transitioned to C++14 and newer as well with the next release.
Your suggested drops sound reasonable to me.

Any objections from DLR (@q-p, @krzikalla et al.) or CMS (@fwyzard et al.)?

[fwyzard]

Thanks for checking: CMS is already using nvcc from CUDA 10.1 in c++14 mode.

[sbastrakov]

Looks reasonable to me as well.

[psychocoderHPC]

I also agree with going to C++14 for the 0.5.0 release.

[ax3l]

Excellent, @SimeonEhrig just confirmed Cling will also migrate to a > CUDA 8 code base via an LLVM/Clang upgrade end of the year.

[BenjaminW3]

Are there any topics left for the 0.4.0 release except the CMake 3.15 support?

[BenjaminW3]

@psychocoderHPC @tdd11235813 @sbastrakov Now again all open issues for the 0.4.0 release are solved. Do you want to add anything else or do a test with PIConGPU? Else I would like to tag the release.

[sbastrakov]

@BenjaminW3 yes, let me test the current alpaka dev with PIConGPU. Edit: I do not expect issues, but to be sure.

[sbastrakov]

I've started with the tests, will continue tomorrow. cupla needs to be updated a little (for now I did it locally). Otherwise did not encounter issues so far, but did not test CPU yet, as Hemera is fully occupied rn.

[psychocoderHPC]

besides the tests we have two additional tasks todo:

• check that cudaRtCheck always reset the CUDA error, else it is not possible to catch the alpaka exceptions in the user code (I will check it and if necessary fix it, today) #844
• write documentation which renamings are needed to go from 0.3.X to 0.4.0

update: cross already existing feature

[BenjaminW3]

The necessary renaming are already part of the release documentation.

[psychocoderHPC]

THx @sbastrakov also mentioned it. It is not obvious^^ and only visible for maintainer but ok for now

[psychocoderHPC]

I extended the list in the initial post. Some users (DLR) require that it is possible to compile without warnings: #842

[sbastrakov]

My tests with PIConGPU passed, as expected.

[BenjaminW3]

@sbastrakov @psychocoderHPC Is there still anything left that absolutely needs to be part of alpaka-0.4.0 or can we create the release?
I know that there will always be new cool features, more tests, better compiler support etc. but at some point we will have to create a release. The less we release the worse it gets for our users to updgrade. If there is something missing we can release 0.4.1 or 0.5.0 easily.
If we create new releases fast enough then there is no reason for patch releases and backports.

[psychocoderHPC]

873 is imo the last strongly required peace.

One think would be also very important is that we can run all examples wit any accelerator bcause currently none of our examples run on different hardware. For that a trait to get a queue for an accelerator based on a type e.g. blocking() or nonBlocking() is required. It is not much work but have not find time for it.

[BenjaminW3]

Ok, #873 should be part of the release. Any time line?
The changes to the examples should be completely independent of any release.

[psychocoderHPC]

The requirement for the example changes coming from our ongoing activities with different vendors.

[BenjaminW3]

But nothing prevents us from releasing a 0.5.0 a week later with the changes to the examples.
The known bug on the other hand is really an issue that should not be released at all.

[psychocoderHPC]

Currently none of the vendors is using alpaka direct because of the missing examples. We need to point everyone to CUPLA.
The work for the missing trait is quite small and should be not more than 1 to 2 days.

I will discuss it next week.

[sbastrakov]

Agree @psychocoderHPC , I think it's important this trait and proper accelerator choice in example thing is done.

[BenjaminW3]

Yes I agree that this is an important topic but nothing prevents us from creating a new release after 0.4.0 once this is implemented.
I do not think that this is worth a discussion at all. A release is nothing special. If we go for really long release cycles and not release often then releases should be time-driven like in boost, llvm and c++. Releases should not wait for features, only for blocking bugs.

[psychocoderHPC]

After #897 #896 #894 #892 is in merged I will move out a release branch that we can test it without pulling in additional feature PRs.

[BenjaminW3]

As all the PRs have been merged, I branched off release-0.4.0!

[BenjaminW3]

@psychocoderHPC @sbastrakov @tdd11235813 Could you please test this with PIConGPU before I officially release it?

[sbastrakov]

Good point @BenjaminW3 . @psychocoderHPC in case you don't have time I could do it later today, after coming back.

[psychocoderHPC]

@BenjaminW3 Thanks for opening the release branch (feature freeze).
I can not test it before next year also we can not expect any feedback from DLR so we should do our testing in the first two weeks of Jan 2020 with a focus to release in the 3rd week of Jan.

[psychocoderHPC]

@krzikalla could you please check/test the release branch of version 0.4.0

[psychocoderHPC]

PIConGPU is working (tested KHI example) with the release candidate 0.4.0 of alpaka

[BenjaminW3]

Thanks for verifying this! Can I create the release now or should we wait for something else?

[psychocoderHPC]

please let us wait that @krzikalla has a chance to verify the changes. I will call him next week via phone. My focus is to release 3rd week of Jan. and to push the bug fix release 0.3.6 next week first so that 0.4.0 is then on the release pool position^^

[psychocoderHPC]

@krzikalla is not able to test the release branch for 0.4.0 in time. This means that we can release this week.

[BenjaminW3]

@psychocoderHPC Is there anything that prevents me from creating a release now?

[psychocoderHPC]

@BenjaminW3 just fond a few min ago a bug. Please DO NOT release. I am currently testing the fix.

[psychocoderHPC]

@BenjaminW3 The bug I thought is not existing in alpaka. Only HIP memcpy 3D is broken but it is not our fault.

Please feel free to press the release button.

Note: we need to check zenodo, I saw that the the wrongly release 0.4.0 last time was registered there. We need to be sure that zenodo is correct.

[BenjaminW3]

Done!

[psychocoderHPC]

thanks I will announce it tomorrow on our mailing list

[ax3l]

Congratulations on the release everyone! :)