kakoune 🚀 - Tests not passed with default Fedora build flags

I think the failed tests deliberately leave a coredump behind, so the failure can be analysed, a backtrace generated, etc. but all those coredumps are presumably cleaned away by the Fedora build machines. Can you reproduce the test failures on your own machine? If so, can you find the coredumps and generate backtraces?

Screwtapello on 24 Nov 2019

Nightly builds work fine, right? If a new official release were to be made, it would work and you could push that new version into mainstream Fedora? cc @mawww

lenormf on 24 Nov 2019

Nightly builds work fine, right?

Will try this very soon.

Can you reproduce the test failures on your own machine? If so, can you find the coredumps and generate backtraces?

Will try this as well. Also i should clarify sorry, without %set_build_flags all tests passed. With %set_build_flags:

Summary: 403 tests, 8 failures

tim77 on 24 Nov 2019

Actually, is this issue somehow a duplicate of #2165 ? Looks like builds on Fedora are already happening. @jkonecny12

lenormf on 24 Nov 2019

Actually, is this issue somehow a duplicate of #2165 ? Looks like builds on Fedora are already happening.

Not really. My build is based on @jkonecny12 and i saved full change log of course. All what i want is fix some packaging issues and push Kakoune in official Fedora repos, not Copr. And i happy to add @jkonecny12 as co-maintainer if he wish with full permissions.

tim77 on 24 Nov 2019

Thanks, I wasn't familiar with the differences there.

lenormf on 24 Nov 2019

Tried with latest version from master and same thing.

Here is a list of this 8 tests which failed if we build Kakoune with default build flags:

complex-pipe
inline-sort
pipe-at-eof
pipe
pipe-to
reload
0-crash-on-pipe-with-selection-access
1504-assertion-on-incorrect-pipe-use

Kakoune returned error 134
Summary: 414 tests, 8 failures

Maybe we can safely just suppress this 8 tests? Or this should been fixed?

tim77 on 25 Nov 2019

All what i want is fix some packaging issues and push Kakoune in official Fedora repos, not Copr

This is unofficially the most awkwardly build workaround I've read in a long time.

But then again, I have no idea how a build of a package it's handled on the fedora projectt... You clearly said though that this is not going to Copr but the official repos

It did built apparently without an issue and according to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/build-01116657.log and https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/build-01116985.log

Why is there a need to change anything?

According to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/01116985-kakoune/builder-live.log.gz there were no problems

And according to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/01116667-kakoune/builder-live.log.gz there were no problems either

https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/

nonumeros on 25 Nov 2019

This is unofficially the most awkwardly build workaround I've read in a long time.

What exactly? %set_build_flags macros in Fedora and Red Hat Enterprise Linux?

According to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/01116985-kakoune/builder-live.log.gz there were no problems

And according to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/01116667-kakoune/builder-live.log.gz there were no problems either

Because this builds without %set_build_flags macros. Did you read this?

Why is there a need to change anything?

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags

tim77 on 25 Nov 2019

What exactly? %set_build_flags macros in Fedora and Red Hat Enterprise Linux?

well, you should direct that question to @jkonecny12, not me

According to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/01116985-kakoune/builder-live.log.gz there were no problems
And according to https://copr-be.cloud.fedoraproject.org/results/atim/kakoune/fedora-31-x86_64/01116667-kakoune/builder-live.log.gz there were no problems either

Because this builds without %set_build_flags macros. Did you read this?

What's the last successful build log with the %set_build_flags then?

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags

thanks

nonumeros on 25 Nov 2019

What's the last successful build log with the %set_build_flags then?

There is not such build with enabled tests. Those 8 failed tests failed with %set_build_flags. The question is what is the best solution if we want Kakoune in official repos and more users will use it:

Disable all test and build with proper build flags to conform Fedora guidelines.
Fix something that will allow build with tests and this %set_build_flags macros.

Don't get me wrong, just wanted to increase popularity of app and attract more users/devs. Copr is similar to PPA on Ubuntu and many user prefer to avoid it. Just asked for some help from more experienced devs/maintainers ¯\_(ツ)_/¯

tim77 on 25 Nov 2019

I've traced the issue back to the -Wp,-D_GLIBCXX_ASSERTIONS compilation flag. It's set when using %set_build_flags, and adding it to CXXFLAGS even on master reproduces the issue.

diff --git a/src/Makefile b/src/Makefile
index 68e8c7c3..94852cbe 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -88,7 +88,7 @@ else
     LDFLAGS += -rdynamic
 endif

-CXXFLAGS += -pedantic -std=c++17 -g -Wall -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-address
+CXXFLAGS += -Wp,-D_GLIBCXX_ASSERTIONS -pedantic -std=c++17 -g -Wall -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-address

 compiler := $(shell $(CXX) --version)
 ifneq (,$(findstring clang,$(compiler)))

Using the | primitive in an interactive session yields the following backtrace:

#0  0x00007ffff7a3ff25 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff7a29897 in abort () from /usr/lib/libc.so.6
#2  0x000055555597d19e in std::__replacement_assert (__file=0x555555cde1f0 "/usr/include/c++/9.2.0/bits/stl_vector.h", __line=1060, __function=0x555555cde6e0 "std::vector<_Tp, _Alloc>::const_reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) const [with _Tp = Kakoune::StringView; _Alloc = Kakoune::Allocator<Kakoune::StringVi"..., __condition=0x555555cddfb8 "__builtin_expect(__n < this->size(), true)") at /usr/include/c++/9.2.0/x86_64-pc-linux-gnu/bits/c++config.h:463
#3  0x0000555555bb80e5 in std::vector<Kakoune::StringView, Kakoune::Allocator<Kakoune::StringView, (Kakoune::MemoryDomain)0> >::operator[] (this=0x7fffffffc800, __n=1) at /usr/include/c++/9.2.0/bits/stl_vector.h:1060
#4  0x0000555555ba9570 in Kakoune::<lambda(auto:21&&, int, int)>::operator()<const std::vector<Kakoune::StringView, Kakoune::Allocator<Kakoune::StringView, (Kakoune::MemoryDomain)0> >&>(const std::vector<Kakoune::StringView, Kakoune::Allocator<Kakoune::StringView, (Kakoune::MemoryDomain)0> > &, int, int) const (__closure=0x7fffffffc78b, lines=std::vector of length 1, capacity 1 = {...}, first=0, count=1) at normal.cc:515
#5  0x0000555555ba978b in Kakoune::apply_diff (buffer=..., pos=Kakoune::BufferCoord(0, 0), before="*", after="*") at normal.cc:525
#6  0x0000555555bc3ac8 in Kakoune::pipe<true>(Kakoune::Context&, Kakoune::NormalParams)::{lambda(Kakoune::StringView, Kakoune::PromptEvent, Kakoune::Context&)#1}::operator()(Kakoune::StringView, Kakoune::PromptEvent, Kakoune::Context&) const (this=0x555555f12e28, cmdline="cat", event=Kakoune::PromptEvent::Validate, context=...) at normal.cc:593
#7  0x0000555555bfa889 in std::_Function_handler<void (Kakoune::StringView, Kakoune::PromptEvent, Kakoune::Context&), Kakoune::pipe<true>(Kakoune::Context&, Kakoune::NormalParams)::{lambda(Kakoune::StringView, Kakoune::PromptEvent, Kakoune::Context&)#1}>::_M_invoke(std::_Any_data const&, Kakoune::StringView&&, Kakoune::PromptEvent&&, Kakoune::Context&) (__functor=..., __args#0="cat", __args#1=@0x7fffffffcc84: Kakoune::PromptEvent::Validate, __args#2=...) at /usr/include/c++/9.2.0/bits/std_function.h:300
#8  0x0000555555b13420 in std::function<void (Kakoune::StringView, Kakoune::PromptEvent, Kakoune::Context&)>::operator()(Kakoune::StringView, Kakoune::PromptEvent, Kakoune::Context&) const (this=0x555555f12e28, __args#0="cat", __args#1=Kakoune::PromptEvent::Validate, __args#2=...) at /usr/include/c++/9.2.0/bits/std_function.h:690
#9  0x0000555555b09bee in Kakoune::InputModes::Prompt::on_key (this=0x555555f12e10, key=...) at input_handler.cc:790
#10 0x0000555555b02ac1 in Kakoune::InputMode::handle_key (this=0x555555f12e10, key=...) at input_handler.cc:32
#11 0x0000555555b00b39 in Kakoune::InputHandler::<lambda(Kakoune::Key)>::operator()(Kakoune::Key) const (__closure=0x7fffffffce60, key=...) at input_handler.cc:1647
#12 0x0000555555b00d3f in Kakoune::InputHandler::handle_key (this=0x555555e67ed8, key=...) at input_handler.cc:1659
#13 0x00005555559b2da6 in Kakoune::Client::process_pending_inputs (this=0x555555e67e60) at client.cc:98
#14 0x00005555559c3627 in Kakoune::ClientManager::process_pending_inputs (this=0x7fffffffd300) at client_manager.cc:103
#15 0x0000555555b5f396 in Kakoune::run_server (session="", server_init="", client_init="", init_coord=..., flags=Kakoune::ServerFlags::StartupInfo, ui_type=Kakoune::UIType::NCurses, debug_flags=Kakoune::DebugFlags::None, files=ArrayView<Kakoune::StringView>) at main.cc:784
#16 0x0000555555b6373f in main (argc=1, argv=0x7fffffffe6b8) at main.cc:1125

To be continued…

lenormf on 25 Nov 2019

❤1 👍1

The above issue seems to be fixed with the following patch:

diff --git a/src/normal.cc b/src/normal.cc
index 1b287e5f..1123b916 100644
--- a/src/normal.cc
+++ b/src/normal.cc
@@ -512,7 +512,7 @@ BufferCoord apply_diff(Buffer& buffer, BufferCoord pos, StringView before, Strin
                            lines_after.begin(), (int)lines_after.size());

     auto byte_count = [](auto&& lines, int first, int count) {
-        return std::accumulate(&lines[first], &lines[first+count], 0_byte,
+        return std::accumulate(lines.begin()+first, lines.begin()+first+count, 0_byte,
                                [](ByteCount l, StringView s) { return l + s.length(); });
     };

lenormf on 25 Nov 2019

However there's different issue, with the pipe-to and reload tests:

#0  0x00007ffff7a3ff25 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff7a29897 in abort () from /usr/lib/libc.so.6
#2  0x000055555597e19e in std::__replacement_assert (__file=0x555555ca8638 "/usr/include/c++/9.2.0/bits/st l_vector.h", __line=1042, __function=0x555555ca8448 "std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) [with _Tp = Kakoune::RefPtr<Kakoune::StringData, Kakoune::StringData::PtrPolicy>; _Alloc ="..., __condition=0x555555ca8418 "__builtin_expect(__n < this->si ze(), true)") at /usr/include/c++/9.2.0/x86_64-pc-linux-gnu/bits/c++config.h:463
#3  0x000055555599828d in std::vector<Kakoune::RefPtr<Kakoune::StringData, Kakoune::StringData::PtrPolicy> , Kakoune::Allocator<Kakoune::RefPtr<Kakoune::StringData, Kakoune::StringData::PtrPolicy>, (Kakoune::Memor yDomain)3> >::operator[] (this=0x7fffffffc970, __n=1) at /usr/include/c++/9.2.0/bits/stl_vector.h:1042
#4  0x0000555555986d2b in Kakoune::Buffer::reload (this=0x555555fea400, data="tchou ", fs_timestamp=...) at buffer.cc:285
#5  0x00005555559ad5a6 in Kakoune::reload_file_buffer (buffer=...) at buffer_utils.cc:94
#6  0x00005555559b5c83 in Kakoune::Client::reload_buffer (this=0x555555e68e60) at client.cc:280
#7  0x00005555559b623f in Kakoune::Client::on_buffer_reload_key (this=0x555555e68e60, key=...) at client.c c:308
#8  0x00005555559b684c in Kakoune::Client::<lambda(Kakoune::Key, Kakoune::Context&)>::operator()(Kakoune:: Key, Kakoune::Context &) const (__closure=0x555555fe5f30, key=...) at client.cc:370
#9  0x00005555559b83b9 in std::_Function_handler<void(Kakoune::Key, Kakoune::Context&), Kakoune::Client::c heck_if_buffer_needs_reloading()::<lambda(Kakoune::Key, Kakoune::Context&)> >::_M_invoke(const std::_Any_d ata &, Kakoune::Key &&, Kakoune::Context &) (__functor=..., __args#0=..., __args#1=...) at /usr/include/c+ +/9.2.0/bits/std_function.h:300
#10 0x0000555555b14e87 in std::function<void (Kakoune::Key, Kakoune::Context&)>::operator()(Kakoune::Key, Kakoune::Context&) const (this=0x555555fe5f30, __args#0=..., __args#1=...) at /usr/include/c++/9.2.0/bits/ std_function.h:690
#11 0x0000555555b0d1af in Kakoune::InputModes::NextKey::on_key (this=0x555555fe5f00, key=...) at input_han dler.cc:1112
#12 0x0000555555b03ac1 in Kakoune::InputMode::handle_key (this=0x555555fe5f00, key=...) at input_handler.c c:32
#13 0x0000555555b01b39 in Kakoune::InputHandler::<lambda(Kakoune::Key)>::operator()(Kakoune::Key) const (_ _closure=0x7fffffffce60, key=...) at input_handler.cc:1647
#14 0x0000555555b01d3f in Kakoune::InputHandler::handle_key (this=0x555555e68ed8, key=...) at input_handle r.cc:1659
#15 0x00005555559b3da6 in Kakoune::Client::process_pending_inputs (this=0x555555e68e60) at client.cc:98
#16 0x00005555559c4627 in Kakoune::ClientManager::process_pending_inputs (this=0x7fffffffd300) at client_m anager.cc:103
#17 0x0000555555b60396 in Kakoune::run_server (session="", server_init="", client_init="", init_coord=..., flags=Kakoune::ServerFlags::StartupInfo, ui_type=Kakoune::UIType::NCurses, debug_flags=Kakoune::DebugFlag s::None, files=ArrayView<Kakoune::StringView>) at main.cc:784
#18 0x0000555555b6473f in main (argc=1, argv=0x7fffffffe6b8) at main.cc:1125

EDIT: seems to be the same problem, fixed with the following:

diff --git a/src/buffer.cc b/src/buffer.cc
index 6088c8a8..92351e5c 100644
--- a/src/buffer.cc
+++ b/src/buffer.cc
@@ -282,7 +282,7 @@ void Buffer::reload(StringView data, timespec fs_timestamp)
                         parsed_lines.lines[(int)(d.posB + line)]});

                 m_changes.push_back({ Change::Insert, cur_line, cur_line + d.len });
-                m_lines.insert(it, &parsed_lines.lines[d.posB], &parsed_lines.lines[d.posB + d.len]);
+                m_lines.insert(it, parsed_lines.lines.begin()+d.posB, parsed_lines.lines.begin()+d.posB+d.len);
                 it = m_lines.begin() + (int)(cur_line + d.len);
             }
             else if (d.mode == Diff::Remove)

lenormf on 25 Nov 2019

@lenormf you must be, and have to be, the man with the master plan.

So -D_GLIBCXX_ASSERTIONS. This turns on cheap range checks for C++ arrays, vectors, and strings. was the key to get to this error.

But I thought @tim77 compiled it earlier with the GLIBCXX_ASSERTIONS or so this is what the log from 2019-Nov-24 23:11:59 says.

just out of curiosity, if the bug was closed with Fedora 28 which reached the end of cycle, why is there a need to implement GLIBCXX_ASSERTIONS at all? At any rate, every time a see the word 'cheap', i have to disassociate it with a good thing.

nonumeros on 25 Nov 2019

With https://github.com/mawww/kakoune/pull/3227 everything compiles, works and all tests passed! Now waiting for review...

tim77 on 26 Nov 2019

Note that the PR might be rejected, and implemented more elegantly - even though in effect, the problem was identified. So stick around to grab the official fix, once it's been merged.

I also noticed that the URL to the patch is:
https://patch-diff.githubusercontent.com/raw/mawww/kakoune/pull/3227.diff

Are you not using Github directly on purpose? You can get a patch out of a PR directly like so:
https://github.com/mawww/kakoune/pull/3227.patch

HTH.

lenormf on 26 Nov 2019

👀1 👍1

Kakoune packaged and available now in official Fedora repos and RHEL/CentOS 8. Thank you all! Special thanks to @lenormf

tim77 on 29 Nov 2019

Yay!

lenormf on 29 Nov 2019

❤3 😄1

Hi everyone,

Sorry I did not replied sooner. I have a newborn son so I wasn't able to react last month or so.

@tim77 great work! I wanted to add Kakoune into the official repositories for a long time but I wasn't able to find time for that :(. It's great you've done this! If you're offer I would love to help you with the packaging. I also see that you updated the package to the newest version :+1:

If you agree, I would use my existing COPR repository as daily builds so we can catch bugs sooner and your version will be the stable one using released versions. I will then update my spec file based on your changes.

jkonecny12 on 18 Dec 2019

If you're offer I would love to help you with the packaging. I also see that you updated the package to the newest version +1

Sure. Added.

If you agree, I would use my existing COPR repository as daily builds so we can catch bugs sooner and your version will be the stable one using released versions. I will then update my spec file based on your changes.

Sure.

tim77 on 18 Dec 2019

Kakoune: Tests not passed with default Fedora build flags

Most helpful comment

All 21 comments

Related issues